Posted by Richard Allen[1]
A new book by Matt Andrews and two co-authors on “Building State Capability: Evidence, Analysis, Action”[2] discusses the challenges of reforming systems of public administration and PFM in developing countries. A core message of the book is that reformers should focus much more on the political and institutional constraints to introducing new policies or projects, rather than (only) on the technical aspects of reform.
Many countries and donor organizations have promoted reforms that look like the “international best practices” of modern states, but such reforms have frequently been unsuccessful. The authors propose a different approach which emphasizes locally customized “best fit” solutions, combined with experimentation and a broad-based dialog among the development partners and other actors in the reform process. They give this approach the ungainly title of “problem-driven iterative adaptation” (PDIA), which originates from earlier writings by Andrews. The PDIA methodology has been presented by the authors in courses at Harvard University’s Center for International Development.
The book provides an interesting cross-country analysis of state capability drawing on three primary sources of data[3]. It analyses the performance of 102 developing countries and emerging markets. The authors conclude that only eight of these countries – most of which are small or oil-rich states - have attained a strong capability, and 49 countries have a very weak or weak capability. Moreover, 36 countries, some of which are fragile or failing states, have experienced a negative growth in state capability in recent decades.
The authors argue that “isomorphic mimicry” – the copying of systems and procedures used in more advanced countries - is the main explanation why capability traps are created and perpetuated in developing countries. Isomorphic mimicry tends to conflate form and function. For example, passing a procurement law is counted a success even if weak enforcement means that it fails to increase competition in the supply of goods and services, and reduce costs. Unfortunately, countries are often pressured by donors into focusing on inputs rather than outputs, and on compliance with processes, rather than improving the delivery of public services.
The PDIA approach comprises four distinct “principles of engagement’: (i) focusing on specific problems in local contexts; (ii) fostering active experimental iterations with new ideas, gathering lessons from these iterations to turn ideas into solutions; (iii) establishing an environment for decision-making that encourages experimentation; and (iv) engaging broad sets of agents to ensure that reforms are viable, legitimate, relevant, politically supportable, and practically implementable. The book describes some novel techniques and analytical tools that are designed to support the PDIA approach, but which may be difficult to apply in low-income countries with poor access to data and analytical skills[4].
By contrast, most existing approaches to reform focus on pre-specifying solutions, locking implementation plans in place through rigid logical framework mechanisms, and relying heavily on individual reform champions.
The book, however, leaves some important questions unanswered.
First, it offers very limited evidence that the PDIA approach leads to better results in practice. The authors refer on several occasions to the reforms of the Swedish budget system in the 1990s that followed a deep financial crisis. But Sweden is not a good example. It is an advanced country with unusually high levels of governance, well-established accountability mechanisms, and strong cooperation and coordination among the various budgetary actors and the legislature. These attributes are lacking in most developing countries.
Second, the distinction between “problems” and “solutions” – which is fundamental to the logic of PDIA - is not as clear cut as the authors suggest. For example, countries may attempt to deal with problems of arrears and poor financial reporting by setting up a treasury single account and a financial management information systems (FMIS). But these “solutions” are usually based on a standard set of design options, with some tailoring allowed for specific country contexts. Similarly, the book cites the allegedly successful reform of internal audit in Burkina Faso. Internal audit, however, is a function in which the “solutions” are largely pre-specified by international norms and standards set by the Institute of Internal Auditors (IIA), thus undermining one of the basic principles of PDIA. In practice, countries have only limited room for choice.
Third, key terms such as “would be reformers” and “change agents” are not clearly defined, and some of the book’s core ideas appear not to have been fully thought through. For example, the authors create the intriguing notion of a kind of séance of policymakers who “do not really know what the solution is … or what surprises they will encounter” in proposing new reform
initiatives. These methods are reminiscent of the “deep dive” approaches used in Australia and elsewhere, where interest groups and decision-makers were brought together to discuss complex policy issues. But how realistic and practicable are they in low-income countries?
Fourth, the authors may overstate the importance and practical relevance of PDIA as a development tool. PDIA assumes unlimited degrees of freedom for experimentation and plentiful time and resources to make it work. In practice, developing countries do not have an abundance of either of these attributes. Some of the technical tools proposed by the authors could be challenging to implement in low-capacity environments: for example, the “log frame-type mechanism that embeds experimental iteration into a structured approach to make policy or reform decisions in the face of complex challenges”.
Finally, PDIA is essentially a technological response to a set of challenges that are fundamentally political in nature, a dilemma that the book fails to answer. Indeed, some PFM challenges may be so large and intractable that practical solutions in low-capability states are impossible to contemplate within a foreseeable future. Perhaps PDIA would be more useful in tackling a subset of smaller issues, in which the scope for open and interactive debate on potential solutions is more manageable, and the political influences less strong? A set of pilot studies of the PDIA approach in African countries by the Collaborative Africa Budget Reform Initiative (CABRI) may help provide answers to some of these questions. Until then, the jury remains out on the usefulness and effectiveness of PDIA as a development tool.
While PDIA is unlikely to replace conventional approaches to implementing reform, some of the techniques and tools described in the book could usefully complement these approaches. Nevertheless, a more balanced assessment of the strengths and weaknesses of PDIA - carried out by analysts who have not been directly involved in designing the new approach and have no vested interest in promoting it - would be welcome.
[1] Richard Allen is a Visiting Scholar with the International Monetary Fund. A longer version of this review will be published in the journal Governance in May 2018.
[2] Matt Andrews, Lant Pritchett, Michael Woolcock, 2017, Building State Capability: Evidence, Analysis, Action, Oxford: Oxford University Press.
[3] The Quality of Governance (QOG) Institute, the Failed State Index (FSI), and World Governance Indicators (WGI).
[4] These tools include the use of “fishbone” (Ishikawa) diagrams to deconstruct complex problems into their components; “triple-A” space change analysis; search frames to conduct experimental iteration; tools to describe and manage the authorizing environment; and the establishment of user groups to help build capacity.
Note: The posts on the IMF PFM Blog should not be reported as representing the views of the IMF. The views expressed are those of the authors and do not necessarily represent those of the IMF or IMF policy.
