Posted by Jack Diamond
While international organizations can give excellent advice on the direction of PFM reform, i.e. why going from “bad” situation A to “good” situation B will provide important benefits to the country, advice on the change processes necessary for moving from A to B is much more fuzzy and more often than not subject to failure. Jack Diamond, former division chief in the Fiscal Affairs Department, recently gave a presentation (The Soft Side of PFM Reform is the Hardest) at IMF HQ on what the PFM profession knows about PFM reform processes, what determines their success and failure, and how this should change our approach to reform implementation.
One of the advantages of having spent a large part of your working life providing TA in the PFM area is that it often provides the opportunity to visit the same countries over a period of time. The downside is that despite the amount of TA having expanded exponentially, and despite the occasional success stories, one is often disappointed to find how little has changed or how long change has taken to implement. In some cases it even appears that countries have gone backwards. Sadly, too often one ends up re-iterating the same advice that was given years before, and then perhaps resigning oneself to the same disappointing results. This leads me to the motivating question of my presentation:
1. Can we do a better job in providing advice on PFM Reform?Certainly, in the reviewed literature on PFM TA there is no shortage of explanations for failure, among them:
- TA advice is unrealistic in its scope. There are evident cases of overloading of the agenda. How often have we visited countries with PEFA Reform Action Plans made up of a long list of actions required to get all performance indicators to an A level, without any prioritization? It is not that any of the actions are wrong or unnecessary by themselves, it is just there is a lack of prioritization and sequence.
- TA advice is unrealistic in the time envisaged for reforms. Indeed, there may be a systematic underestimation of the time required, especially when reforms are a component of conditionality for accessing loans with fixed deadlines. There is also a tendency to be “optimistically realistic” about timing—setting tight deadlines just to get the authorities moving faster, or moving at all. Part of this may also be in recognition that given their short-term horizons politicians are often not interested in reforms that take years to complete.
- TA advice is not properly geared to country needs. Certainly, there is some evidence of donors pushing their own solutions based on financing availability, and a “one size fits all” approach in advancing standardized “reform packages”. No surprise here--this is the standard modus operandi of loan officers in all fields of donor assistance.
- TA advice is given and financed without real political commitment. It is argued that politicians do not really support reform but rather are lured by the financing that comes with it. One cannot blame the politicians. PFM TA has become a crowded field, and more and more donors are willing to fund almost all aspects of PFM reform. In this cornucopia politicians can afford to cherry pick the most lucrative reforms, and afford to be greedy, taking on more reforms than their administrative capacity can handle.
While accepting the validity of some or all the above reasons for TA failure, it is difficult not to suspect that there is something more fundamentally wrong with the approach to PFM reform. The argument that I would propose is that perhaps we have concentrated too much on the “hard”, or technical side of reform, and not spent enough effort developing the “soft” side.
2. What do we mean with the “soft” side of PFM reform?
The hard side of PFM reform—the purely technical aspect—is usually not difficult to specify, even though it turns out difficult to implement. Our TA often stops at this hard side: we diagnose the technical problem, prescribe the technical solution, and recommend steps to implement this technical solution—and then move on. What have we missed? Firstly, to be successful reforms need to be customized to fit specific country circumstances: this implies there can be no one standard technical solution. Secondly, and related to the first point, we should recognize that non-technical or “external” factors should influence how we design reforms. Experience has shown how adverse external factors—political instability, macroeconomic stress, an unsympathetic minister of finance, etc—can sink good technical solutions. Thirdly, whether a reform is successful or otherwise often depends critically on how it is introduced and sustained—change management cannot be ignored and every technical solution needs some social engineering. However, too often this aspect of how to put in place a sustainable delivery mechanism for the proposed reform is not adequately addressed.
These then are the main dimensions that constitute the “soft” side of PFM reform, namely:
- How we tailor reform to specific country circumstances. Often we find, especially in low income countries, that PFM weaknesses abound, so the potential reform agenda is quite wide. However, not all weaknesses can be addressed simultaneously, so the choice of priorities has to be country specific. These priorities should be determined to ensure countries own the reforms, also taking into consideration different capacity constraints and hence the feasible pace of reforms.
- How we take into account external factors when we prescribe reforms. These factors are specific to a country and can be analyzed at three levels. (See the attached paper distributed at the seminar: Non-Technical Context of PFM Reform and its annex table). At the highest level are the general conditioning factors determining the political, economic, socio-cultural and technological environment of the host country. In the middle, are the factors that arise from the institutional design of the PFM system that affects its capacity to accept and implement reform. At the lowest level are factors that arise from the internal structure of PFM organizations that are directly affected by the reform—the managerial culture, the leadership, skills and other capacity constraints. All these factors should be analyzed to assess their impact on recommended reforms that, in turn, should be re-designed in light of this analysis.
- How we put in place a sustainable implementation strategy. Among the elements of this strategy are adequate change management arrangements, measures to ensure critical constraints are removed and resources are in place, and last but not least, ensuring local participation, ownership, and pressure for the reform. (Many of the latter issues were covered in other seminar presentations, and are discussed in my companion paper but will not be addressed here).
3. What does recognizing the soft side of PFM imply about how we view PFM advice?
First and foremost it implies that deciding on viable reform options should never be viewed purely as a technical matter, but rather we should recognize a trade-off between:
- What needs to be done. This is the “hard” technical side of PFM advice, an area where TA is most comfortable.
- What can be done. This requires taking into account the many external factors that impose constraints (and may offer opportunities) that are specific to each country.
- What is wanted. This requires taking into account the local political demands concerning reform priorities.
Clearly, PFM TA has not entirely ignored the latter two dimensions, but it can be questioned if it has got the balance right between the three dimensions. For example, should the authorities’ demands always dominate? Providers of TA too readily argue the advice they give is in direct response to what the authorities have asked for, and this they say it is as it should be since it will ensure maximum political commitment. But the argument can be overdone. What if the politicians’ choice is over-ambitious? What if the choice is motivated more by politics or rent-seeking rather than reform impact? What if it leads to a bias in PFM reforms, towards those that are highly visible or fast to implement (because politicians’ time horizons may be short)? What if the reform is associated with one prominent individual who someday in the future might disappear? The latter considerations suggest that what is wanted needs to be balanced by what technically should be done and what can be done.
It is also apparent that TA advice has tried to modify purely technically determined advice with the practical realities confronted on the ground. Many of the typical reform strategies are attempts to resolve this trade-off between the technically required and the practically possible. For example, the strategic approach is advocated, where reform actions are chosen to most easily show success, even though this “low-lying” fruit may not be the most productive in terms of PFM reform. Or choosing the weakest link first—identify bottlenecks and remove them first. However, the way in which these bottlenecks are identified—technically? or, by their feasibility? or, by how much they are locally desired?—allows room for resolving the above trade-offs, but usually in a non-transparent way. Another approach is to consider the time required—choosing the reform activity to match the country’s feasible time horizon for successful implementation—i.e. emphasizing what can be done in a given “window of opportunity”. Unfortunately, experience shows that reforms nearly always underestimate the time required, so the strategy is based on unsure ground. An increasingly popular strategy, the platform approach, is an amalgam of these various strategies. It chooses a cluster of reform actions for each program that are linked so as to resolve bottlenecks, it tends to choose low lying fruit for the first platform to build on success, and it phases in reforms stepwise with a view to the time required for implementation, with rolling plans to add further flexibility.
What seems lacking in the continuing debate about best strategy is the failure to acknowledge that very rarely in PFM reforms will all three dimensions—what needs to be done, what should be done, and what is demanded--be perfectly correlated. Indeed, often the best we can hope for is some degree of overlap that allows enough scope to frame meaningful TA recommendations. However, sometimes we confront extreme misalignment—for example, the authorities demand reforms that we might feel are not in priority areas nor should be attempted due to existing constraints. Then what to do? Other times, what needs to be done is clear, and the authorities are convinced and support such change, but realistically given the present constraints of the country the technical solution does not seem feasible at this time. Recognizing the “soft” side of PFM TA requires working with the authorities not only to find a reasonable balance between, but also to increase the degree of overlap in, the three dimensions. Coming to this consensus is usually recognized as the first step of a change management strategy. Having said all that leads me to my final question.
4. How should we modify our TA advice in light of the above?
First, it is suggested we should spend more time analyzing the external factors that are likely to impact the specific PFM reforms being contemplated. How this analysis might be framed, and how operationalized, is dealt with more fully in the attached paper. These external factors are important to the TA advice we give because of the risks they pose to successful reform. The process envisaged is designed to assess key non-technical factors with regard to their risk impact, and come to some agreement on the aggregate degree of risk posed in a particular country context. This need be no more sophisticated that dividing countries into high, medium, and low risk categories.
Second, it is also necessary to recognize that different external factors will affect a given reform action differently, so that risk will depend on the type of reform. It is also necessary, therefore, to classify reforms by their degree of risk. Identified within the PFM literature are a number of dimensions regarded as affecting the degree of risk inherent in specific reforms, namely:
- Scope of reform: the number of reform actions, their degree of complexity, and their interconnectedness all increase risk of failure.
- Time required to complete the reform actions: the longer the time the greater the danger of things going wrong.
- Number of institutions involved in the reform: the fewer the institutions (“concentrated reforms”), the lower the risk, and vice versa, (“de-concentrated” reforms involve higher risk).
- Degree of change implied: some reforms involve changing procedures (de jure) others are more fundamental and involve behavioral change (de facto), which are more difficult and hence imply more risk of failure.
Using these dimensions it should be possible to come to an assessment of the risk posed by each proposed reform action. Again this may be simply categorized as high, medium, and low.
It is then recommended that a risk-based approach to reform be adopted to better tailor TA to the specific country context. The guiding principle should be that a high risk reform should only be attempted in a low risk environment. Applying this principle, how do previously used TA strategies match up? The strategic or “low-lying fruit” approach, typically covers reforms narrow in scope, de jure, concentrated, and implementable relatively fast. This is a low risk strategy suitable for a high risk country. The weakest link first approach really depends on how bottlenecks arise and how they are identified. However, in a high risk country this approach should be relied on only if the bottleneck can be resolved with narrow, concentrated, and time-efficient actions. The platform approach is usually wide in scope, includes both de jure and de facto reforms, typically in a de-concentrated way over a longer period of time. Therefore, using this risk-based assessment, this approach should only be recommended for low risk countries. Basing reform on what is locally demanded would depend critically on a country’s risk profile. This implies accepting locally demanded narrow, fast implemented, de jure reforms for high risk countries, but accepting wider, more protracted, de facto reforms for low risk countries.
Apart from suggesting how TA can be better matched to a specific country context, this risk-based approach could also guide specific TA advice. Hopefully, this should help avoid previous criticisms of TA being unrealistic in scope, under-estimating the time involved, and failing to ensure and local commitment. The above approach highlights an important trade-off: that between risk and total reform impact. The greater the scope of reforms contemplated, the more time implied, the greater the number of institutions involved, and the higher the degree of behavior change implied, then the higher is the reform impact but also the higher the risk.
This suggests a reorientation in our TA advice. To lower the risk of failure TA advice should be focused more narrowly on fewer reform actions (in contrast to past over ambitious approaches that have overloaded reform programs). Similarly, long-haul programs should be avoided for the majority of countries, or broken down into shorter more manageable components. The timing of each component should then be made more realistic by more detailed advice on how to introduce, sequence, and sustain reform actions. Similarly, the more reform action focuses on those reforms requiring significant behavioral change, or a large number of institutions, the more detailed the advice should be on the modalities of managing the change process. Perhaps more controversially it also suggests when not to give advice. If the analysis of risk arising from external factors exceeds tolerable levels, one should be brave enough not to offer TA advice, allow a pause in the reforms, and work with donors and the authorities to address the major risks perceived. This too, should be regarded as part of the “soft side” of PFM reform.
