« Timor Leste in Full View | Main | Elephants, Blind Men, and the Problems of Specialization in PFM »

November 29, 2010

Where and Why do PFM Reform Efforts Succeed?

Posted by Joanne Asquith

Over the last decade or so, donor support for activities in public sector financial management (PFM) has grown more than ten fold, from US$85.1 million in 1995 to US$930.6 million in 2007.  Nevertheless, surprisingly little evidence and analysis exists on the comparative performance of PFM systems across countries and over time, on the factors that underpin successful PFM reforms, and the role that donor agencies play in the reform process.

To help to fill this gap, an analytical study of quantitative cross-country evidence on public finance management in developing countries was undertaken by a team of researchers at the Overseas Development Institute. The study is part of a broader evaluation of PFM reforms in developing countries initiated by the evaluation departments of DANIDA, SIDA, DFID and the AFDB.

A number of findings from the cross-country analysis are of relevance to donor approaches and policies on PFM reforms:

  • Economic factors are most important in explaining differences in the quality of PFM systems. Aid-related factors, on the other hand, have more limited explanatory power. As a consequence, PFM systems are more likely to improve responding to changing economic circumstances, rather than to donor efforts.
  • More specifically, countries with higher levels of per capita income, with larger populations and with a better recent economic growth record are characterised by better quality PFM systems. On the other hand, state fragility, defined as being in a conflict or post-conflict situation, has a negative effect on the quality of PFM systems.
  • The analysis finds a positive and significant, albeit weak, correlation between donor support to PFM reforms and improvements in PFM systems and, on average, countries that received more PFM-related technical assistance have better PFM systems.  However, these average effects cannot be taken as causal and universal, and need to be further investigated.
  • The results remain consistent through a number of robustness checks and model changes. Interesting additional results come from using more recent data or focusing on low-income countries only. In these cases, the share of total aid provided as general budget support is also positively and significantly associated with better PFM quality. In other words aid modalities, and not just direct support to PFM reforms, are seen to affect the quality of PFM systems in some of the poorer countries where most donor efforts are concentrated.
  • Finally, various aspects of donor support differ in their relationship with more specific PFM processes. A longer period of donor engagement, for example, is associated with better performance in upstream and organizationally central processes. This may be due to donors’ historical tendency to pay more attention to these simpler reform areas, but could also reflect the fact that downstream, and organizationally deconcentrated processes take longer to improve.
  • The level of donor PFM support is also strongly associated with centralised PFM processes highlighting how donor PFM support seems to focus more on rules, procedures and specific actors within government.  

At the same time, these results suffer from a number of serious limitations and challenges, including the following:

  • Data quality remains an issue, especially with regard to information about PFM support from donors. 
  • The direction of causality cannot be proved; so the results could reflect the fact that donors tend to provide more PFM-related assistance (and more general budget support) to countries that have already achieved a certain success in improving the quality of their PFM systems.
  • Limited time series data on the quality of PFM systems.

These limitations and challenges point to the need to interpret the results of the analysis presented in the paper with caution. Moreover, they highlight the need to complement the quantitative findings with in-depth qualitative research at country level, explaining not only if and when donor PFM support has had an impact on PFM systems, but also why and how it did.  

The next stage of the evaluation is a series of country case studies starting shortly in Burkina Faso, Malawi and Ghana.   

Download PFM Quant Study

Note: The posts on the IMF PFM Blog should not be reported as representing the views of the IMF. The views expressed are those of the authors and do not necessarily represent those of the IMF or IMF policy.

TrackBack

TrackBack URL for this entry:
https://www.typepad.com/services/trackback/6a00e54ef0059588340147e03cf243970b

Listed below are links to weblogs that reference Where and Why do PFM Reform Efforts Succeed?:

Comments

Thanks Joanne,
A useful report. On the issue of the direction of causality and the fact that, as the report states “the association is very weak: an additional 40-50 million US$ per year would correspond to a
half-point increase in the average PEFA score (equivalent to, say, a change from C to C+).” This leads to two conclusions- donor support goes to countries that perform well and are perceived as using existing TA effectively- thus countries perceived as “difficult” will get less support over time. The second is that donor concentration on downstream processes like IFMIS highlights the fundamental point that donor support is at best a catalyst and not the prime mover to reform- this must come from the countries themselves. The donor’s historical tendency to pay attention to simpler reform areas, and the weak association mentioned above, highlights the point often made, that long term engagement and not a focus on “quick wins” would be a more sustainable way to greater reform.
As I am sure the proposed study in Ghana would show, this was a country which, between 2001-2004 showed the greatest improvement, measured under a HIPIC assessment framework, anywhere in the world then. Some of it was due to donor TA (mainly IMF) but a lot of it had to do with the economic factors which propelled the country at that time. Last, and not the least, it was the people who were at the helm of affairs then and pushed the process that made a difference too, and a significant one. So while empirical studies may seemingly discount this factor, it is and remains a crucial one in any PFM reform program.

Good paper. Two points:
1) Simple averaging of PEFA scores doesn't properly account for relative importance of indicators to systemic outcomes. I would argue that use of quantitative assessments of fiduciary and development risks based on PEFA type scoring better captures relative importance and systemic issues.
2) Absorptive capacity is not specifically addressed. While aid and resource dependencies are included as variables, these are inadequate measures for quantifying the different dimensions of absorptive capacity (macroeconomic, microeconomic and institutional). One would expect covariance problems without better quantification approach (e.g. aid and resource dependency is theoretically related with GDP growth and GDP per capita when there is evidence of a resource curse phenomenon). Moreover, evidence for the different theories (e.g. macroeconomic v’s governance) and forms (e.g. aid-induced, single industry and conventional resource curses) of common pool problems are important to consider when assessing PFM system effectiveness.
PEFA was not really designed for inter-country comparisons for point in time or over time. While PEFA does provide a good basis to quantify systemic fiduciary and development risks and support estimation of absorptive capacity space, various enhancements and extensions would make it a more powerful tool for longitudinal studies (including cost-effectiveness analysis of PFM assistance). For example, the level of budget fragmentation and quality of medium-term budgeting are important determinants of inherent development and fiduciary risks. Standard PEFA is not able to draw out the subtleties required to search for stronger relationships between explanatory variables and different measures of PFM system quality. More detail is unfortunately required in this area (e.g. along the lines of an MTEF Development Rating System as proposed by Le Houerou and Taliercio back in 2002).

Interesting study. The poor performance of upstream strategic budgeting was surprising... or perhaps not. The conflict/post conflict correlation was as expected, but i was surprised that the wider issue of political stability or longevity was not tested as one of the other independent or control variables. My experience is that rapid turnover of finance ministers and permanent secretaries/CEOs is devastating for PFM reform. A committed minister and permanent secretary/CEO will need 5 to 10 uninterrupted years to deliver on their PFM reforms.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Back to top of page
©2007 IMF. All Rights Reserved. About Us | Terms of Use