Posted by Bill Dorotinsky
In an April 21, 2008, post, we described the origins of the PEFA framework, with a promise to blog about the framework itself. This post provides a brief overview of the assessment framework and indicator set.
The authoritative reference for the PEFA indicators is the PEFA Public Financial Management Performance Measurement Framework handbook, available in English, French, Spanish, Portuguese (Brazilian and Continental), Russian, Ukrainian, and Arabic.
What the framework is and is not
The PEFA assessment framework consists of 28 high-level performance indicators of a country's public financial management (PFM) system, with 3 supplemental indicators on donor practices that impact a country's public financial management system (click on the image to the right for a full-size table of the 28 indicators). The PEFA framework was developed and tested over several years, drawing on the previous indicator sets and other sources.
The PEFA Framework was designed specifically for central government, but it has been applied at subnational level with minor modifications (see Guidelines for application of the PEFA PMF at Sub National Government level).
The performance indicators measure high level system performance. That is, looking at how well the assessed area is performing, as objectively as possible. As with any performance measurement system, when outcome or result cannot be assessed, points earlier in the 'impact chain' are measured, which might entail processes or system features. The indicators provide an over-view of performance, but would not necessarily provide the reasons why performance was weak (one would know why the score was assessed at a 'c', for example, but not perhaps whether this was due to human capital, IT systems, management, legislation or policy).
The pedigree of the indicator set is noteworthy. It builds on the aforementioned HIPC indicators, and also draws on numerous resources such as the IMF’s Fiscal Standards and Codes (ROSC) and internationally-accepted standards such as the IMF Government Finance Statistics, International Public Sector Accounting Standards (IPSAS), and INTOSAI Lima Declaration and other standards. The clarity and explicit criteria and ranking represent a clear advance over previous tools.
A unique feature of the framework is the specification of performance for each rating level of each indicator (the expectation of evidence to justify the rating in the accompanying performance report). The clear, more objective (more evidence-based) design of the indicator set should allow tracking of progress (or regress) in PFM system performance over-time. This is a big advantage over conventional instruments, and provides a more robust learning framework for reforms --- what works, under what circumstances.
The PFM performance measurement is intentionally designed to be simple and evidence-based allowing countries to conduct self evaluation. The Strengthened Approach (provide a hyperlink) envisions countries taking leadership role in assessing their PFM performance and in designing a reform program to improve the PFM performance.
Full overview of the PFM system
The 28 indicators cover the entire PFM cycle (revenue, expenditure, procurement, financial assets/ liabilities management, etc.). Click the graphic representation of coverage across the PFM cycle below to see an expanded version.
A clear advantage of the instrument is this complete coverage. As noted in a previous post on HIPC expenditure tracking, the results of earlier indicator-based assessment work in Africa and Latin America was that the PFM systems were out of balance. Many reform programs were underway in budget formulation, while budget execution was so weak as to make improvements in formulation somewhat meaningless. A holistic view of the PFM system, such as through a PEFA assessment, would reveal more clearly the weakest links, and enable better targeting of reform programs.
Structure and Content of the Indicator Set
The 28 PEFA indicators are grouped according to six aspects of PFM system performance.
In addition to the country PFM system indicators, the PEFA Framework includes three indicators of Donor Practices (Indicators D1- D3) that impact country PFM systems. Predictability of direct budget support; donor information for budgeting and reporting; and use of national procedures.
Since the Framework was introduced in 2005 --- indeed, even while under development from 2003 --- some have observed that there are dimensions of PFM systems not included in the indicator set. Two are commonly pointed out, and are worth noting here.
Corruption. Corruption, or more appropriately anti-corruption, is not a 'feature' of a PFM system. Rather, it is a symptom of a poorly performing management and accountability system that goes well beyond the PFM system as normally defined. Moreover, corruption cannot be measured directly, but only through 'perceptions' of corruption, with all the problems that entails. Including a 'corruption' indicator is thus meaningless in the evidence-based, objectively verifiable scoring, framework of PEFA. What can be done is to identify the weaknesses in the PFM system that enable corruption to flourish, and the PEFA Framework does enable such an analysis. Work has been done to develop a methodology for using the PEFA Framework to assess the degree to which a PFM system encourages or discourages corruption, and also to provide a basis for developing a reform strategy to strengthen the PFM system against corruption. (For more information, see our September 27, 2007 blog post.)Indicator calibration and scoring
Each indicator is calibrated on four point ordinal scale (A, B, C, D). The PEFA Framework handbook provides the detailed, explicit requirements for each score. These requirements are based on internationally-recognized ‘good practice’.
Indicators have between 1, 2, 3 or 4 dimensions each, for a total indicator set of about 74 dimensions. To provide detailed information & transparency of scoring, each dimension must be rated separately, and the PEFA Framework includes the methodology for arriving at the over-all indicator score (A, B, C, D) from the dimensional scores.
Recent Developments
Increasingly the PEFA indicator set is being used by donors to assess fiduciary risks in supporting partner countries. We will post a separate blog in the future on the interrelationship between PFM performance and fiduciary risks perceived by donors in supporting partner countries.
