First, by creating two classes of PEFA assessment report – the first with the quality “stamp” and the second without – the mechanism will likely put pressure on agencies and governments sponsoring PEFA assessments to “volunteer” to go through the validation process. Not having an endorsement will signify publicly that the reports are sub-standard in an important sense. Nevertheless, it is unfortunate that the Steering Committee did not agree to make “PEFA check” a mandatory procedure for all assessments.
Second, “PEFA Check” will not include an endorsement that a report has been completed within a defined period and that the final report has been published. It would have been helpful if the quality certificate had included these elements.
Third and most important, by focusing on process and not substance, the Steering Committee has missed an important opportunity. The Secretariat, when requested, already reviews the technical compliance of PEFA scores with the assessment methodology. This is a much more important check than the endorsement of process. It would have been a relatively simple matter to add to the “PEFA Check” a certification by the Secretariat that, for each assessment report, all the indicators reviewed comply with the required methodology. Such an endorsement would not strain the capacity of the Secretariat, one of the somewhat questionable reasons given for not going further on quality control.
Why have the PEFA partners been so reluctant and so slow to grasp the quality control nettle?
- First, institutional pride. The decentralized process of carrying out PEFA assessments was a decision taken at the beginning of the program, and in retrospect looks mistaken. Individual agencies have insisted on relying on their own internal quality control mechanisms, and understandably resist intrusion on their turf. The rigor and discipline of these different quality control mechanisms varies hugely, however, and is a major source of variable quality in assessment reports, on which the new procedures will have a limited impact.
- Second, a Secretariat which is kept under a tight rein by the Steering Committee, and has been given neither the mandate nor sufficient resources to exercise an independent and effective role in the quality control process.
- Third, by the same token, the Steering Committee - for all its success in leading the PEFA program - displays familiar characteristics of an international bureaucracy whose members have widely dispersed values and incentives: lack of flexibility, failure to recognize problems until too late, and slowness to reach agreement on reasonable solutions that would strengthen the PEFA program.
Agencies taking the lead in PEFA assessments could unilaterally address some of the above-mentioned weaknesses in “PEFA Check” by committing to:
- Use experts with direct experience of working in the countries concerned to review the reports. Such experts would be more likely to identify weaknesses in the scoring of the indicators.
- Publish external comments received on draft PEFA reports.
- Publish PEFA reports, subject to approval by the national authorities.
By meeting this challenge, each stakeholder will have an opportunity to show where they stand on the quality assurance issue.
Note: The posts on the IMF PFM Blog should not be reported as representing the views of the IMF. The views expressed are those of the authors and do not necessarily represent those of the IMF or IMF policy.
