Yet one sometimes observes a thoughtless, and somewhat excessive, enthusiasm for the use of PI which actually deserves some caution. And very often, the challenges of using PI in the budget process are underestimated by PI’s many advocates.
Countries should continue to use PI …. but cautiously
Some policy domains are more suited to PI than others. Education, health, social entitlements, police and urban transportation are popular areas for PI development, while justice, foreign affairs or defense are considered less suitable, along with research, regulatory and administrative functions of government.
It is important to keep in mind that most PI is only a proxy for real public policy results. Indicators of outcomes are not the results themselves. So, when the actual value of an indicator diverges from its targeted path, the right reaction is to conduct further examination in order to understand the meaning and causes of this divergence, and then take relevant actions. Any mechanistic use of indicators should be avoided. Indicators should be taken as warning signs aimed at drawing policymakers’ and managers’ attention to potential problems in delivering public services. Caution is warranted especially when indicators are deemed to trigger automatic rewards or sanctions. As Christopher Hood has wisely noted, the PI system sometimes “hits the target but misses the point”.
Information asymmetry is one of the major shortcomings of PI. Many indicators are generated directly by delivery agencies, and hence are under their control. In the same vein, agencies tend to focus their attention and action on the indicators within their domain of direct interest and influence, and neglect anything else. There is a temptation for such entities to play with indicators and even to manipulate them. This is summarized in Goodhart’s law: a measure that becomes a target may thereby cease to be a good measure. To mitigate possible distorting effects of PI, attention has to be paid to their incentive aspects. Furthermore, independent control of data reliability and relevancy is needed. The selection, production and use of PI ideally should be overseen by independent bodies.
PI overload should be avoided. Many governments seek to multiply the number of indicators in the illusory hope of better reflecting actual results and impacts. But too much data tends to have the reverse effect, and to obscure the issues they are designed to illuminate. A limited and careful choice of PI is desirable. This selective approach to PI is not only necessary to prevent information congestion but also to spare resources. Collecting PI is costly, especially on outcomes where data are outside the direct control of delivery agencies. A careful cost-benefit analysis should be made prior to embarking on an ambitious PI framework. Any indicators which are irrelevant or unused should be abandoned.
How to increase the use of PI in the budgeting process
The effective use of PI in budgeting is progressing only gradually. Many countries include PI in their budget documentation but only limited use of it is made in decision making. In most countries, budget negotiations typically draw on other forms of information and decision rules: spending history, the focus on incremental change, legal constraints, the balance of power and other political considerations. Even countries deeply engaged in performance budgeting recognize that PI often plays a secondary role in budget decisions and prefer to use the term ‘performance-informed budgeting’ as opposed to ‘direct performance budgeting’.
Cultural reasons and conservatism may partly explain this limited success. Budget officials still tend to maintain a focus on inputs which is their historical core responsibility and skill. They may fear being drawn onto the results field, where they are in a weak position since arguments for expenditure increases are many and often powerful.Spending ministry officials consider that goals and results are their prerogative and are reluctant to accept that finance ministry officials also address this type of issue. Beyond that, two more substantial types of reason should be considered: methodological challenges are not always properly addressed and policymaker involvement remains weak.
A first methodological difficulty arises from attribution issues. The chain of causation leading from inputs to outcomes is complex. A given outcome may result from several factors and the amount of public resources allocated is only one among many such factors which include private spending, tax and regulatory policy instruments, together with the natural and cultural environment. The weighting of the relative contribution of these factors is very challenging. Most attribution issues are related to the outputs/outcomes link.
Another technical challenge is to have a full understanding of the value chain linking expenditure to results. It is necessary to analyze the several steps that transform financial resources into results, be it outcomes or outputs. In this value chain, inefficiencies may occur at every step. The longer the chain, the more difficult it is to understand.For instance, if an improvement of health outcomes is considered a desirable policy goal, more resources may not yield the expected results for several reasons. Increased public demand may raise input prices, technical inefficiencies may worsen because of congestion issues, and the outputs mix deemed to contribute to outcomes may prove irrelevant. One cannot expect any strict proportionality between input changes and outcome changes, neither upward, nor downward.
Finally, policymakers are not sufficiently involved in the choice of PI in budgeting activities. The PI chain should start from the top. Ideally, the results expected from public policy initiatives are outcomes, the ultimate impact that citizens and politicians are interested in, rather than outputs. The determination of desired outcomes, however, is often contentious and the choice of indicators is a sensitive issue that depends on the goals assigned to a given policy. These goals are within the politicians’ remit. PI cannot be developed only within the circle of experts, public managers and academics, without politicians’ involvement—ministers and parliamentarians as well as political parties, CSOs and citizens. Budgeting is a political activity: allocating resources to public policies implies setting priorities and making hard choices. Policymakers should take responsibility for defining clear and explicit policy goals and the PI that goes with them. This is an essential prerequisite to the development of any PI framework and performance budgeting system. Frequently, however, it is overlooked when PI systems are being designed and implemented.
If properly addressed, these difficulties can be overcome. First, the focus should be put on outputs for two reasons: i) there are fewer attribution issues in the resources/outputs link than in the resources/outcomes link; ii) outputs are easily computable, via the costing of activities that yield them. However, some references to outcomes should be kept: the type of outputs (or output mix) deemed to support each outcome should be defined beforehand by policy makers. Thus, outputs appear to be the cornerstone of performance budgeting, at the crossroads between policy-making and operational management. They are both what is demanded by politicians and what is delivered by public managers. Second, there should be an explicit determination of policy goals without which neither a PI framework nor performance budgeting may function. In this respect, both experience and realism suggest to select limited issues on which politicians’ attention might be solicited: setting clear goals and budget priorities. A greater personal involvement of ministers in performance budgeting and management often appears illusory, given their crowded agendas.
[1] Technical Assistance Advisor, PFM1 Division, Fiscal Affairs Department, IMF. This article is based on a “selected issue paper”(SIP)prepared by the author during a recent IMF Article IV mission.
Note: The posts on the IMF PFM Blog should not be reported as representing the views of the IMF. The views expressed are those of the authors and do not necessarily represent those of the IMF or IMF policy.
