Posted by Dirk Kraan[1]
The IMF’s new resident advisor for South Eastern Europe, Dirk Kraan challenges the practicality of performance budgeting, one of the favourite innovations of New Public Management. Has performance information on a government program provided by a line ministry ever provided a critical assessment of its success? Are Ministries of Finance really interested in performance, let alone equipped to deal with it? Some of the hard questions that will be discussed in this post.....
The history of performance budgeting now stretches over forty years. Arriving at the Dutch Ministry of Finance in 1980, I well remember how we tried to cope with the article of the Budget Code introduced in the early 1970s prescribing that the budget documentation had to provide “performance information”. Around the same time the ideas of the Planning, Programming, Budgeting, System (PBBS) and Management by Objectives (MBO) had been blown over the ocean and been adapted to European conditions in the form of “Rationalisation des Choix Budgétaires” (France), Program Budgeting and Review (UK), and “Policy Analysis” (the Netherlands). The latest attempts to implement these reforms were still very much alive in 1980.
But then, a few years later, under the influence of a severe economic crisis in many western countries, a different wind started to blow. Under the Reagan administration, the Thatcher government and the “no-nonsense” cabinet of Dutch Prime Minister Lubbers many of the earlier reforms were abandoned and the emphasis shifted to effective consolidation procedures: the US’ “Grace Commission”, the UK’s “scrutinies”, the Dutch “Reconsideration procedure”, were early forms of what we now call “spending reviews”. At the Dutch Ministry of Finance we were delighted. Henceforth, Finance was in the lead, not the line ministries. Effectiveness was the lead principle, effectiveness of cuts, that is, not effectiveness of sectoral policies, which was not our concern. We all thought that performance budgeting was over and that henceforth the responsibility for results was returned to where it belonged: in the line ministries. In Finance we could again focus on our core tasks: making sure that means would meet ends, providing our political masters with fiscal space for new policies and combating waste (the three classical functions of the budget).
But then in the 1990s New Public Management came along, blown over the oceans as well, but different oceans this time. We went on study missions to Australia and New Zealand and were exposed to an entirely different philosophy of public management. The budget had to be reclassified according to programs and desired program outcomes, input line items had to be abolished, and program managers needed freedom to manage (“let managers manage”). Everything that the market could provide, had to be outsourced (the “yellow page test” in Australia stipulated that if the yellow pages of the telephone directory, which listed the telephone numbers of business companies, indicated that a service could be delivered by a private company, government production had to be terminated). New Public Management had a profound effect on governments all over the world. As it turned out, it was attractive to right leaning governments as it promised savings and a larger role for private sector markets, as well as to left leaning governments, as it promised to show that government could be as efficient as the private sector (“Reinventing government” by Osborn and Gaebler (1993) was embraced by Tony Blair as well as Bill Clinton).
In the Ministry of Finance we were less enthusiastic. This looked like a rebirth of performance budgeting. We lost control of input line items, we had to negotiate again with line ministries on the basis of outputs and outcomes. In these negotiations we were always at a disadvantage, because regardless of the time we spent on studying the intricacies of sectoral policies, we could never match the accumulated expertise available to our counterparts. The Dutch Budget Office reacted by creating a special “study unit” to help the expenditure units in their supervisory tasks, which meant in practice collecting evidence that programs did not work, were not efficient and were wasteful, information usually withheld from the Ministry of Finance. I served for a number of years as the head of that study unit and enjoyed the work a lot, although I have to admit that it did not help much in compensating the adverse effects of New Public Management.
Where do we stand now? The record of practices commonly labelled as “performance budgeting” is mixed at best. Some countries have made steps back, including the Netherlands and the UK. France, which introduced relatively late a quite radical form of performance budgeting in the form “Loi Organique relative aux Lois des Finances” (LOLF) is struggling to make the best of it, but there is a core of sceptics at Bercy (the Ministry of Finance and Economics) that wants to make steps back as well. The US is also struggling: the US Program Assessment Rating Tool, PART, has been abandoned and been replaced with a more pragmatic approach aimed at performance management and transparency of results rather than performance budgeting. Nowhere in the world, performance budgeting is seen as an undivided success story. This is not to say that there are no valuable elements in the performance budgeting reforms or, for that matter, in New Public Management. Reclassification of the budget according to programs has made the budget a more readable document and has largely been successful in the countries where it was implemented. Managerial accountability for results is generally seen as a basic condition for effective public service delivery. Ministerial accountability for results has changed the character of parliamentary debate for the better and has facilitated the involvement of civil society in public policy development with positive effects on democratic governance. Out-sourcing has led to savings and has boosted private sector markets.
Despite these positives, overall the experience with performance budgeting has, I think, been disappointing. The problems originate in the link between budgeting and performance information. This link has always been emphasised as a crucial part of the reforms, but there are fundamental difficulties in how this link is supposed to work. These difficulties have plagued the reforms from the beginning and reformers have not always faced up to them.
The difficulties are of two kinds:
There are fundamental conceptual problems concerning the role of performance information in budgeting;
There are practical problems concerning the incentives for using performance information in the budget process.
I shall make a few remarks about both kinds of difficulties.
Conceptual problems
There are many definitions of performance budgeting but a common denominator could read as follows: performance budgeting is the use of performance information (information on results of government spending in terms of outputs and outcomes) in the budget process.
In view of this broad definition I see three major conceptual problems that, so far as I know, have not been answered in a satisfactory way, nor in the academic literature on public administration, nor in government documents or in documents of international organisations.
First, the objectives of government programs are hard to specify, let alone to measure in quantitative terms. The question is not that policy objectives in some areas are vague or fluid (notorious is the ministry of foreign affairs: promoting peace in the Middle East, etc.). The point is that in a democracy the objectives of programs cannot be anything else than the objectives of the politicians who decide on the financing of the programs. It is impossible to catch these objectives in performance indicators. Objectives of politicians are always complex. They often do not only apply to the social and economic conditions of the citizenry at large (longer life expectancy, better quality of life, etc.) or of certain subgroups of the population (availability of contraceptives to women) but also to specific means of service delivery (keeping open the hospitals in rural areas, reducing waiting time for certain forms of surgery, etc.). A complete description of a politician’s objectives in a specific policy area would always require a lengthy document, with dozens of different objectives and preferred means of service delivery.
Probably individual politicians would have serious difficulties in describing their own objectives and preferred means in a systematic fashion. Politicians discover their own objectives and preferred means only after they have faced the questions on which they have to decide. Moreover no single politician is responsible for any program. Many politicians have to concur before a program gets financed. Each has its own preferences and often the document that describes the “official” objectives of the program consists of compromise formulations in which all politicians involved can read what they want from the program.
Second, there is no continuity in the definition of program objectives (outcomes) or government services (outputs). If you look at the budget documents of countries that have started to define outcomes and outputs some ten years ago, it turns out that the definitions change permanently (making longitudinal analysis of effectiveness or efficiency impossible). This is understandable. New social or economic research leads to new insights in effective policy instrumentation and thus to the definition of new outputs. Political objectives (outcomes) change also from day to day, first because new politicians are coming in, but also because politicians regularly change their minds about what they want to achieve. For instance, if a dangerous prisoner escapes, you see a sudden upsurge in the importance of the aim of prison safety. If statistics show an upsurge in traffic incidents, you see an upsurge in the importance of road safety. Etc.
Third, the idea that resources should be withheld from ineffective programs is unrealistic. Politicians who are confronted with evidence that a program is ineffective, react by giving more resources, often simultaneously redesigning the program. They reason that program objectives are not achieved because of a lack of resources or deficient program design, not because the objectives are wrong or unrealistic. That is understandable: politicians believe in their own objectives.
Incentive problems
Sensitivity to incentive problems is essential for successful public financial management reform, but such sensitivity has not always been shown by performance management reformers. In this connection I mention three problems that again, so far as I know, have never been adequately addressed.
First, apart from the fact that it is conceptually impossible to catch the objectives of politicians in performance indicators, the use of these indicators for the purpose of providing or withholding financial resources leads inevitably to perverse incentives. This problem cannot be dismissed by saying that the indicators “need further improvement”. Even if it were the case that program objectives were clear and unambiguous, the use of quantitative indicators for providing or withholding financial resources would lead to undesirable behavioural effects. This is in fact the same problem that has been diagnosed already in the nineteen thirties and forties as the main problem of central economic planning. While lecturing on this subject in the past, I have sometimes asked my audience to come up with a good example of a quantitative indicator that would not lead to a perverse behavioural effect. Life expectancy as an indicator of public health policy is about the closest of a reliable indicator that I have heard in reply. But obviously, this would lead to terrible effects if it were used, say to finance health insurance companies, or regional health authorities, pulling resources away from all non-life threatening diseases in favour of life extending treatments that neither patients nor physicians want.
Second, performance information in the budget is provided by line ministries. The Ministry of Finance is supposed to check this information but is often unable to do so by lack of capacity and expertise. Performance information rarely allows any critical assessment of government policies. Line ministries make sure that only positive information is provided, particularly if they expect that critical information can be used against them through reallocation of financial resources. The consequence is that the budget documentation surges in size and simultaneously becomes less, rather than more transparent in relation to its primary purpose, namely to provide information about the government’s budgetary policies.
Third, the Ministry of Finance is not only ill equipped to check performance information but also lacks the incentives to do so. In the absence of a consolidation target, it is almost impossible for a Minister of Finance to win a battle over a program’s budget on the basis of the merits of the program, if the line minister stands behind the program. Every finance officer knows this. There may be some finance officers who bring program deficiencies to the attention of the Minister of Finance out of sense of professional duty, but sometimes this will not even be appreciated by the minister because it may put pressure on him to start a fight that is lost from the start. This is not to deny that performance information may be a factor that leads to the cutting of a program’s budget, but “performance budgeting” is not the way to do it.
Permanent performance dialogue and spending review
All problems mentioned above originate in the link that performance budgeting seeks to establish between performance information and budgeting, not in performance information per se. It can be argued that parliamentarians and the citizenry at large have always been interested in the level and quality of public service provision and the social and economic outcomes of government activity more generally, but the performance movement that started in the 1970s aimed at providing this information in a more systematic and transparent way. This was certainly a valuable innovation in its own right. It has contributed a lot to the quality and democratic legitimacy of governance in the countries where it occurred. But the next question is how this information can be used for the improvement of public policy, if not in the regular budget process. Elsewhere I have written on this at length (for instance: Kraan 2007[2]), but it may be useful to briefly mention here the three main purposes I see for performance information.
First, and probably most importantly, performance information is useful to hold executive politicians to account in parliament and in the public debate. Line ministers should be required to publish plans on what they intend to achieve, expressed in terms of public services provision (outputs) and social and economic objectives (outcomes). This can be plans for the medium and long term or annual plans. Necessarily this will involves quantitative targets, next to more qualitative descriptions of objectives to be achieved. Everybody should be aware however - and normally everybody is aware - that these objectives are subject to permanent amendments and revisions, in the light of new insights about the relation between outputs and outcomes (the primary objective of applied social and economic research) as well as new political objectives: new politicians coming in and new ideas of incumbent politicians. Hence the performance dialogue should be “permanent” and not be annual (as the budget cycle usually is). Also for this reason (next to the more fundamental reason that any discussion of performance information in a budgetary context leads to wrong incentives), the performance dialogue should be organised separate from the budget. Evidently, the performance dialogue is constrained by the annual budget, but within a given budget, much is possible in terms of policy reform if the number of line items is not too large. Moreover supplementary budgets can be used in case of complete overhaul of existing programs.
Second, there is a role for performance information in steering the executive agencies that are under the umbrella of a line minister, even if on “arm’s length” from the ministry, but for which the minister still holds responsibility. Agencies can only effectively be monitored and supervised if the minister knows what they are doing. This requires a permanent performance dialogue between the ministry and the agencies. This dialogue can be best organised in the form of regular meetings between the agency manager and the division of the line ministry that is responsible for policy development for the program concerned. The latter division is equipped to understand what the agency is doing and how its production process is set up, unlike the finance division of the line ministry. In this governance relation too, everybody should, and normally does, understand that objectives may change from day to day and that the minister is accountable to Parliament and society if she/he does not react to new insights or political demands. Hence, this dialogue too, should be permanent and not annual. Again, the budget is a constraint, but a flexible constraint in practice.
Third, performance information can play a role in budgeting, but not in the regular budget process. Here the central notion is that the budget process is characterised by asymmetric incentives. Line ministers have strong incentives to develop proposals for new spending of high quality. In general high quality is an important factor for success in finding support and funding and in gaining the approval of the Ministry of Finance. On the other hand line ministers have little incentives to develop proposals of high quality for savings on current spending, basically for the same reason: high quality is an important factor for finding support for the cuts, and line ministers don’t want cuts in their programs. Therefore, consolidation proposals have to be developed outside the regular budget process. This is the basic rationale for procedures currently known as “spending review”. Forerunners in the development of such procedures were the UK, Canada, Australia and the Netherlands. The spending review procedure in the UK exists since 1998, the spending review procedure in the Netherlands (also known as Reconsideration Procedure) since 1981 . These procedures are characterised by a dominant role of the Ministry of Finance, but simultaneously strong involvement of the relevant line ministry, which has the required expertise and information. The procedures make use of working parties, with participation of external experts and chaired by prominent independent persons. The secretariat is provided by the Ministry of Finance. The reports focus on savings, and usually have to describe one or two proposals with prescribed cut percentages. These procedures have proven to be quite effective. A large share of the proposals has been implemented, leading to sizeable savings in each of the countries concerned. In addition, a large share of the total cuts has been taken from the spending review proposals. Performance data are an essential component of the information base for an effective spending review. In practice these data are not always provided by the line ministries, but also collected from other sources. However, performance information is not the only prerequisite for developing high quality saving proposals. Equally important are indications of inefficiency (usually not directly deducible from performance information in view of the lack of longitudinal data), and indications of deficiencies in policy design (for instance use of subsidies to encourage good behaviour instead of prices to discourage bad behavior). Moreover, there are the obligatory saving options which usually involve reduction of service levels. A growing number of countries are interested in spending review procedures or have started implementing them (Sweden, Denmark, Norway, France).
Final remarks
The IMF has since many years been promoting the implementation of performance budgeting in member states. Some of more recent IMF documents on the subject recommend a prudent, basic approach, particularly in low income countries and in countries where there are still major shortcomings in the processes of public financial management (for instance see Robinson, Last 2009[3]). This is a commendable. Nevertheless, in my view, more restraint would even be more commendable.
I have noticed that elsewhere in this blog, Allen Schick has argued that the “basics of PFM” should be realized first, and in his view, performance budgeting is not a basic PFM requirement in countries that are still struggling to overcome major shortcoming in public sector management, such as unstable budgets, low quality public services, and poorly trained and uncaring government employees. He adds that international organisations including the World Bank and the IMF have not always adhered to this order of priority. As often, I fully agree with Allen’s remarks. But he also seems to imply that “elements of performance budgeting” do belong to best practice, although not to basics. Here I might not entirely concur, although it depends on what he means by “elements”. In my view it is more clear to say that thus far the countries that traditionally have been at the forefront of public sector innovation have not worked out a viable and successful practice of performance budgeting and are partly coming back on practices that they have tried to implement in the past. As long as that is the case we can not speak of best practice, not for so-called advanced countries, nor for transition counties. However, other somewhat related innovations, such as publication of performance data, reclassification of the budget on the basis of programs and holding line ministers and agencies to account for results, have been successful and can be seen as best practice, although not as basics.
[1] Dirk Kraan holds a MA in Law and a PhD in economics. After teaching 10 years at Erasmus University in Rotterdam, he joined the Budget Office of the Dutch Ministry of Finance where he worked for 20 years in several positions. In 2002 he joined the OECD in Paris, where he worked in the Budgeting and Public Expenditures Division of the Directorate of Public Governance and Territorial Development. Dirk joined the IMF in June 2012 as Regional Public Financial Management Advisor South East Europe, and is based in Ljubljana, Slovenia.
[2] D.J. Kraan, D.J. (2007), “Programme Budgeting in OECD Countries”, OECD Journal on Budgeting, Vol. 7, No. 4, pp. 7-47.
[3] M. Robinson, D. Last (2009), “A Basic Model of Performance-Based Budgeting”, IMF Technical notes and Manuals, Washington D.C., IMF.
Note: The posts on the IMF PFM Blog should not be reported as representing the views of the IMF. The views expressed are those of the authors and do not necessarily represent those of the IMF or IMF policy.
