Posted by Tej Prakash.
A recent research study on outcome-oriented performance measurement systems[1] points out that while it is easy to measure outputs, it is far more difficult to measure outcomes, especially as many public services are delivered a complex network of contracts, outsourcing, collaboration between private sector and NGOs.
The authors focus on community driven evaluation of programs that directly impact the communities. These are community indicator projects (CIP) which are housed by independent organization (NGO, etc.) and focus on high level community conditions that contribute to the quality of life rather than program specific functions. These are also unlike government performance measurement indicators which are data and indicator driven. The emphasis of CIP is on information sharing and establishing cross program links so as to get the broader picture. It recognizes that specific program may not control broader outcomes but can influence them.
The authors studied three cases: Washington State’s Government Management Accountability and Performance (GMAP) Program; King County, Washington’s “AIMS ( Annual Indicators and Measures) High” Program; and, Oregon’s Progress Board.
The focus of GMAP program in Washington State is on preparing transparent and easy to understand reports on programs relating these to outcomes that matter to the community. The objective is not compliance but improvement in program delivery. So, it analyzes gaps, trends and differences, checks for unintended consequences and analyzes risks, contribute to decision making.
In the second case, under the AIMS program of King County, performance measurement indicators are organized under nine broad categories such as natural resources, land use and transportation, economy, education, and equity. All these indicators converge in the overall objective of improving the quality of life, and the emphasis is on all departments working together to improve overall performance. Each performance measure seeks to answer certain questions such as why is this measure important and what else influences it. In assessing any high performance measure, it looks at all the related programs that contribute to it.
The third case study is of the Oregon Progress Board (OPB) which is an independent oversight agency that works towards the goals of ‘Oregon Shining” plan. This plan has three main goals: quality jobs for all Oregon, caring and safe communities and healthy environment. Each of these three goals has two benchmarks. For example benchmark categories for ‘jobs for all’ are economy and education. One interesting feature of the OPB is their website which provides a wealth of information on progress under different programs, and it has a report generating capacity. The reports also indicate if progress is being made under any category and it includes caveats, if any. The benchmarks can be drilled down to county level and compare progress on any benchmark in any county with other counties in Oregon.
The authors draw some conclusion on how to create effective outcome oriented performance measurement systems which move from compliance to performance orientation. A study of many such programs revealed that the size of staff, in the organizations that are tasked with measuring performance, does not matter. A clear chain of command and ‘one boss’ is important. There should be constant engagement with the implementing program departments. The second important factor are enthusiastic staff. Political support for measurement initiatives is important. And finally, organizational culture that rewards performance matters.
The authors also suggest that performance measurement (PM) systems include designing intermediate program outcomes. It helps establish the link between program outcomes and broader community goals. Actionable and meaningful indicators that can be used for performance improvement are important. These indicators should be few and focused. At the same time, community involvement in designing these indicators makes them more meaningful.
Reports on PM should use simple language. Data should be presented around themes such as improved environment rather than in silo like programs so that interdependence among program contributing to common themes is emphasized. Performance information should lead to learning to improve the performance. And finally, media should be used in publicizing the results. Progress towards meeting goals should be reported regularly. The use of web should be made as much as possible. Sustaining an outcome indicator system involves building relationship with other service providers, engaging top leaders meaningfully and institutionalizing the process in bureaucracy (such as issuing an office order).
This study carries the current research in PM a step further, linking it to community and to broader themes that specific programs contribute to. It acknowledges that this approach can be more easily implemented at the local levels compared with state or national levels. The standard literature on performance measurements at the national levels, which is a part of the literature on performance based budgets, lacks the community dimension. The main challenge is to develop such community-based involvement at the national level programs and link the programs to broader themes of quality of life. However, this study demonstrates that it is daunting but not an impossible tasks. Some of the messages are equally relevant for the national levels: keep the broader outcomes in focus and link all programs to such themes, keep the community involved, get the support of political leadership, keep the language and the messages simple, and always learn from the feedback and improve the systems based on such learning.
--------------------------------------------------------------------------------
