Risto Ivanov
Evaluation market is being captured by audits, economists and management consultants. The audit professionals are well organized though established international standards for financial, organizational and performance audit. The performance audit concerned to public sector and is based on 3E indicators (economy, efficiency and effectiveness) that are overlapping with evaluation methodology. The demand for performance audit is increasing in Western Balkan Countries proportionally to the increase in public depts. and allocated EU funds. At the moment the performance audit market is manly covered by State Audit Offices which practice is to use top-dawn perspective, focusing mainly on the basic question: Are things being done in right way? The bottom-up perspective is focused on the effects of the activity on entity and larger community, answering basic question: Are the right things being done?
The objective of this article is to present opportunities for evaluators in performance evaluation of governments’ policy, programs and projects. The methodology of the brief research, carried out in September 2017, is focused on analyses of existing performance audit gaps, identification of fields to be covered by evaluators and chances for evaluators to introduced bottom-up perspective in performance evaluation.
What is performance Audit?
Performance auditing is an independent and objective examination of government undertakings, systems, programs or organizations, with regard to one or more of the three aspects of economy, efficiency and effectiveness, aiming to lead to improvements.
Figure 1 Principle of performance Audit – 3E (economy efficiency and effectiveness)
Implementation of principles during project performance means: Economy, minimizing the costs of resources; Efficiency, getting the most from the available resources; and Effectiveness meeting the objectives set and achieving the intended results.
Market actors and size
The market actors on the supply side are mostly recruited by auditors, Economists, management Consultants and State Audit Bureaus.
Auditors are the best organized segment whose market positioning is built with the International Standards of Supreme Audit Institutions, ISSA 3000. It is hold by INTOSAI International Organization of Supreme Audit Institutions[1].
Economist defined market positioning by Licenses and certificates about performance evaluation. Management Consultants have developed CMC[2] (certified management consultant) and PMP[3] (project manager professional) where evaluation is one of the six phases of project management cycles.
The market position of the State Audit Bureaus in western Balkan Countries[4] is the defined by the Laws and they are mainly engaged in performance audit of the national Government policies, programs and projects.
On the demand side for performance audit and evaluation the following segments are visible in WB region: National Governments (State Budget Programs); International Donors programs and projects ; EU Programs and funds ; World Bank Loans ; CSOs ( Civil Society organizations) that are beneficiaries of the granted projects.
The brief research carried out among State Audits Bureaus about the market size for the performance audit market is presented on table below
Table 1 Number of Performance Audits (PA) implemented in five WB Countries
In the last four years the total number of PA reports is 342. The biggest number of PA reports are implemented in Croatia and Macedonia. Having in mind the number of the programs and projects that used public resources it is small number of performance audit. In some Countries is difficult to differentiate between Financial Audit and Performance Audit so those table cells are marked as N/A (Not Available).
Beside the number of PA reports brief research was focused as well on the principles that were evaluated. On the Macedonian case is obviously that the huge number of the reports concerned only to two principles (economy and efficiency).
Table 2 Principles implemented in PA reports – Macedonian case
Two main conclusions can be drown concerning PA: (1) – the State Audit Offices is more focused on Financial Audit than to Performance Audit; and (2) more than 80 % from analyzed PA Report in Macedonia evaluated only two principles, economy and efficiency.
Performance Audit Gaps
The identified gaps are chance for professional evaluators to enter market for performance evaluation:
- EU citizen oriented policy that means to evaluate from citizen (bottom) to policy, programs, projects (UP) , answering the question: Are the right things being done? . However the evaluation of public policy, programs and projects is still focused on the way how the public resources are spend and monitored, meaning the answers about the question: Are things being done in right way?
In addition the assessment of 3E’s is usually avoiding or it is not compared with baseline indicators.
- In the most of the public sector institutions and NGOs the performance evaluation is carried out by internal evaluators without visible capacity to implement it with high quality standards. According to Macedonian practice the demand for external performance evaluators is increase by International donors.
- The recommendations from performance audits refer to legislations and institutional improvements rather than effects to audited entity or larger community, steaming from dominant top-dawn perspective in performance audits. Briefly stating that up or the money- resources are more important than dawn or citizens.
- In the Macedonia case the amount of the state budget that is spent for Programs is 1 billion euro per year and additional 300 million euro from EU and International Donors. The principle of effectiveness is more than important for 2 million citizens, meaning what are achieved goals compared to desired goals.
- Existing PA does not included qualitative assessment techniques. It is linked with the principle of effectiveness and citizens satisfaction from implemented policy, program or project. Briefly stating that bottom, the people are more important than up or the money-resources.
- Evaluators’ community is not well organized and lacking networking to enter on the market for assessment of performances in public policy, programs and projects.
Chances for professional evaluators
The existing market gaps can be overcome by better cooperation among evaluators in WB countries. Bottom-up perspective of performance audit is chance for WBEN to conquer the evaluation markets through cooperation on developing common baseline and comparative indicators for 3Es. The identified good practices in performance evaluation can be network potential to promote relevance of WBEN to support national VOPEs for beater market positioning.
WBEN and National VOPEs could start to promote the importance of evaluation of effectiveness of Governmental policy, programs and projects, focusing on achieved results that improve citizens’ lives. The Governments from WB Countries have to be aware of usability of performance evaluation in drawing lessons about the changes that have been occurred, comparing with EU living standards and improvement of quality of citizens lives.
The chances for professional evaluators to enter this new market are linked with three types of activities of the Governments related to:
- Readiness to involve performance evaluation as systematic solution in terms of accomplishment of strategy goals, meaning that evaluation is part of each adopted policy, program and/or projects. It will demonstrate interest for citizens’ needs steaming from policy implementation;
- Willingness to allocate resources for external professional evaluators who will carry out performance evaluation. It is recommended the share from 0,5 to 2% from the total budget of the programs/projects to be allocated for performance evaluation;
- Drawing lessons learnt from performance evaluation as base for definition of future policy in the fields it is concerned to. It will be step towards respecting citizens’ needs and the progress of its living standards.
Implementation of the recommendations above will open new perspectives for the professional evaluators in WB Countries and usage of their big potential to contribute towards successful citizen oriented policy.
[4] http://www.dzr.mk/DesktopDefault.aspx?tabindex=0&tabid=1;
http://www.revizija.gov.ba/default.aspx?template_id=79&langTag=bs-BA&pageIndex=1; http://www.dri.rs/home.4.html
Bibliography:
- European Court of Auditors, Performance Audit Manual 2015
- The International Standards of Supreme Audit Institutions, ISSAI, are issued by the International Organization of Supreme Audit Institutions, INTOSAI Performance Audit Guidelines: ISSAI 3000 – 3100
- Access to links
State Audit Office – Republic of Macedonia http://www.dzr.mk/DesktopDefault.aspx?tabindex=0&tabid=1
State Audit Office – Republic of Croatia http://www.revizija.hr/en
Audit Office of institutions of Bosnia and Herzegovina http://www.revizija.gov.ba/default.aspx?template_id=79&langTag=bs-BA&pageIndex=1
State Audit Institution – Republic of Serbia http://www.dri.rs/home.4.html
