Croatia is due to become the 28th member state of the European Union on the 1st of July. Ahead of the country’s accession, Simona Milio writes on efforts to build Croatia’s capacity for evaluating its use of EU funds. All EU member states that receive Cohesion Policy funding are required to have a strong evaluation system in place, yet the Croatian system is still a work in progress. She argues that Croatia should make greater use of universities to help set up systems capable of improving the country’s evaluation capacities.
When it comes to Cohesion Policy and EU funds spending, the European Commission has put a strong emphasis on evaluation capacity. Evaluation judges programme implementation on the basis of the outputs, results and impact on society. As the Commission states, “the purpose of evaluation is to check the raison d’etre of a public intervention, to confirm both reproducible success stories and failures not to be repeated.”
The capacity to evaluate the results of the implemented resources is relevant at three different stages: ex-ante, in itinere, and ex-post. Ex-ante evaluation serves to clarify the needs of the territory and to verify the existence and nature of the problems within the territory; it helps to ensure that the final programme is as relevant and coherent as possible. The mid-term evaluation (in itinere) is performed halfway through implementation of the interventions, and it serves to highlight and adjust eventual deficiencies. An ex-post evaluation serves to examine whether the needs or problems still exist and whether the programme has achieved the expected results. Ex-post evaluation provides exceptional support and guidance to the administration in programming future financial interventions and organising the implementation of actions so that subsequent programming cycles are successful. Also, on-going and thematic evaluation is deemed necessary. However, evaluation not only guarantees a more effective and efficient approach to investing EU funds, but also increases the accountability of government.
All EU Member States that receive Cohesion Policy funding are required to have a strong evaluation system in place. Therefore, the Croatian administration, in view of the forthcoming EU accession on 1 July 2013, has invested a conspicuous amount in the Europe Aid project “Ex-ante evaluation of programming documents and strengthening evaluation capacity for EU funds post-accession”. As part of this project we have created a number of training modules to strengthen evaluation capacity for EU Cohesion Policy funds management. The training modules were based on a thorough training need assessments analysis which showed that the institutions involved in managing Structural and Cohesion Funds (SCF) and the staff currently assigned to perform the evaluation functions in the same institutions do not have a remarkable level of capacity and knowledge concerning evaluation.
The structures currently entrusted with the Managing Authority or the Intermediate Bodies have had little exposure to some parts of the EU programme cycle not only, as is obvious, with reference to SCF, but also where pre-accession instruments are concerned. Changing institutional settings (changing Ministry denominations that imply changes in the “ownership” and scope of the policies, delegated powers, aggregations of policies and areas of interventions, etc.) have broken the chain of policy and programme cycle, when applied according to the EU principles. Furthermore, often Ministries have been recipients of programming documents elaborated elsewhere in the overall administrative structure, with little “ownership” from the relevant administration. When evaluation has been carried out, this has been managed at the level of coordination structure, with the line ministries involved mostly in the position of providing data at the onset of the analysis and comments at its end. It should also be noted that the scope of these evaluation exercises was often rather limited, and could therefore not expose the individual structure and its experts to significant relations with the independent evaluator.
Nevertheless, the participants to the capacity and knowledge assessment have consistently reported appreciation for receiving useful comments and recommendations, which highlighted the benefits and the positive potential of a proper evaluation exercise. This definitely sets a positive backdrop for capacity building activities at the institutional level. Indeed, evaluation and capacity building is something that cannot be imposed from above, it must be welcomed from the administration.
Unfortunately, personnel turnover and ‘in office’ rotation have removed experts with prior experience of programming, evaluation, partnership involvement and management from the staff of the current structures designated to implement SCF. Considering the complexities of managing the SCF policy and programme cycle, special care should be put in the management of human resources to ensure that expertise matured over time is retained and is also appropriately exploited to support and facilitate capacities and knowledge in the organisations.
Also, analysis of the training delivery framework has shown that at present comprehensive training on evaluation is not provided either on a regular or occasional basis. Furthermore, Universities and other institutions (including private entities) have not moved earlier to propose and implement courses aimed at producing evaluators, or professionals with academic skills closer to the needs of an evaluator.
It will not come as a surprise, then, that the academic background and experience of personnel selected to perform the evaluation function show a less than desirable level of specialisation. However, as mentioned, personal motivation appears rather high, and personal willingness constitutes a very consistent resource to bank on in order to develop capacity in the present institutional and organisational setting.
The literature on institutionalising evaluation has exponentially increased since the 1980s and has possibly peaked in the past decade. The discourse has evolved from various backgrounds, but there is a general consensus among evaluation researchers that there is not a unique pathway to institutionalising evaluation and that systems look different in the various political and cultural environments. In this case, it is important that the Croatian administration takes the initiative to solicit or incentivise universities and other potential providers to set up comprehensive systems to supply evaluation capacities. The availability of such systems would, on the one hand, provide skilled personnel for public administration, while on the other hand it will also contribute to building an evaluation network at local levels that can link Croatia to the wider European evaluation network.
This article is the result of work carried out in Croatia over the period January 2012 – April 2013. More details are available here.
Note: This article gives the views of the author, and not the position of EUROPP – European Politics and Policy, nor of the London School of Economics.
Shortened URL for this post: http://bit.ly/Zyk6Cf
Simona Milio – LSE
Simona Milio is Associate Director of the Social and Cohesion Policy Unit at LSE. Her publications include From Policy to implementation in the European Union (IB Tauris, 2010).