National
Development Agency

The National Development Agency commissioned Korzó – Szeged Ltd., the company operating the REmix Energy Efficiency and Renewable Energy Agency, to carry out the evaluation of the indicators specified in EPIOP energy calls (EPIOP-2004-1.7.0.F and EPIOP-2005-1.7.0.F); the evaluation was completed in October 2008. The objective of the evaluation carried out was to review and evaluate the results of indicators defined in the energy calls of the Environmental Protection and Infrastructure Operational Programme on the grounds of the targets set by the EU, the government and the call. The other aim of the study was to facilitate the better and more efficient implementation of EPIOP calls and to provide support for implementing the 2007-2013 programming period. 

The evaluation identified the following:

• Commitments assumed in the 2004-2006 programming cycle were largely fulfilled, more specifically,
• 87.6% in the case of the index for „Built-in renewable new electric power capacity” (25 MW); 109.6% in the case of the index for „annual volume of electric power generated with the help of the new renewable capacity” (50 GWh/year); 107% in the case of the index for „annual energy savings induced by energy efficiency in equivalent of heat” 1110 000 GJ/year.

Evaluation of EPIOP indicators

The National Development Agency commissioned EX ANTE Consulting Ltd. to carry out the evaluation of the indicators of the Environmental Protection and Infrastructure Operational Programme; the evaluation was completed at the end of 2007. The consulting firm completed the evaluation of the indicators of the Environmental Protection and Infrastructure Operational Programme at the end of 2007. The objective of the entire interim evaluation project, and therefore indicator analysis as well, can be divided into two parts. The short-term objective of the study was to facilitate the better, more efficient and effective implementation of the OP running currently as well, establish the grounds of PC and OP changes, support the structured summarisation of implementation experiences and provide support for planning the 2007-2013 programming period. In addition, the long-term objective relates to developing evaluation methodology capable of supporting the planning, implementation and ex-post evaluation of development programmes on a scientific basis, as well as practical grounds adjusted to prevailing demands.    

Evaluators identified the following:

• Due to the heterogeneity of the programme, the EPIOP indicator system is extremely diverse; however, the number of indicators is limited in compliance with the requirements stipulated by the European Commission. This heterogeneity especially made it difficult to devise general indicators at a programme and priority level. In generally terms, the quality of the (measure-level) indicators specified in the Programme Compliment document (PC) was better than those of the OP.  
• According to the evaluators, suitable coverage was one of most pertinent problems. The table below presents the ratio of the projects of the given measure covered by PC output and result indicators (presented in two separate columns) measured according to gross amount paid. The table demonstrates that this can be a very low value in the case of certain indicators.  
• The reason for inappropriate coverage primarily relates to the way it was not possible to enforce the use of the given indicator in the case of the given project, or to how the evaluation committee did not override the decision of the beneficiary made in this regard.

	Output indicator	Result indicator
1.1.Improving water quality	63%	100%
1.2. Treatment of animal waste	58%	98%
1.3. Healthcare waste, construction and demolition waste	71%	100%
1.4. Reducing environmental loads	71%	71%
1.5. Nature and flood protection	100%	70%
1.6. Reducing air pollution and noise levels	49%	0%
1.7. Developing energy management	4%	100%
2.1. Development of the highway network	100%	56%
2.2. Environmentally friendly transport	26%	100%
3.1. Technical assistance for EPIOP tasks	44%	13%

• There was a problem with interpretation on many instances, especially as regards the focus of the given indicator (the scope in which the indicator should be measured); moreover, the base values also caused a great deal of misunderstanding (particularly in the case of „concerned population” type indicators).
• Deeper analysis was not possible due to missing indicator registration primarily in the case of the transport infrastructure development and the technical assistance priority. This could neither be complimented by interviews in the case of the transport infrastructure development priority, since, due to the lack of intermediate bodies, the Managing Authority is responsible for implementing these measures, which, however – due to staff shortages – failed to perform this function at a suitable level last year.       
• The lack of a standardised system made it difficult to carry out analysis, i.e. disregarding SMIS in terms of indicator administration. Several reasons underlying this were exposed during interviews (SMIS was set up at a later date than that of the launch of the programme; necessary developments were not completed, etc.), as a result of which, indicators were registered in a separate database and in the form of monthly reports and other tables, thus creating problems for standardised and transparent data flow.  

Evaluation of Hungarian environmental protection development approaches in the context of the assessment of the objectives and efficiency of EPIOP environmental protection measures

The National Development Agency commissioned ÖKO Ltd. to carry out the evaluation of the measures of the Environmental Protection and Infrastructure Operational Programme; the evaluation was completed in summer 2008. The final report was compiled in August 2008.  

The following measures were the object of the measure: EPIOP 1/A Improving the quality of drinking water;  1/B Development of sewage disposal and treatment; 3. Healthcare waste, construction and demolition waste management; 4. Reducing environmental pollution to protect subsurface waters and drinking water supplies; 5. Nature protection and sustainable flood protection; 6. Measuring air pollution and noise levels.

The objective of the evaluation completed was to review and evaluate the efficiency and effectiveness of the measures of the Environmental Protection Infrastructure Operational Programme on the grounds of the targets set by the EU, the government and the call. The study put forth a series of recommendations to improve the quality and effectiveness of currently running EPIOP calls, and therefore contributes to the efficient management of the 2007-2013 programming period.    

Main findings of the evaluation:

• Output indicators are in all cases presented as clearly identifiable categories in the case of the monitoring data of the projects; however, serious interpretation problems often arose in the case of result indicators.      
• Horizontal indicators with an identical approach were generated within the measure; however, during the course of our assessment, there wasn’t a single horizontal indicator that was measured across the board in the case of each project, in spite oh how this would be the primary and most relevant data source in terms of the object of the present assessment.  
• The mode of specification of indicators was misunderstood in many cases and beneficiaries interpreted this requirement in different ways, particularly in the case of „growth” type indicators. In these cases, what we take as the base value needs to be clearly defined, i.e. 0 or the current value. It would be worthwhile compiling a guide on these requirements.
• Data control in the feasibility studies was made difficult owing to how the feasibility studies were compiled in very different structures within the measures; estimates were defined on the basis of significantly varying methodology (e.g. in the case of management of construction and demolition waste). It is definitely necessary to standardise these in the future to be able to appropriately carry out ex-post evaluations. 
• The 4 EPIOP environmental protection priorities only has a direct impact on 13 of the numerically quantified targets of the 18 objectives of National Environmental Protection Programme II, and EPIOP fails to contribute to the implementation of over half of the objectives.

Evaluation of EIOP 1.2 measure - Animal Waste Management

The National Development Agency commissioned KPMG Advisory Ltd. to carry out the evaluation of the measure 1.2. Animal waste management within the framework of the Environment and Infrastructure Operational Programme (EIOP). The final report was compiled in December 2008. The main objective of the evaluation was to create an evaluating report that examines the necessity of the measure, the adequacy of determining the beneficiaries and the realization of the intervention in a structured form, according to specific evaluative aspects. In the frame of EIOP 15 projects out of 41 submitted applications have been given financial assistance totaling 2.68 million HUF between 2004 and 2006. By December 31, 2008 all projects have been realized.

In the framework of the measure 85 dead animal pits and sites have been recultivated. Regional animal waste collection plants, composting sites have been established at several points in Hungary and two processing, sterilising plants have been reconstructed. Besides, the collecting vehicle fleet has been renewed. As a result of the project approximately 150 m3 collecting capacity has been established  – which on an annual basis (counting with one

transportation per week) – allows the collection of approximately 7400 m3, i.e. 5900 tons of animal waste.

Main findings of the evaluation:

Closing down dead animal pits and sites

Based on interviews and collected information the problem of dead animal pits seems to have been solved although there is not sufficient information available to assess the real situation. All dead animal pits have been closed down by meeting veterinary hygienic requirements, but further tasks lie ahead as regards recultivation of remaining dead animal pits and sites.

Status of regional waste collection and processing plants

Most applicants envisaged the development by establishing animal waste collection plants and by transporting the collected waste to either to ATEV (Animal Protein Processing Plc., later referred to as ATEV) or SZATEV treatment plants. Estimating the demand for collection plants is difficult as there is no commonly recognized database and a comprehensive logistical conception between the stakeholders of the sector. A key factor in the course of collecting is speed (within 24 hours) which should operate in an overall, countrywide system.

In the planning and beginning period of EIOP more animal waste originated from the territories of the municipalities which significantly decreased as animal keeping habits changed. Today very rarely arises the need for municipalities to take over animal waste however regional waste collection plants were built to meet this demand. Collection plants will very likely face utilisation difficulties. This is especially true for the Hajdúszoboszló and Tótkomlós collection plants as they are situated close to the ATEV plants. Utilisation difficulties will most probably arise in the case of collection plants with rural settlement structures (as pet animals of weight less than 50 kilograms can be buried in the garden of the owner in these cases), while there remains a more significant need for regional waste collection plants in cities. True consequences regarding the quantity of delivered animal waste may be drawn only one year after all the plants have been put into operation. Despite the fact that the projects ensure the coverage of only some regions, the accepted applications can be geographically embedded into the current system.

Keeping regional collection plants in the ownership of municipalities

With the animal waste collection plants the municipality has overtaken a traditional, however not obligatory extra task that is accompanied by further management and marketing activities while financing in the current situation can be critical.

Technical content of the projects

The technical content has excessively enlarged, and due to regulative constraints it is not possible to narrow down the projects to a minimal content that would be still adequate to fulfil its task. As regards the efficient operation of the collection plants it is necessary that appropriate collecting vehicles are available.

Keeping ATEV (waste management) in public hands

In case of animal waste category 1 and 2 there is no competition. As this field does not function on a market basis, it is not due the regulation that ATEV operates in a monopolistic way. On a national level the presence of ATEV is considered useful as it is able to collect animal waste category 1 and 2 with small additional costs beside collecting animal waste on a business basis. If ATEV did not run this activity, the collection of animal waste would be more expensive, and applying the polluter pays principle would put further burden on farmers.  

Continuation of the measure in the framework of EIOP

In general the development project documents envisaged a different project composition (closing down of dead animal pits and development of processing). Instead of this collection plants were established and a countrywide vehicle fleet was set up. The development of the vehicle fleet is useful, however the operation of the collection plants will necessitate further attention. It is not only the quantity of delivered waste that justifies the measure, but also the requirement as regards collection of animal waste must not be accompanied with epidemic, public health or environmental risks.

Continuation of the measure according to an agreed strategy is justified. The strategy would include the development of the vehicle fleet, the energy efficient development of processing, improving the countywide coverage of the collecting network and taking advantage of biogas possibilities.