2018 Romania: Opportunities for Adolescents Formative Evaluation Report Romania

Background:

There are around 2 million adolescents (defined as children from 10 to 18 years old) living in Romania, out of which 47% are living in urban areas. Adolescents in Romania were historically targeted by HIV/AIDS programmes and policies and more recently through youth employment measures . Adolescents’ needs and problems are not clearly identified and therefore national policies, programmes and projects cannot provide an adequate response to these needs. Previous research  conducted by UNICEF Romania shows that adolescents face major problems like poverty, social exclusion, and challenges related to education, health and employment. There are several risky behaviours that adolescents are most vulnerable including the use of alcohol, tobacco and illegal drugs, premature and unprotected sex. The objective of this evaluation is the modelling project OPPORTUNITIES FOR ADOLESCENTS, implemented in Romania between June 2014 – December 2017. The modelling project was designed and has been implemented by UNICEF Country Office in Romania, in partnership with a large number of stakeholders, both at central and local levels. The modelling project was designed to respond to the needs of adolescents in Romania, by testing its proposed intervention model at local level (5 cities in Romania- Bucharest, Bacău, Cluj Napoca, Constanța, Iași) via 5 newly established Resource Centres for Adolescents, one in each of the 5 cities.

Purpose/Objective:

The purpose of the formative evaluation is therefore to analyze the status of implementation, determine if the modelling project needs any adjustments and collect evidence related to the modelling project’s effectiveness and efficiency in reducing adolescent risky behaviours. The evaluation will also provide information on model’s relevance and will help identify lessons learned and good practices. The evaluation includes an assessment of the extent to which the evaluated modelling project was based on a child rights approach and whether age and gender issues are considered in providing support services to adolescents.

There are two main objectives to be attained by this evaluation:

1. Measure the relevance, effectiveness, efficiency and elements of sustainability of the modelling project in reducing risky behaviours in adolescents in 5 urban areas;
2. Identify the lessons learned and good practices and make recommendations for modelling project adjustment, as well as recommendations on how to promote legislation and policy development related to adolescents (information related to legislation, institutional building, and monitoring and evaluation mechanisms) with a particular focus on the definition of adolescents’ friendly services.

The overall scope of the evaluation is to assess the performance of the current modelling project “Opportunities for Adolescents” and make proposals for improvement or adjustment in terms of design, what has worked and what has not worked, internal and external factors that affected in a positive or negative way the modelling project implementation. The evaluation covers the period of implementation June 2014 – June 2016. The focus is on the five cities where the modelling project is implemented (Bucharest-4th  sector, Bacău, Cluj-Napoca, Iaşi, Constanţa) and on the policy development at national level. The evaluation process includes all of the project stakeholders.

Methodology:

The evaluation methods used were: Desk analysis, Concept mapping, Interviews, Focus groups, Survey and Case studies. In total, the evaluation process included 48 interviews at central and local level, 12 focus groups, a survey and 4 case studies at local level.

Data sources were selected based on:

• the evaluation logic (where can the needed information be found);
• need to ensure a complete coverage of all of the modelling project stakeholders, both at central and local levels. In so doing, the evaluators have made sure that all of the perspectives on the modelling project have been captured: of duty-bearers, of rights-holders, of the project managers at different levels, of employees at the Resource Centres for Adolescents, of other stakeholders at national or central level.

Several measures were taken to ensure data quality: data collection instruments underwent an approval process by UNICEF, interviews and focus groups were properly organised, triangulation was ensured. All of the data planned to be collected was collected in good conditions and no significant limitation of access to data or methodological limitations were encountered. The entire evaluation process was however delayed by two changes in the team leader position and by a lengthy process of designing, improving and approving the data collection instruments. The delay has had no negative consequences on the data collection process per se, but the evaluation report will be available much later than foreseen. Nevertheless, all its conclusions, recommendations and lessons learned retain their validity and can still be put to use, in spite of the delay.

Findings and conclusions:

The project was adequately informed by thorough analyses that indicated the problems and gaps standing in the way of realisation of adolescents rights. Central level activities had a direct focus on vulnerable adolescents: improved legal provisions, improved intervention methodologies for adolescents using drugs and adolescents living on the streets. At local level, activities of the Resource Centres for Adolescents were designed to reach vulnerable adolescents as well, either directly or by referral of vulnerable adolescents to the centres’ activities by other public services.

There are clear benefits for adolescents and parents due to the centres’ activities:

• Adolescents (including vulnerable adolescents that are referred to the centre) have developed their socio-emotional abilities, are well-informed and have a better understanding regarding the consequences of risky behaviours and are more able to make healthy choices and avoid such behaviours;
• Parents have a better understanding of the challenges facing adolescents and their emotional and support needs are more able to provide such support.

Each of the 5 Resource Centres for Adolescents had reached thousands of adolescents by June 2016. Main constraints identified are: unsatisfactory cooperation between the local partners and the centre and the unsatisfactory involvement of the city administration in the modelling project. The modelling project has been less successful in reaching the most vulnerable adolescents: adolescents living in the streets, Roma adolescents, adolescents that are have an addiction. The yearly cost per supported adolescent, as average of all the 5 centres, was 36 USD in year 1 of operations and 28 USD in year 2 of operation. The model is completely replicable in other towns. All of its operating elements are independent of the town characteristics and can be built anywhere else and it is a matter of local priorities, willingness and resources to initiate such a project.