Global Evaluation of UNICEF’s Drinking Water Supply Programming in Rural Areas and Small Towns, 2006–2016

Background:

(See Executive summary within the evaluation report) UNICEF has long been a leading organization in the water supply sector, having led advocacy, policy and programme efforts in developing countries, in both humanitarian and development contexts. Drinking water supply became a standalone programme area in the UNICEF Strategic Plan 2014–2017. The UNICEF Executive Board approved the organization’s 1995 Water and Environmental Sanitation strategies and the subsequent WASH Strategy 2006–2015. In 2016, UNICEF adopted a new strategy to guide its work towards SDG 6. UNICEF develops drinking water supply services in the field for underserved communities and establishes the conditions for their adequate management and use. UNICEF also supports governments and the sector enabling environment at the national and global levels through policy advocacy and institutional strengthening. At the country level, the balance between downstream and upstream work and UNICEF’s various forms of engagement depend on the specific sector context, needs and capacities. Each UNICEF country office has a particular set of objectives and approaches that were developed with the host government and implements these in close collaboration with the government. Implementation partnerships are often established with non-governmental and civil society organizations, and through construction contracts with private sector service providers. At the beginning of its WASH Strategy 2006–2015, UNICEF was supporting water supply in more than 80 countries and reported 5.5 million direct beneficiaries of its development programming. The number of beneficiaries progressively increased while the geographical spread of UNICEF operations slightly reduced, with 10.6 million beneficiaries reported in 65 countries in 2016. UNICEF’s financial investment in water supply has steadily increased from 103 million USD in 2006 to 265 million USD in 2016, totalling approximately 1.7 billion USD over the period.

Purpose/Objective:

SDG 6 emphasizes the need for Member States and their development partners to scale up efforts, establish efficient and sustainable water supply management systems, ensure the quality of drinking water and adopt a stronger equity lens in order to leave no one behind. This global agenda for 2016–2030 provides UNICEF with an opportunity to reflect on the relevance and quality of its drinking water supply portfolio, which, until now, had not been evaluated at the global level. This evaluation assesses UNICEF’s global drinking water supply programming. Its aim is to fill specific knowledge gaps, draw lessons and contribute to improving the appropriateness of UNICEF’s strategies and activities at all levels of the organization. The findings will help determine how UNICEF can adjust its approach and optimize its contribution to the related SDG going forward. The results will also assure internal and external stakeholders regarding the level of performance and the quality of the programmes that they have supported and contribute to evidence-based learning in the water sector more broadly. The evaluation covers the timeframe 2006–2016, corresponding to the period of the UNICEF WASH Strategy 2006–2015, as well as the first year of the UNICEF WASH Strategy 2016–2030. It focuses on water supply for drinking purposes and on rural settings and small towns, which are UNICEF’s key areas of intervention. Purely emergency operations are excluded from the scope of the evaluation, as are non-community settings such as schools and other institutions.

Methodology:

UNICEF’s programming for rural and small town drinking water supply (abbreviated as ‘RWS’ in this report) is evaluated based on six criteria: effectiveness, efficiency, sustainability, equity, innovation and relevance. The evaluation follows a theory-based approach. It assesses the level of performance and UNICEF’s achievement against what was explicitly or implicitly intended in the UNICEF WASH Strategy 2006–2015, against the situation at the beginning of the evaluation period, or against sector comparators, standards and good practices. It unpacks the implementation and result chain and investigates the conditions of programme implementation and the underlying factors that influenced successes and failures. A colour-coding system is used to convey an evaluative judgment in a reader-friendly way. The evaluation relies on a wealth of quantitative and qualitative data. The evaluation team reviewed more than 2,800 UNICEF and non-UNICEF documents, datasets and systems; carried out an online survey to which 521 staff and partners from subnational, country, regional and global levels responded; interviewed 63 key informants; and conducted in-depth case studies and field visits in eight countries from various regions: the Plurinational State of Bolivia, Cambodia, Chad, Ethiopia, Guinea, Nepal, Rwanda and Zimbabwe.

Findings and Conclusions:

Based on the performance scoring system applied to the areas of inquiry chosen for this evaluation, relevance and effectiveness were the greatest strength of UNICEF’s RWS programming. Equity and efficiency were found to be moderate, while innovation and sustainability were the areas of relative weakness. Key findings by evaluation criteria are to be found in the text box at the end of each findings chapter. Overall, cross-cutting conclusions are presented at the end of the report and summarized below:

1. Numerous comparative advantages and strengths have made UNICEF a major and increasingly well-positioned and trusted partner in RWS.
2. UNICEF made significant, effective and increasing contributions to realizing the international development agenda, reducing geographic inequities and strengthening the sector enabling environment.
3. UNICEF only partially delivered on its corporate RWS commitments, primarily because the WASH Strategy 2006–2015 was not designed and rolled out to ensure that it could guide UNICEF’s action. Similar concerns remain for the WASH Strategy 2016–2030.
4. Although UNICEF’s RWS programming made significant progress in some of these areas, especially towards the end of the evaluation period, it lacked a holistic and systematic approach to equity, sustainability, efficiency, scalability and integration. A conceptual framework and deliberate strategy to achieve these objectives would have helped UNICEF maximize its contribution to global development.
5. Continuous improvements have been made in monitoring, reporting, evaluation and knowledge management. Persisting weaknesses, however, constrained UNICEF’s ability to effectively deliver on its commitment to accountability and to continuously learn and improve.

Recommendations:

Recommendations are summarized below and detailed at the end of the evaluation report.

Recommendations targeted to UNICEF Headquarters: 1. Develop global RWS programme guidance 2. Define a financing strategy 3. Establish global priorities, partnerships and incentives for policy advocacy, innovation and knowledge; these initiatives should focus on advancing equity, sustainability and scalable models for achieving universal access by 2030 4. Improve programme planning, monitoring, evaluation and reporting systems and processes for increased alignment with the SDG agenda 5. Adapt human resources to the evolving programme needs

Recommendation targeted to regional offices, in support to Headquarters and country offices: 6. Support and complement Headquarters-led initiatives and roll them out in priority countries 7. Strengthen regional capacity to support and oversee the quality of country programmes and their alignment with global priorities

Recommendation targeted to country offices: 8. Prioritize equity in RWS programming 9. Adopt a more systematic, holistic and sector-oriented approach to sustainability 10. Provide an integrated intervention package to beneficiary communities whenever necessary

Lessons Learned:

The evaluation generated important lessons not only for UNICEF’s RWS programming but also for UNICEF as an organization, for the UNICEF WASH section and for the broader RWS/WASH sector.

1. Overall, UNICEF has a good understanding of its organizational strengths and comparative advantages in the WASH sector, but is less clear on its added value and specific niche in RWS.
2. In a context where the global development agenda has considerably expanded in scope and became more ambitious, where the interdependence of development processes has become stronger, and where the relative contribution of donor assistance is becoming smaller, UNICEF’s positioning, credibility and success will also be determined by its strategic selectivity.
3. Many (but not all) of the weaknesses identified through the evaluative analysis were mentioned in the WASH Strategy 2016–2030 as priority areas for UNICEF to address.
4. There is some disconnect between UNICEF’s priorities and work at the global and country levels. Linkages are relatively weak between the UNICEF global WASH strategy documents and the RWS country programmes on the one hand, and between the UNICEF country situation analyses and UNICEF RWS country programmes on the other hand.
5. In this regard, the decentralized structure and decision making in UNICEF presents both advantages and disadvantages. The disadvantages have not yet been addressed through the establishment of appropriate processes and capacities at each level.
6. Taking up the challenge of SDG 6 will require increasingly complex and multidisciplinary approaches. Trade-offs need to be recognized, objectives prioritized, the inter-sector focus invigorated and the key underlying constraints tackled in a holistic manner.
7. Internal capacities, partnerships, innovation, knowledge and continuous learning are critical, interdependent, and will need to be strengthened to bring about transformational changes and set up both UNICEF and the sector for success.

You can find the following documents attached:

• Evaluation report with executive summary
• Annexes
• Executive summary in French
• Illustrated summary
• Headline findings infogram (UNICEF's Contribution to Water Sector in a Nutshell)
• 8 country case study reports: Bolivia, Cambodia, Chad, Ethiopia, Guinea, Nepal, Rwanda and Zimbabwe.