2019 Myanmar: Formative Evaluation of MMCT in Chin and Rakhine States
Sustainable Development Goals: 3
- SDG 3 - Good Health and Well-Being
The Ministry of Social Welfare Relief and Resettlement (MSWRR), Government of Myanmar, through the Department of Social Welfare (DSW), is leading the social protection agenda outlined under the National Social Protection Strategic Plan. The DSW is in charge of the implementation of the key flagship programmes - one of which is the Maternal and Child Cash Transfer (MCCT) programme. The MCCT programme began in Chin State in June 2017, later expanded to Rakhine State and Naga land in January 2018 and further expanded to Kayah and Kayin States in October 2018. The ultimate objective of the MCCT programme is to improve nutritional outcomes for all mothers and children during the first critical 1,000 days of life, from conception to 24 months of age, that can perpetuate an intergenerational cycle of poor nutritional status. In line with this, the MCCT programme aims to empower pregnant and lactating women with additional purchasing power (MMK 15,000 per month/10.5 USD) to meet their basic needs during the first 1,000 days, along with complementary awareness sessions on nutrition, health and hygiene. All pregnant women who enrol in the programme continue to receive programme benefits until their child reaches the age of 24 months. As of October 2019, the MCCT programme in Chin State had registered 33,723 women beneficiaries and made 13 bi-monthly payments. The Rakhine MCCT programme was rolled out in January 2018 and as of October 2019, the State had made 9 quarterly payments and covered 124,719 beneficiaries across the State.
Evaluation purpose, objectives and intended audience
The Country-led Formative Evaluation of the Maternal and Child Cash Transfer Programme in Chin and Rakhine States in Myanmar was commissioned by UNICEF Myanmar, in partnership with the Department of Social Welfare. The primary purpose of this formative evaluation was to foster learning and improvement within the MCCT programme by reviewing the design and implementation modalities of the program. It also sought to set out lessons learned (from the MCCT programme and other cash transfer interventions in the region) to strengthen the programme in Chin and Rakhine to set the ground for scaling-up.
Being country-led, this evaluation is integral for creating a culture of utilizing evidence and increasing accountability - which is a key factor in establishing capacities and systems so that the government can evaluate its development activities in the future. The primary user of the evaluation will be DSW, MSWRR - being the lead agency for the MCCT programme as well as implementing departments i.e. the Department of Public Health (DoPH) of the Ministry of Health and Sports (MoHS) and the General Administration Department (GAD). The findings of the evaluation can also be useful to relevant social protection development partners like UNICEF, LIFT, the World Bank, WFP, Save the Children and IRC, among others. Secondary users will include other government agencies and civil society organizations involved in cash transfer programming in Myanmar, UNICEF’s Regional Office for East Asia and the Pacific (EAPRO) and others.
The key objective of this evaluation was to analyse the extent to which the MCCT programme has been appropriately designed and the efficiency and effectiveness of its implementation. The evaluation further sought to understand the usage of the cash transfer amount, the effectiveness of social and behaviour change interventions and whether the cash spending is translating into the intended objectives of the program. Other objectives of the evaluation include an assessment of the MCCT institutional capacity at various levels and an analysis of the effectiveness of support by the development partners.
The scope of this evaluation is formative (learning-oriented) in nature. The evaluation was not intended to be impactful or summative; it looked at evaluating the processes, procedures, implementation mechanisms and finding out whether beneficiaries in Chin and Rakhine States were satisfied with the services provided under the programme from 2017 till date.
A non-experimental research design as well as theory-based mixed methods and utilization-focused approach were followed – combining quantitative and qualitative primary data collection while drawing upon inference from key programme documents including policy, design and implementation documents. Primary data collection tools included Key Informant Interviews (KIIs), Focus Group Discussions (FGDs), Beneficiary Surveys and Case Studies. An important methodological aspect of this evaluation was its participatory and learning-oriented nature, involving stakeholders in the design and development of the evaluation process through interviews at both national and sub-national levels. Further, workshops were conducted, at both the inception stage and after the preliminary analysis of the findings, with the government, implementing bodies (at National and State level) and development partners, which helped validate and triangulate findings, obtain a wide range of perspectives and yield insights into the dynamics of the program. Another workshop to present the final evaluation report is planned.
Given the scope, a multi-stage systematic random sampling method was adopted. All 5 districts in Rakhine and 4 districts in Chin were covered as part of beneficiary survey. A sample of 409 beneficiaries in Chin and 463 beneficiaries (including beneficiaries in IDP camps) in Rakhine were reached. The evaluation team made serious efforts to make the sample as representative as possible and hard-to-reach areas (Paletwa township) and conflict prone zones (Maungdaw township) and 4 IDP camps - one each in Myaypon and Kyaukphyu and two in Sittwe were included to get an accurate picture of the beneficiary experience. Along with beneficiaries, the insights of spouses and community members were gleaned through FGDs. The evaluation team was able to complete data collection under challenging circumstances including heavy rains, landslides and security risks due to armed conflict in certain areas.
Analysis was conducted using the modified Organisation for Economic Co-operation and Development (OECD)/Development Assistance Committee (DAC) criteria of relevance, efficiency, effectiveness and sustainability as well as equity, gender equality and human rights considerations. The United Nations Evaluation Group’s (UNEG) ethical considerations and the evaluation guidelines guided the evaluation team.