Develop a monitoring, evaluation and learning strategy

How will you measure change and impact?

Select an evaluation strategy

Selecting an appropriate evaluation strategy is essential for generating credible and reliable evidence to answer questions about a programme’s performance and impact. This evidence can be used to improve the current programme, develop plans for scale-up, and inform the design of future programmes. An evaluation strategy should be identified early in the programme design phase to ensure sufficient budget, time and resources are allocated to fulfil the scope and complexity of the evaluation. VAWG programmes often involve non-linear and complex change. Evaluating programmes of this nature requires innovative approaches and tools that capture contribution as well as attribution and unexpected impacts. Evaluations of VAWG programming should also recognise that in contexts of resistance and the roll-back of rights, holding the line on women’s rights can be a success in itself.

Guiding Principles

Survivor-Centred Approach

Do no harm approach

Leave No One Behind, Equity and Non-Discrimination

Spotlight Initiative

Approach and Learning

All Spotlight Initiative programmes were expected to conduct mid-term evaluations as well as final evaluations. In addition a number of global level and thematic assessments were conducted to identify learning and impact. Key approaches and learning included:

Conducting mid-term assessments to promote lesson learning within programmes. Programmes were expected to conduct Mid-Term Assessments while a programme was ongoing. This was to support programmes to identify opportunities for improvement, consider lessons learned, and develop recommendations to inform the development of the next phase of programming. A variety of data collection methods was used, including key informant interviews, focus group discussions, document reviews, and online surveys.

Conducted evaluations at the global level to promote learning for future programming. For example, Spotlight Initiative engaged in a global level final evaluation to assess the impact of Spotlight Initiative overall. The main objective of the final evaluation is to assess the Initiative’s overall performance, inclusive of its design (notably its commitment to UN Reform and the principle of Leaving No One Behind), its management and operations (the extent to which it is “fit for purpose”) and the results it has achieved (vis a vis its global results framework and other measures of success). The evaluation is based on document review, eight country case studies, five regional case studies and five non-Spotlight Initiative/counterfactual country case studies, key informant interviews and targeted questionnaires.

Commissioning thematic assessments to learn about progress on cross-cutting commitments: Spotlight Initiative also commissioned a global thematic assessment focusing on three key themes: (1) the meaningful engagement of civil society; (2) the implementation of the commitment to ‘leave no one behind’ (LNOB); and (3) support for strengthening women’s movements and cross-movement building. This assessment takes an in-depth look at how Spotlight Initiative has performed against these three themes, recognised as critical for more transformative, intersectional and sustainable approaches to ending VAWG. The thematic assessment is based on a document review and ten country case studies.

Commissioning an review of the Spotlight Initiative model: In 2021, an independent study, Imperative to Invest, was conducted to assess the impact potential of addressing VAWG through models like Spotlight Initiative. This used a mixed-method analysis to arrive at a data-driven perspective. It estimated that the Initiative’s comprehensive model is up to 90% more effective in preventing violence than other models.

Top Tips

How to select an evaluation strategy - top tips based on learning from the wider sector.

Click a tip for more information.

Consider which evaluation strategy is most appropriate for your programme

Develop clear evaluation questions

Consider different evaluation approaches and methods

Once a programme has identified the specific questions it seeks to answer, the next step is to select the most suitable type of evaluation. There are three types of evaluation design that could be considered: Non-Experimental Evaluations, Experimental/Randomised Control Trials, and Quasi-Experimental Evaluations. There are benefits and disadvantages to each of these approaches, which are detailed in the following table.

Evaluation Design	Considerations
Non-Experimental Evaluations	These evaluation approaches involve comparing the results from studies carried out at the start of an intervention, at the mid-point of an intervention and at the end of an intervention. While less ideal, it is also possible to do an ex-post assessment, once the programme is finished. They are often best suited to situations where it is not ethical to have a comparison or ‘control group’ that does not receive an intervention. These approaches are often cost-effective, may use both quantitative and qualitative methods, and can capture non-linear and complex change. Because these approaches do not involve a ‘control group’, they are unable to attribute a particular outcome to an intervention because they cannot be sure that other factors were not involved in this outcome. However, they are able to show where an intervention has likely contributed to an outcome.
Experimental/ Randomised Control Trials	This evaluation approach involves taking two groups of people who are comparable in some relevant respects (such as education level, geography, income, urbanisation rate), and randomly assigning one group to receive an intervention, while the other ‘control group’ does not. By monitoring the difference in outcome between the two groups, programmes can better attribute change to an intervention rather than to other factors. Randomised Control Trials (RCTs) may be considered the ‘gold standard’ approach in some evaluation contexts but are not always appropriate for evaluating programmes to end VAWG. This is because they often require large sample sizes, are costly, time-consuming, complex to design, implement, monitor and quality control. In addition, there are ethical concerns about providing initiatives to end VAWG to one community, while withholding it from another for the purposes of rigorous evaluation. That said, these ethical issues can be mitigated against, for example through phased roll-out, by providing a waiting list where ‘control communities’ receive the intervention later down the line. However, these arrangements rely on evidence of effectiveness and continued funding. Efforts to establish RCTs to evaluate programmes to end VAWG should think carefully about the ethics and logistics of these evaluations. In particular, they should consider how to provide referral pathways and support to control communities.
Quasi-Experimental Evaluations	These evaluations are similar to RCTs in that they involve a comparison group, but the groups are not randomly chosen. Instead, they are selected based on some other criteria such as convenience, accessibility, or another targeted characteristic. These evaluation approaches face similar ethical and logistical challenges to RCTs. They include but are not limited to Regression Discontinuity Design and Instrumental Variable approaches.

Consider which evaluation method(s) are appropriate

Once a programme has identified the overall approach of their evaluation, the next step is to determine which evaluation method (or methods) will be best suited to this approach and the questions the programme is trying to answer. There are a variety of methods for conducting evaluations, detailed in the following table. It is important to distinguish between evaluation methods that can capture instances of correlation (where there may be connection between two events) and causation (where one event is shown to have caused another event).

Method	Details	Examples
Quantitative methods	These involve closed-ended questions and usually capture numerical data. They are well placed for identifying whether change is occurring and the scale of that change. This data can also be analysed to identify patterns of change. However, they are less able to capture nuance around why change is happening. Quantitative data can be collected from large numbers of people, for example, through online tools, which can increase the credibility of findings.	Randomised Control Trials Questionnaires with closed-ended questions Large data sets such as Demographic and Health Surveys (DHS)
Qualitative methods	These involve open-ended questions and are used to gather in-depth information and insights from participants. This data can be used to understand how and why change happens over time, what contributed to this change, the significance of the change, and the unintended impacts of programming. Qualitative data is usually time-consuming to collect and so often involves smaller sample sizes.	Focus Group Discussions Key Informant Interviews Questionnaires with open-ended questions Outcome harvesting Most Significant Change
Mixed Methods	This approach draws on both quantitative and qualitative methods and sources to improve the nuance, credibility and reliability of findings.
Participatory methods	This approach involves collaborating with and meaningfully engaging affected populations as rights holders, change makers and co-producers of knowledge. It centres survivors’ and practitioners’ perceptions, understanding and learning. Affected populations are involved in the design of evaluations and help to set the questions asked. The process respects their agency and their right to be involved in the evaluation of programmes that impact them.	Body Mapping Time sequencing Photo voice Participatory mapping and power analysis Journal discussions

Ensure the evaluation process adheres to high standards

View Sources

Case Studies

Conducting a Mid-Term Assessment in Malawi, 2021

Spotlight Initiative

In 2021, the Spotlight Initiative team in Malawi conducted a mid-term assessment to take stock of the first phase of the Spotlight Initiative programme, which ran from 2019-2022. The aim was to assess the relevance, efficiency, effectiveness, and sustainability of the programme and to develop relevant recommendations to improve project implementation.

The evaluation team conducted Key Informant Interviews (KIIs) and Focus Group Discussions (FGDs) with 62 individuals. These were spread across national and sub-national levels and included six government officials, three EU Delegation staff, 12 representatives from UN organisations, two members of the Civil Society National Reference Group, six implementing partners, and four other stakeholders including the Chairperson of a Media Network.

In addition, the evaluation team conducted four FGDs with 30 beneficiaries. For example, to assess their work supporting out-of-school boys and girls of primary school age, they interviewed five boys and seven girls involved in their programme. They interviewed the boys and girls separately to promote free expression of their perceptions and views. In addition, to assess Spotlight Initiative’s work developing a Community of Practice on child protection, the evaluation team conducted a focus group discussion with one chief, two representatives of a Community Victim Support Unit, one Child Protection Officer, two Mentors, and one Chairperson from the Village Savings Loans. This was the only focus group discussion where there were both male and female participants. This decision to have a mixed-gender FGD was made following feedback from these individuals that having a mixed group would not affect their ability to share opinions.

To supplement the KIIs, the evaluation team also developed an online survey to gather views about and experiences of the programme. This was responded to by 57 individuals including 14 government officials, 22 representatives from UN Agencies and 21 implementing partners. It was also supplemented by a document review that included key programme materials and an internal mid-term review that was undertaken in 2020.

Challenges: The Spotlight Initiative evaluation team faced challenges in data collection due to the limited time available. This led to multiple interviews being rescheduled due to delays. The flexibility of participants was essential to enabling an additional eight interviews to take place. In addition, the evaluation team were unable to review certified financial reports for the final quarter of 2020, meaning that the document review was incomplete.

Impact: The Spotlight Initiative team developed a proposal for amendments to the programme based on the findings from the Mid-Term Assessment. This proposal included the development of a Malawi Grassroots Action Plan (GAP), to support the capacity building of grassroots actors partnering with Spotlight Initiative. It also included scaling up the use of mobile legal assistance in response to the challenges to accessing assistance during COVID-19. The Spotlight Initiative team presented this proposal along with the findings from the Mid-Term Assessment to the National Steering Committee. The National Steering Committee supported the proposal and provided recommendations that the programme 1) conduct consultations on sustainability to further develop the sustainability strategy, 2) strengthen involvement of the central level ministries, 3) and conduct orientations on the new proposal at district level to ensure understanding and buy-in at Government level.

Region: Africa

Stakeholders: National government, Civil Society Organisations, Community leaders

Resources and Key Links

Spotlight Initiative Resources

Key Links

Better Evaluation (n.d.) Evaluation Methods
International Training Centre (2021) 14-point checklist for effective monitoring and evaluation within the UN
Participatory Methods (n.d.) Useful Methods and Ideas
UN Women and WHO (2020) RESPECT Framework Monitoring and Evaluation Guidance
WHO et al (2023) Checklist for ensuring the quality of violence against women surveys

Questions	Reflections
Are there processes in place to ensure the findings from the evaluation strategy inform decision-making processes, organisational learning, accountability and stakeholder empowerment?
Does the evaluation strategy ensure the evaluators are able to conduct an independent evaluation to ensure findings are impartial, objective, unbiased and credible?
Is the evaluation strategy in line with ethical standards that respect human rights and gender equality?
Is the evaluation transparent and does it support accountability to communities, stakeholders, partners and donors?
Does the evaluation strategy promote universal principles like human rights and gender equality?
Does the evaluation strategy include efforts to build national MEL capacity?
Does the evaluation strategy adhere to high levels of professionalism?
Has the programme appointed a competent head of evaluation to support the planning, design and management of an evaluation?

Learning Centre

Select an evaluation strategy

Select an evaluation strategy

Approach and Learning

Top Tips

Case Studies

Resources and Key Links

Related Pages

Spotlight Initiative Resources

Key Links