Monitoring, Evaluation & Learning (MEL) - WYG
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
creative minds safe hands
Monitoring, Evaluation &
Learning (MEL)
WYG’s dedicated Monitoring, We help funders and implementers ensure MEL addresses
accountability, learning, and evaluation needs – while
Evaluation and Learning (MEL) being able to share knowledge and communicate results
effectively. In response to evolving client needs, we also
division supports our clients and undertake non-fiduciary compliance and audit work, ensuring
their implementing partners programmes operate in accordance with gender, conflict
sensitivity, safeguarding and other sector specific guidelines
across the globe to undertake and obligations – while our work on result harvesting supports
programmes to adequately capture output and outcome
independent investigation results, and consider the strength of evidence underpinning
and evaluation, to design and result claims.
operationalise MEL systems,
and to improve MEL capability.
We believe that appropriate
and robust MEL inputs improve
the impact and sustainability of
development interventions, and
increase value for money (VfM).
Monitoring, Evaluation & Learning (MEL) www.wyg.comWe combine sector knowledge with in-depth MEL experience We pride ourselves on undertaking contextually appropriate
to advise on and undertake MEL activities at each stage of the and robust investigation and evaluation. We deliver MEL
programme lifecycle. We: support services which are tailored to programme needs,
• undertake investigation and draw on evidence to inform the and provide clear and practical recommendations that can be
design of new programmes operationalised to improve performance and sustainability,
while capturing learning to be shared more widely.
• develop and periodically update Theories and Change (ToCs),
result frameworks (logframes), VfM metrics and gender
strategies Where we work
• create, improve and help operationalise MEL strategies,
We work in fragile and conflict affected environments (FCAS)
systems and tools
as well as emerging economies middle- and income countries,
• put in place knowledge management and knowledge and our services can be tailored for the UK and US domestic
sharing systems markets. Our current flagship programmes include the largest
• help develop a culture of learning and reflection, and improve MEL contract awarded by HMG to date - the Evaluation and
operational processes to enable ongoing adaptation Learning of the £1.2B Cross Government Prosperity Fund - and
• design and undertake independent verification, monitoring we support the Conflict, Stabilisation and Security Fund in West
and evaluation, including undertaking qualitative fieldwork Africa to improve MEL across its portfolio of programmes in
and developing quantitative tools and interpreting data Mali, Nigeria, and Sierra Leone. We hold a portfolio of DFID
evaluations, and continue to grow our client base, working
• provide training and build capacity of partners.
with other donors and supporting UK and US private sector
WYG operates at the cutting edge of MEL, and continues to partners, non governmental organisations (NGOs) and funders
expand its use of technology solutions. We also develop new to improve MEL.
approaches to:
Our MEL team have wide sector experience, and partner with
• ensure appropriate MEL and reporting systems are in place in sector specific experts where required to ensure MEL solutions
complex, multifaceted and system approach interventions address the unique needs of each focus area – while aligning
• undertake thematic and across portfolio (not only project) with international best practice in monitoring, evaluation and
investigation and evaluation learning. Our current projects include work in the areas of:
• undertake developmental evaluation and ongoing learning, • governance and accountability
feeding in lessons ‘in real time’ to improve impact and • peacebuilding, security and justice
sustainability
• private sector and economic development
• evaluate programmes which claim to operate in a problem
• livelihoods and rural development
driven and adaptive or politically smart manner
• climate, green cities and energy
• evaluate ‘hard to measure’ outcomes and impacts, such as
soft power, influence, research uptake and secondary benefits • education .
• support donors to operationalise payment by results (PBR).
Monitoring, Evaluation & Learning (MEL) www.wyg.comOur credentials
WYG has an in-house MEL team of 25, a core pool of trusted associates and a wider network of sector and geographic experts.
This resourcing approach allows us to offer value for money to our clients, while ensuring we can provide the best sector specific
MEL solutions. Our staff are experienced development practitioners drawn from development advisory, capacity building, NGO
and academic backgrounds, and all our technical staff hold Master’s degrees in a related field. They are supported by experienced
in-house project management and financial experts specialising in development. Our team holds 15 different nationalities and
have worked across the globe.
WYG MEL operates in accordance with OECD DAC Quality Standards for Development Evaluation and hold membership of the UK
Evaluation Society and European Evaluation Society. We are committed to balancing rigour and explanatory power with qualitative
nuance and practical recommendations. In addition to operating in accordance with DFID quality assurance (EQUALS) guidance,
WYG is supporting DFID in the operationalising of adaptive management approaches, testing development evaluation approaches,
and informing thinking on the review of politically smart and adaptive programmes through our programmes.
We emphasise continuous professional development, and embrace a culture of learning and reflection. WYG MEL aims to
contribute to the broader debate on development policy and to share our experiences with others in the industry through
publications and attendance at events.
Delivering complex and portfolio level MEL
To deliver development at scale, aid interventions are increasingly becoming larger and lengthier in scope and scale. Donors
are recognising the timeframes required to achieve impact when dealing with problems which, at its core, requires a change in
behaviour, and understand the need to address challenges using a systems approach which considers the complex linkages and
interconnections at play. Together with an emphasis on achieving value for money and reducing overhead and management
costs, this means that development programmes are becoming bigger and more complex.
This has resulted in an increased demand for MEL services at portfolio level, and the need to improve MEL, knowledge
management and communication across workstreams, implementing partners, programmes and even government departments.
WYG has specific experience in undertaking MEL of long term, sizable and multifaceted programmes, and continue to develop
new solutions to support portfolio level MEL.
Nigeria, Sierra Leone,
Mali and Sahel region
Provision of MEL Services
for CSSF West Africa
2018 – 2021, DFID
Conflict and Security Niger, South Sudan, Madagascar, Yemen,
Pakistan, Nepal, Bangladesh, Cambodia, Myanmar
Southern Africa Independent Process Monitoring and Evaluation of the DFID
contribution to the UNICEF Programme – Accelerating Sanitation
M&E of Climate Resilience Infrastructure
and Water for All (ASWA) in Neglected, Off-Track Countries
Development Facility (CRIDF)
2014 – 2018, DFID / UNICEF
2017 – 2020, DFID
Water and Sanitation
Climate and Private Sector Development
Democratic Republic of Congo China, India, Brazil, Mexico, Colombia, South East Asia, Nigeria,
M&E Support Unit to the International South Africa, Kenya, Horn of Africa, Turkey
Security and Stabilisation Support Prosperity Fund Evaluation and Learning
Strategy (ISSSS) 2017 – 2021, FCO and cross-departmental HMG
2015 – 2018, DFID Infrastructure, Business Environment, Anti-corruption, Digital,
Conflict, security and peacebuilding Trade, Health, Future Cities
Monitoring, Evaluation & Learning (MEL) www.wyg.comProvision of MEL Services for CSSF West Africa; 2018-2021, DFID
Conflict and Security
Nigeria, Sierra Leone, Mali and Sahel region
WYG and our partners IMC Worldwide and First Call have been appointed as MEL advisors to the UK Conflict Stability and Security Fund (CSSF) in West
Africa. This £135M fund consists of a large portfolio of security & justice, serious organised crime, stabilisation, security sector and defence reform
programmes in Nigeria, Sierra Leone, Mali and the Sahel region, implemented by the FCO, DFID, MOD, Home Office and other HMG entities.
WYG is supporting HMG to identify MEL weaknesses in, and to make improvements to, both MEL systems and capacity. This includes ensuring
appropriate Theories of Change (ToCs), result frameworks (logframes) and VfM metrics are in place, and that robust MEL systems suited to the local
context and programme needs are developed and operationalised. WYG is also advising the fund on portfolio level MEL to measure effects and impact
of the fund as a whole, and is supporting independent verification and evaluation. Other services include undertaking reviews of programmes’ ability
to meet HMG and international gender, conflict and security, and other relevant standards and obligations, and measuring ‘hard to measure’ outcomes
such as influencing and soft power. Advise and support on generating a culture of learning and reflection and implementing politically smart and adaptive
approaches are also provided, as is support to inform the evidence base on what works and why in a number of thematic areas.
Prosperity Fund Evaluation and Learning; 2017 – 2021, M&E of Climate Resilience Infrastructure Development
FCO and cross-departmental HMG Facility (CRIDF); 2017 – 2020, DFID
Infrastructure, Business Environment, Anti-corruption, Digital, Climate and Private Sector Development
Trade, Health, Future Cities Southern Africa
China, India, Brazil, Mexico, Colombia, South East Asia, Nigeria, Climate Resilience Infrastructure Development Facility (CRIDF) is a DFID
South Africa, Kenya, Horn of Africa, Turkey supported programme working to provide long-term solutions to water
issues that affect the lives of the poor in Southern Africa. The focus is
The UK cross-government Prosperity Fund aims to remove barriers to
not on building short-term water infrastructure, but on working with
economic growth and promote the economic reform and development
organisations to show them how they can better build, manage and finance
needed to reduce poverty in partner countries. The Fund also looks to
their own water infrastructure. CRIDF is working with 12 different countries
create opportunities for international business including UK companies as a
in Southern Africa that share water resources. In so doing, CRIDF aims to
result of this economic growth, as a secondary benefit. The Fund’s budget
improve the lives of over 200 million people.
is £1.2 billion over 6 years across a portfolio of projects. The Prosperity
Fund forms a strategic proportion of the UK’s total Official Development WYG is designing and delivering the internal M&E of CRIDF. The M&E
Assistance (ODA) commitment of 0.7% of Gross National Income (GNI). Framework is underpinned by an adaptive management strategy. The M&E
It represents a shift in the ODA resources spent on assisting countries to Framework itself will not change during the programme lifespan, but the
grow and develop by focusing on multi-year programmes in middle-income tools and templates that support the framework will be adapted based
countries where 70% of the world’s poor live and where 60% of global on the learning from the process. The M&E system designed by WYG is
growth will come from by 2030 (OECD). WYG leads a three-company based on measurement of deliverables at the different stages of the results
consortium offering evaluation and learning services to the Prosperity Fund chain, as described in the CRIDF logframe; and at the activity, project and
across Asia, Latin America and Africa. programme level. WYG is collecting a mixture of quantitative data (through
tools like satisfaction surveys/scorecards and the ICF transformational
change tool) and qualitative data (through case studies designed to capture
policy influencing which has led to change, test ToC assumptions, identify
Independent Process Monitoring and Evaluation of
barriers to behavioural change and understand beneficiary perceptions).
the DFID contribution to the UNICEF Programme –
Accelerating Sanitation and Water for All (ASWA)
in Neglected, Off-Track Countries, 2014 – 2018, M&E Support Unit to the International Security and
DFID / UNICEF Stabilisation Support Strategy (ISSSS);
Water and Sanitation 2015 – 2018, DFID
Niger, South Sudan, Madagascar, Yemen, Pakistan, Nepal, Conflict, security and peacebuilding
Bangladesh, Cambodia, Myanmar Democratic Republic of Congo
DFID was supporting UNICEF’s implementation of the Accelerating Over a three year period, WYG served as the M&E Unit located within the
Sanitation and Water for All in Neglected, Off-Track Countries programme, UN Stabilization Mission (MONUSCO) in the DRC. From our office in Goma,
which involved delivery of water, sanitation and hygiene (WASH) both we helped implement the International Security and Stabilization Support
through direct implementation and strengthening of the enabling Strategy (ISSSS), the agreed framework to support the transition from
environment to improve and sustain levels of access to WASH services. The peacekeeping to peacebuilding in Eastern DRC. ISSSS is an umbrella of
aims and objectives of the IPME, being delivered by WYG, were to assess projects, funded and implemented by different donors, UN agencies, local
programme results and performance, and formulate recommendations to and international NGOs.
allow programme improvements to be made in real time. Given that women WYG designed and implement an M&E Strategy for ISSSS, which
and girls were disproportionately impacted by lack of access to WASH tracked progress of all ISSSS projects and the contribution of the ISSSS
facilities, the evaluation paid particular attention to gender issues. programme as a whole to stabilisation outcomes, in turn informing future
As part of the IPME, WYG provided customised capacity building programming. Support included advising ISSS implementing partners
support to UNICEF’s WASH country programmes to strengthen their on data collection and data quality, and interpreting portfolio level
monitoring and evaluation systems and ensure their relevance for data to provide a picture of progress and inform recommendations for
measuring changes over time. This involved harmonising DFID/UNICEF improvement. To ensure sustainable uptake of MEL, WYG also helped
WASH indicators and milestones to ensure reporting was consistent build stakeholders’ capacity in M&E and providing ad hoc and ongoing MEL
and meaningful; strengthening quality of data collection, and analysing technical assistance.
processes to measure value for money and sustainability. WYG also
conducted monitoring and verification of ASWA results and evaluated
ASWA’s processes to understand reasons behind the achievement/non-
achievement of results
Monitoring, Evaluation & Learning (MEL) www.wyg.comFacilitating adaptive management and real time learning
In response to the growing acknowledgement that traditional approaches to institutional reform have often had limited impact, a
movement to ‘do development differently’ arose. Three key approaches in particular have gained traction within the development
community in an attempt to increase impact through provision of more appropriate support: the need for programmes to (1) work
in a politically smart manner, (2) be problem driven, iterative and adaptive, and (3) be demand led.
A central requirement of these three approaches is the need for ongoing data to inform programme decisions at both operational
and strategic level – it is necessary to understand what is working and what not, the changes in partner needs, and developments
in the local political contexts that can help or hinder reform. As a result, the ability to generate information on an ongoing basis
(as opposed to periodically reporting on programme process and performance) is paramount.
Following its acquisition of LDP (responsible for the design and delivery of the highly successful LASER programme) WYG has
generated proven track of supporting and evaluating adaptive and politically smart programmes, and of supporting projects to
document and share learning.
Zambia Afghanistan
Independent Evaluation of the Zambia Strategic Support to the Ministry of Interior Affairs
Accountability Programme (ZAP) Phase Two (SSMI-2)
2015 – 2019, DFID 2016 – 2019, DFID
Governance Governance, Conflict and Security
Nepal Somaliland
Mid-Term Review of Nepal’s Accelerating Investment and Energy Security Resource Efficiency in Somaliland
Infrastructure (MTR AiiN) (ESRES) Real Time Learning (RTL) Process
2018 – 2019, Landell Mills/DFID 2015 – 2018, DFID
Investment and Infrastructure Energy
Monitoring, Evaluation & Learning (MEL) www.wyg.comIndependent Evaluation of the Zambia Accountability Mid-Term Review of Nepal’s Accelerating Investment
Programme (ZAP); 2015 – 2019, DFID and Infrastructure (MTR AiiN) 2018 – 2019, Landell
Mills/DFID
Governance
Zambia Investment and Infrastructure
Since 2015 WYG has been serving as the Independent Evaluation Partner Nepal
(IEP) on the DFID funded Zambia Accountability Programme (ZAP), a In partnership with Landell Mills, WYG undertook a mid-term review of
five year, £26.5 million programme, aimed at ‘improving accountability a DFID flagship adaptive programme – the Accelerating Investment and
and responsiveness – up and down – in the delivery of public goods and Infrastructure in Nepal (AiiN) programme. Following on from Nepal Centre
services’ in Zambia. for Inclusive Growth (CIG), one of the first projects to test an adaptive
ZAP is as a politically smart and adaptive programme and draws on approach, AiiN operates in a politically smart and demand driven way,
evolving knowledge of the local context to respond to and exploit changes addressing investment climate reform and infrastructure development
in the political environment. ZAP focusses on three thematic areas through three pillars of work.
(political processes, education and inclusive growth), with a cross-cutting WYG provided the strategic and technical leadership through the team
knowledge sharing element, targeted simultaneously at the local, provincial leader, and led on aspects related to adaptive programming and politically
and national level. Interventions in the education and political processes smart working. WYG was tasked with reviewing the Theories of Change
workstreams are spread geographically across the country. at portfolio and programme level, undertaking ToC refresher workshops
As IEP, WYG undertook a theory-based evaluation of ZAP to help DFID and making recommendations for improvement. WYG also developed an
consider if a multi-pronged, bottom up and top down approach to improve approach to review the quality and appropriateness of adaptation, and
accountability as delivered by ZAP was suitable, and to understand what considered both the appropriateness of the project management and
was working or not, why, how and in what contexts. Using a Development MEL approach and systems. Other inputs included evaluating the gender,
Evaluation (DE) approach, we generated ongoing learning through cyclic institutional development and stakeholder engagement inputs, and
investigations to help DFID-Z and its implementing partners to adapt and verifying outcomes to inform payment by result decisions.
refine ZAP by testing the intervention hypotheses; to support interrogation A year on from the MTR, we are undertaking a further review of the ToC,
and sense-making of evaluation findings; to monitor the impact of the proposing evaluation questions, recommending the revised MEL strategy
changes made to the programme as a result; and to document lessons. for the programme extension and advising on programme lesson learning
The IEP support itself was adaptive and evolving in nature, and as ZAP and knowledge sharing.
moved through its life cycle greater emphasis was placed on result
identification, in response to DFID-Z’s changing needs, with result
harvesting helping to ensure a detailed understanding of programme
achievements. In the last year of ZAP, we are focussed on exploring the
Energy Security Resource Efficiency in Somaliland
outcomes of interventions, documenting reflections and learning, and (ESRES) Real Time Learning (RTL) Process; 2015 –
providing recommendations to improve the design and delivery of future 2018, DFID
large scale, multifaceted and politically smart programmes
Energy
Somaliland
Energy Security and Resource Efficiency in Somaliland (ESRES) is a
Strategic Support to the Ministry of Interior Affairs
programme funded though the International climate Fund (ICF) to improve
Phase Two (SSMI-2); 2016 – 2019, DFID access, affordability and sustainability of electricity in Somaliland. ESRES
Governance, Conflict and Security is intended to support innovative renewable energy and energy efficient
Afghanistan technologies, governance approaches and business models to improve
access to electricity for essential institutions (especially hospitals and
WYG is leading the evaluation of Strategic Support to the Ministry of health centres/clinics) and local communities.
Interior Affairs Phase Two (SSMI-2) – the first problem driven iterative and
Using a locally-developed evidence base, WYG’s real time learning worked
adaptive (PDIA) programme operating in an FCAS environment – on behalf
with the ESRES programme during its Phase 1, focusing on updating the
of DFID.
programme ToC and improving implementation during its 30-month pilot.
SSMI-2 is a technical assistance project working within specific areas of The ESRES implementation programme itself was adaptive and the RTL
the Ministry of Internal Affairs (MoIA) in Afghanistan, aiming to achieve methodology mirrored this by adopting a flexible research and engagement
improvements in the Ministry’s capacity to manage its resources, with programme running alongside the activities of the implementing agent and
particular focus on the Afghan National Police (ANP). Our evaluation the wider stakeholder group (community institutions, community members,
examines what works, in what circumstances, why and the extent to which businesses, IPPs and service providers such as hospitals and schools). RTL
the results claimed by SSMI-2 can be evidenced and sustained. engagement activity included field-based hybrid mini-grid assessments,
WYG has been tasked with monitoring the performance and progress of qualitative focus group discussions with community representatives, key
the project through its lifetime, examine the effectiveness of the project informant interviews among all stakeholder groups and case studies. The
in supporting the Ministry of Interior Affairs (MoIA) in improving service RTL assignment aimed to generate robust evidence on the achievements
delivery of the Afghan National Police (ANP) through improved allocation and limitations of ESRES Phase 1. This guided programme implementation
and use of resources, understanding how the combination of interventions and contributed to the design of its expansion phase.
has led to those changes and what the critical elements for effective
implementation are; and collecting data, information and lessons to build
the evidence base. This has been done through extensive in country
qualitative data collection throughout the life of the programme.
Highlights of the SSMI evaluation has included an appraisal of the
programme’s Political Economy Analysis (PEA) and logframe, and an
assessment of the problem driven iterative adaptation (PDIA) approach
.
Monitoring, Evaluation & Learning (MEL) www.wyg.comConducting independent and theory-based evaluations
In addition to verifying programme performance and results, WYG is an industry leader in the evaluation of ‘difficult to measure’
outcomes and complex, multifaceted programmes. We often support DFID to verify not only what programmes have achieved
– but what worked (or not), when, why and how. Through theory-based evaluations, we help understand the requirements for
success, and are able to support the design of future programmes more likely to succeed.
Afghanistan Ghana
Evaluation of the Comprehensive Agriculture and Rural Market Development in the North of Ghana
Development Facility (CARD-F) Independent Evaluation
2015 – 2019, DFID 2014 – 2017, DFID
Agriculture and Rural Market Development Agricultural and Rural Market Development /
Private Sector Development
Nigeria Nigeria
Independent Evaluation for Promoting Pro-poor Independent Evaluation of the Stability and Reconciliation
Opportunities in Commodities and Services Markets Programme in Nigeria (NSRP)
(PrOpCom) Mai-karfi Programme in Nigeria 2013 – 2018, DFID
2013 – 2018, DFID Conflict and Security
Agricultural and Market Rural Development,
Private Sector Development
Evaluation of the Comprehensive Agriculture and Rural Development Facility (CARD-F), 2015 – 2019, DFID
Agricultural and Rural Market Development
Afghanistan
The evaluation of the Comprehensive Agriculture and Rural Development Facility (CARD-F) seeks to strengthen agricultural value chains, aiming to
increase rural incomes, create jobs and improve the trade balance mainly through import substitution in Afghanistan. The purpose of the evaluation is to
examine whether CARD-F Phase II achieved its goal to provide jobs and increase the incomes of beneficiaries, and have a wider positive effect on the
rural economy. The independent evaluation is focusing on two value chains: Poultry and Protected Horticulture (‘greenhouses’), which together constitute
74% of planned expenditure under CARD-F Phase II. The evaluation adopts a mixed methods theory-based approach employing Contribution Analysis,
examining the underlying theories of change for both value chains with both quantitative and qualitative data.
WYG has overall responsibility for delivery of this evaluation, working with our on the ground partner, Altai, and collecting data in four stages: a 2016
baseline, two midlines (2017 and 2018), and an endline in 2019. Data is being collected through enumerated surveys with beneficiaries and key informant
interviews with a wide range of stakeholders in the two value chains. The evaluation is also draws on secondary national-level data to understand
agriculture sector and economy-wide changes (e.g. on market price monitoring, trade figures). WYG designed the evaluation methodology and field
instruments, and have provided the Project Director, Quantitative Lead, and Conflict and Gender Specialists, Contribution Analysis Expert, and Data
Analysts to the CARD-F evaluation team. Altai, manages the field data collection while WYG leads on data analysis and production of key reports.
Monitoring, Evaluation & Learning (MEL) www.wyg.comIndependent Evaluation for Promoting Pro-poor Market Development in the North of Ghana
Opportunities in Commodities and Services Markets Independent Evaluation; 2014 -2017, DFID
(PrOpCom) Mai-karfi Programme in Nigeria;
Agricultural and Rural Market Development /
2013 -2018, DFID
Private Sector Development
Agricultural and Market Rural Development, Private Sector Ghana
Development Ghana MADE is a Making Markets Work for the Poor (M4P) programme,
Nigeria intended to stimulate economic growth and reduce poverty in the Northern
Savannah, through improved incomes and resilience of poor farmers and
Over the five year period (2013-18) of the PrOpCom (Promoting Pro-poor
small-scale rural entrepreneurs in northern Ghana. WYG used qualitative
Opportunities in Commodities and Services Markets) Mai-karfi programme,
and quantitative data collection and analysis to conduct a theory-based
we are delivering an independent process and impact evaluation of its
impact and process evaluation. The objective of the MADE evaluation were
achievements in relation to key OECD Development Assistance Committee
to: i) assess whether M4P delivered within the context of “thin markets”,
evaluation criteria.
such as the northern savannah can deliver benefits for poor producers
PrOpCom Mai-karfi operates under the M4P (Making Markets Work for the and entrepreneurs; ii) contribute to the evidence base on what works in
Poor) approach of changing market systems in order to address inequitable Northern Ghana; iii) inform the international debate on the value of the
growth. It builds on the positive experience of its predecessor programme M4P approach, and thereby feed into future policy and funding decisions
in Nigeria and aims to increase the incomes of 650,000 poor men and relating to tackling poverty reduction and achieving the MDGs; iv) inform
women in northern Nigeria by stimulating sustainable growth in selected DFID and Government of Ghana of sustainable approaches to developing
rural markets to make them more inclusive for poor people. market systems and which interventions have the greatest impact.
WYG’s evaluation combines quantitative and qualitative survey methods to The WYG evaluation of Ghana MADE drew on data collected by the MADE
assess the impact of the PropCom Mai-karfi programme. In relation to M4P, programme, as well as conducting fieldwork to fill gaps or supplement
our work is helping build an evidence base on the relevance, efficiency, existing data. WYG gathered qualitative data via key informant interviews
impact and sustainability of M4P programmes, providing guidance on what and focus group discussions to build up a picture of the programme in
measures are effective and thereby informing future developments. action, and to set out, revise, and substantiate or invalidate the programme
WYG drew on quantitative beneficiary level data, collected from 3,800 ToC. Contribution Analysis was used as an analytical framework to come
farmers and small rural enterprises, as well as qualitative key informant to robust conclusions about the programme’s contribution to outcomes.
interviews (KIIs), focus group discussions (FGDs) and case studies. This The evaluation framework focused on a series of ‘keystone nodes’ within
evidence is being used to build up a picture of the programme in action, market specific theories of change identified as central to the success of
and to set out, revise, and substantiate or invalidate the programme ToC. the M4P designed interventions. This innovative keystone node approach
The evaluation used contribution analysis to construct a ‘contribution provided both opportunities to learn about the MADE programme as well as
story’ and to come to robust conclusions about how and to what extent about the evaluation process itself.
PropCom did (or did not) contribute to observed outcomes. The evaluation
has been designed with flexibility built in since PropCom itself is a flexible
programme by design. Independent Evaluation of the Stability and
The evaluation provides in-depth analysis of the programme’s Reconciliation Programme in Nigeria (NSRP); 2013-
contribution towards raising household incomes, and seeks to understand
2018, DFID
the relationship between the M4P approach, poverty reduction and
systemic change within markets. Related knowledge and research has Conflict and Security
been published, including a paper on Warehouse Receipt Systems for Nigeria
food grains.
WYG undertook the evaluation of the Stability and Reconciliation
Programme in Nigeria (NSRP), a five year, £39m programme (2012-2017)
aiming to help reduce violence and improve stability in selected regions
of Nigeria (the northern states of Kano, Borno, and Yobo; the middle
belt states of Kaduna and Jos; and the southern states of Delta, Bayelsa,
and Rivers). This would be done by 1) improving conflict management
mechanisms, 2) mitigating against drivers of conflict, 3) increasing
participation of, and reducing violence against, women and girls and 4)
improving conflict prevention policy and practice. NSRP supported existing
or created new opportunities for state and non-state actors to engage
through multi-stakeholder’s platforms (MSPs). WYG used a hybrid theory-
based/case-based design, triangulating both primary data collected by
WYG (often in conflict affected areas) and internal NSRP monitoring data.
WYG developed a series of nine case studies looking at specific NSRP
interventions using RAPID Outcome Assessment (ROA) methods, with key
informant interviews and a ROA workshop used to collect data for the case
studies. WYG also gathered a mixture of qualitative and quantitative data
to assess participation and voice of multi-stakeholder platform members,
as well as the outputs and outcomes of platform activity. This data was
collected via a survey of 271 NSRP multi-stakeholder platform participants.
Social Network Analysis was subsequently carried out on the survey data
using the web-based application ‘Kumu’ to assess the network properties
of the platforms and their member organisations. Finally, WYG employed
Qualitative Comparative Analysis to explore the conditions associated with
NSRP platform success and to provide a programme level overview which
reinforced the insights from the case studies and platform survey.
Monitoring, Evaluation & Learning (MEL) www.wyg.comGenerating knowledge and research
HMG invests more than £1B in research every year, but the outcome and uptake of research can remain elusive. In response to
client needs, WYG is expanding its offering to support the uptake of research and the influence of policy, helping to position the
UK as an exporter of expertise. In addition, we help our clients to consider if and how research related work have led to change,
and how and why this has happened. We are also increasingly supporting programme to reflect on the knowledge contained in
interventions, and how this can be shared more widely.
Sub-Sahara Africa Western Balkans: Albania, Bosnia and Herzegovina,
(Ethiopia, Tanzania, Ghana, Malawi, Mali and Zambia) Kosovo, Macedonia, Montenegro, Serbia
Sustainable Agricultural Intensification Research and CSSF Target audience Analysis in Western Balkans
Learning in Africa (SAIRLA) (TAA WB)
2015 – 2020, DFID 2018 – 2019, FCO
Agriculture and Rural Development Media audience Analysis
Rwanda
Evaluation of Results Based Aid (RBA) in Rwandan
Education
2013– 2015, DFID
Education
Monitoring, Evaluation & Learning (MEL) www.wyg.comSustainable Agricultural Intensification Research and CSSF Target audience Analysis in Western Balkans
Learning in Africa (SAIRLA); 2015 – 2020, DFID (TAA WB); 2018 – 2019, FCO
Agriculture and Rural Development Media audience Analysis
Sub-Sahara Africa (Ethiopia, Tanzania, Ghana, Malawi, Western Balkans: Albania, Bosnia and Herzegovina, Kosovo,
Mali and Zambia) Macedonia, Montenegro, Serbia
WYG is leading the implementation of the Sustainable Agricultural WYG, in partnership with Ipsos, was commissioned to provide a robust
Intensification Research and Learning in Africa (SAIRLA) programme, which target audience analysis (TAA) in the Western Balkans that articulates the
seeks to generate new evidence and design tools to enable governments, perceptions and attitudes of individuals towards:
investors and other key actors to deliver more effective policies and • political, social and economic circumstances
investments in sustainable agricultural intensification (SAI) that strengthen • political actors (both domestic and foreign)
the capacity of poorer farmers’, especially women and youth, to access • the various outside parties actively engaged in the region
and benefit from SAI. As grant manager, SAIRLA oversees the use of (specifically towards the UK, EU, NATO, US, Russia, Turkey, the Gulf
£5M invested by DFID in eight research programmes operating across six countries and China)
countries on SAI research, while our partner National Research Institute • media consumption.
(NRI) supports technical quality assurance of research.
This research explored why and how these perceptions came about
WYG is also responsible for delivering technical support to five in-country and identify key influencers in terms of media sources in each of the six
learning alliances, which aims to bring together local stakeholders to Western Balkans (WB6) countries: Albania, Bosnia and Herzegovina (BiH),
support the uptake of research and the influencing of policy based on Kosovo, Macedonia, Montenegro and Serbia.
robust evidence. The cross-cutting nature of the research and social
The TAA provided an evidence base for future activity by HMG, thereby
learning allows SAIRLA to compare and contrast evidence, highlight
informing and improving HMG strategic communications and messaging in
potential synergies and identify SAI design tools that can be shared across
the region.
and adapted from country to country.
The Target Audience Analysis drew on IPSOS’ expertise to collect both
WYG leads on the effective communication, dissemination and sharing
qualitative and quantitative data to assess which audience the UK would
of lessons. In addition, WYG is also responsible for the design of and
be better placed to target for future programming, conducting quantitative
implementation of the programme monitoring and evaluation.
surveys, focus group discussion (FGD) and key informant interviews (KII).
In parallel, WYG conducted media monitoring between September and
November 2018 to assess the kinds of content produced by different
international actors and media channels. This also identified the main
Evaluation of Results Based Aid (RBA) in Rwandan
media channels and their respective audience numbers and demographics.
Education; 2013– 2015, DFID
WYG then analysed and triangulated all data and produced a final report,
Education including 6 annexes specific for each country analysed.
Rwanda
The objective of the Results Based Aid (RBA) programme was to pilot the
provision of additional results-based aid on improvements in the number
of students completing primary (P6), lower secondary (S3) and upper
secondary (S6) education; and the competency of teachers in Rwanda
to use English as the means of instruction. RBA was composed of three
key elements: payments based on results; recipient discretion on how
results are achieved; and verification of results acts as the trigger for
disbursement. Adopting a realist evaluation approach, WYG undertook
process and impact evaluations of the programme between 2013 and 2015.
The purpose of the WYG led RBA evaluation was to determine whether
the additional incentive of RBA impacted on completion at key stages
of education and additional teachers becoming competent in the use of
English as the medium of instruction. The evaluation also considered the
response of the Government of Rwanda and other key actors to RBA and
sought to establish the various processes that led to observed results.
The evaluation focused on impact; via an econometric model developed
by WYG, which explored trends in and factors affecting completion. The
evaluation also examined process; through annual literature reviews and
qualitative primary research conducted at national, district and school-
based levels to explore the response to RBA and the drivers of change
in relation to completion and English language proficiency. Lastly, the
evaluation also included in-depth modelling of the potential value for
money (VfM) of RBA.
P-0683/03/19
Monitoring, Evaluation & Learning (MEL) www.wyg.comYou can also read
