FROM IMPACT MONITORING TO IMPACT ORIENTATION - For effective development and humanitarian aid
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
POLICY PAPER | 2020 FROM IMPACT MONITORING TO I MPACT ORIENTATION For effective development and humanitarian aid
From impact monitoring to impact orientation CONTENTS Introduction 3 1. Increase flexibility and make change the norm 4 2. Enable long-term projects 6 3. Emphasise approaches based on partnership and participation 7 4. Promote training and build knowledge 8 5. Use digitalisation r esponsibly 10 6. Take account of civil society’s scope for action 12 VENRO Members 13 Imprint 15
From impact monitoring to impact orientation INTRODUCTION Non-governmental organisations (NGOs) want The association and its members have learned their project activities to make a positive impact. a great deal in recent years through discussions This goal forms an intrinsic part of their identity. about how to achieve outcomes and impacts. Even But good intentions alone are not enough. In the choice of words reflects this point. In 2010, recent years, many VENRO member organisations we were still talking about “impact monitoring”. have built up considerable capacities for outcome We were primarily concerned with methods and and impact orientation. They have changed their instruments for monitoring and evaluation. At that structures and processes to implement outcome time, the VENRO policy paper “Quality before and impact orientation on all levels and in all areas Proof ” triggered a discussion about the goals and of the organisation. NGOs also encourage similar orientation of impact monitoring in development processes among their partner organisations in cooperation. Through their engagement with the countries of the Global South. outcome and impact orientation, VENRO members have learned that it is not simply a matter As an umbrella organisation, VENRO actively of monitoring, but also of a strategic orientation supports its member organisations in their towards outcomes and impacts in all phases of efforts to implement outcome and impact project work. Outcome and impact orientation oriented working methods. We hold training should form an integral part of the organisation’s events, facilitate experience sharing via our management system. impact orientation working group, and engage in professional debate. What is impact orientation? monitoring and analysis as well as learning The ↘ VENRO code of conduct from impact analyses and evaluations.” [...] “Transparency, organisational management “Outcomes and impacts in this sense refer and monitoring” defines outcome and impact to the changes that can be attributed to the orientation as follows: “Outcome and impact project or programme. They may be short- orientation consists not only of measuring term, long-term, planned, unplanned, positive outcomes and impacts, but also refers to the and negative.” organisation’s work methods and attitudes. Outcome and impact orientation comprises Alongside the usual goals of outcome outcome and impact based planning, and impact orientation, such as learning,
From impact monitoring to impact orientation management, transparency and accountability, into the organisational processes of outcome VENRO member organisations have other and impact orientation, and take ownership key objectives: the empowerment of target of the projects. groups as well as participation and ownership by local groups. Outcome and impact The goals of outcome and impact orientation orientation should therefore be designed to in the work of NGOs are explained in detail boost empowerment and self-efficacy for in the ↘ VENRO policy paper from 2010 people in the projects. To achieve this, project “Quality before Proof ”. staff and target groups should be integrated This policy paper summarises the experiences of particularly the German Federal Ministry for VENRO member organisations with implementing Economic Cooperation and Development and the outcome and impact orientation in their work. German Foreign Office as they provide funding It is aimed at both NGOs and donor institutions, that supports the work of NGOs. 1. I NCREASE FLEXIBILITY AND MAKE CHANGE THE NORM NGOs in the North and South often find during Funding stipulations imposed by donor insti the course of a project – via impact monitoring – tutions are often rigid, making it more difficult for that their original plans are not working out. To NGOs to be as flexible as they need to be. Long reach their goals, project managers therefore have drawn-out decision-making and coordination to abandon or change paths that were initially processes can impede or delay project success. planned and have already been taken. NGOs should explain and justify strategy changes, but they should not be held up by cumbersome, NGOs still frequently fail to adapt to changed time-consuming application processes when they circumstances. There are both internal and external need to adapt their activities and strategies. In reasons for this. Within NGOs, there is sometimes some cases, this inflexible approach results in a belief that making changes in the project could donor institutions funding the agreed activities but cast a negative light on those in charge of planning, not achieving the agreed outcomes and impacts. or on the NGO as a whole. Project teams are afraid of looking unprofessional if they bother donor The same applies to budget planning. To allow institutions with change requests during the project project adjustments, it must be possible to term. Sometimes NGOs are also put off by the use funds flexibly. Small NGOs in particular amount of bureaucracy involved in making changes. rarely have the financial resources to prefinance
From impact monitoring to impact orientation necessary changes until a change request is Recommendation to the German Federal approved. Moreover, they cannot take the financial Ministry for Economic Cooperation and risk that funding for the modified measures may Development and German Foreign Office: not be approved. →→VENRO calls on the donor institutions to allow When it comes to reporting, this means less greater flexibility. Funding should be focused on attention should be given to measures that have the project outcomes and impacts. The donor been carried out. Reporting should instead focus institutions should transfer decision-making primarily on the results of the project strategy powers for activity and strategy adjustments, and the achieved outcomes and impacts. including budget adjustments, to the project- executing organisations. Tools for outcome and impact with changes and respond to changing oriented working political or environmental conditions. The In many organisations, the theory of change adaptive management approach essentially has proven to be one of several suitable focuses on rapid learning processes, flexible approaches for outcome and impact oriented implementation, and more delegation of working. Using the theory of change, project decision-making at local level. Under this teams can produce an outcome and impact approach, the question “Are we doing the model in which they set out in detail and right thing to reach our goals?” comes before constantly review the path from project the question “Are we taking the steps we activities to the project goal. This outcome planned to take?” and impact model goes beyond the linearity of previous such models. With the theory of In the context of emergency measures and change approach, before the project begins, fast-changing circumstances, so-called real- project teams no longer only describe what time evaluations, are increasingly proving they are doing, but also how they expect this to be a useful project monitoring tool. Real- to work and why. The theory of change shows time evaluations focus on implementation, how and why the project actually contributes rapid learning and the adaptation of activities to economic, environmental or social change. and project strategies. They are a particularly fast way for project teams to assess Adaptive management can be a useful implementation, learn quickly, and adapt the tool that enables NGOs to deal flexibly project to changing conditions.
From impact monitoring to impact orientation 2. E NABLE LONG-TERM PROJECTS Through their projects, NGOs often aim to put Recommendation to the German Federal sustainable structures in place and change attitudes Ministry for Economic Cooperation and and behaviour. Changes of this kind take time. To Development (BMZ) and German Foreign achieve and demonstrate long-term impacts on the Office: target groups, long-term partnerships and longer project periods or successive project phases of up →→ VENRO calls on the donor institutions to to twelve years are necessary. It should therefore be facilitate the funding of long-term projects. possible to consider and plan at least two follow- up project proposals from the beginning of the project onwards. The outcome and impact matrix and corresponding horizon should also be adapted to the up to twelve-year project duration from the outset.
From impact monitoring to impact orientation 3. E MPHASISE APPROACHES BASED ON PARTNERSHIP AND PARTICIPATION NGOs encourage their partner organisations in aspects. Standard indicators can reveal trends the South to develop and pursue their own and therefore be a useful addition to outcome understanding of outcome and impact orientation. and impact monitoring – especially if the target It has proven to be useful for target groups to groups also consider them to be useful. define their own expectations about project outcomes and impacts, and say how they will However, a given standard indicator may conflict know whether these outcomes and impacts have with the participative approach. It could weaken been achieved. Therefore, for VENRO member target groups’ participation and empowerment. organisations, the following questions are essential: Standard indicators convey the impression that How do the project’s target groups see the out the donor institutions or NGOs know what comes and impacts? What changes for them? And changes are best for local communities. This above all, what do they want to see changed?1 reinforces the imbalance of power between Northern NGOs and their partner organisations. At the same time, there has been a greater focus NGOs therefore need to be able to decide for on NGO reporting and accountability in recent themselves whether to use standard indicators, years. This focus has come about in part as a depending on the situation. result of the aid effectiveness debates, but also because of increasingly critical enquiries from Recommendation to the German Federal the public. NGOs are called upon to report Ministry for Economic Cooperation and on the outcomes and impacts of their work, Development (BMZ) and German Foreign and to demonstrate their successes. They use Office: quantitative and qualitative indicators for this purpose, and it can be helpful to use standardised →→ VENRO calls upon the donor institutions to indicators. A standardised indicator is one which refrain from making standard indicators a has a common definition and is used consistently requirement for project funding. in multiple projects or programmes in a particular field, sector or region. Furthermore, its data must be collected using the same method. The Relevant international processes advantage of standardised indicators is that they In line with governmental aid effectiveness do not have to be developed from scratch for agendas of recent years, civil society each project, and they record changes in the same organisations around the world have published principles and guidelines setting out what can be expected of them in terms For example, with the involvement of VENRO, the of working efficiently and implementing “NGO IDEAS” project developed tools and methods for impact monitoring and evaluation that are effective projects. In VENRO’s work, this attuned to the participative and open values of is reflected for example in the ↘ VENRO NGOs. guiding principles for developmental projects
From impact monitoring to impact orientation and programmes which are based on the In humanitarian aid, too, existing quality Istanbul Principles elaborated by the CSO standards have been revised in a broad Partnership for Development Effectiveness. international consultation process, and are This international platform for civil society summarised in the Core Humanitarian organisations, mainly from the Global Standard (CHS, 2014). This sets out nine South, has strongly influenced VENRO’s commitments that NGOs can use to improve examination of and engagement with outcome the quality and effectiveness of assistance and impact orientation. In Istanbul, following they provide to communities and people a two-year global debate, NGOs agreed on affected by crisis. To a limited extent, the eight “development effectiveness principles” CHS also describes outcomes and impacts, in 2010. The Istanbul Principles are quality for example in terms of strengthening principles for development processes and resilience and the importance of participatory practices, which VENRO has adopted in its project approaches. It is recognised by local guiding principles for developmental projects and international humanitarian NGOs, the and programmes. They are an expression of International Red Cross and Red Crescent the basic understanding shared by all VENRO Movement, and the United Nations. members. 4. P ROMOTE TRAINING AND BUILD KNOWLEDGE Over the past ten years, many NGOs have Outcome and impact orientation in NGOs requires placed greater emphasis on outcome and a willingness to embrace continuous learning on impact orientation in their own organisations. the part of staff members and partner organisations. They have set up monitoring and evaluation They have to be able to assess what is happening in units, for example, or modified application projects and in their environment, respond to new and reporting formats for their partnership developments, and work out appropriate changes. projects. NGOs invest a great deal of time and Project managers both shape events with their plans money in training their staff – both in Germany and are inspired by events to devise new plans. and in local partner organisations. Learning Not every NGO employee in the North or South processes take place in concert with the research is adept at these management tasks. Therefore, to community, governmental organisations and implement outcome and impact orientation in their other NGOs, and should be expanded further organisations, NGOs must provide financial and in the future. personnel resources for training.
From impact monitoring to impact orientation Once a project has been completed, NGOs and Recommendation to the German Federal their partner organisations in the Global Ministry for Economic Cooperation and South should know what impacts the project’s Development (BMZ) and German Foreign achievements continue to have. This requires time Office: and money beyond the end of the project, for example to carry out ex-post evaluations. →→ VENRO calls on the donor institutions to finance NGO staff in Germany who facilitate outcome and impact oriented project monitoring with the partner organisations. To this end, the donor institutions must make allowances in their funding for training and boosting the capacities of German NGOs in respect of outcome and impact orientation. The importance of evaluations in their work. During the course of projects, outcome and impact orientation interim evaluations can be very useful for Since the VENRO policy paper on impact steering projects in the desired direction. monitoring (“Quality before Proof ”) was Depending on the context, questions being published ten years ago, it has become clear asked and financial resources, the people that the most rigidly demanding methods carrying out the project can conduct the and evaluations are not necessarily the most evaluations themselves, and incorporate effective. In particular, this applies to so- the findings into future planning processes. called rigorous impact evaluations. Their Final evaluations and the resulting rigorous methods, including control groups recommendations can encourage further that do not form part of the project, should action by target groups, but mainly serve the be viewed critically in terms of cost and NGO learning process. They also serve to effort versus benefit. Furthermore, such demonstrate accountability to donors, the methods neglect key working principles such public and donor institutions. as empowerment, ownership and shared learning. Most actors today employ a mix of For NGOs to make use of the findings methods. Rigorous impact evaluations are and recommendations from individual one approach among several whose benefits evaluations, they need to be able to for NGOs are often limited. They tend to be analyse the quality of data, methodological used rarely, for example when NGOs want to weaknesses, and informative value of study the effectiveness of particular project those evaluations. Meta-evaluations help approaches. them to do so. International institutions and scientific institutes in particular Nevertheless, evaluations are an important seek to promote learning beyond an part of an outcome and impact oriented individual project by means of systematic work method. They enable NGOs to learn, reviews. These reviews provide a scientific and can help to improve the quality of analysis of many high-quality evaluations,
From impact monitoring to impact orientation and a summary of the findings. So far, of little use to NGOs because they are not unfortunately, these analyses have been sufficiently practice-oriented. 5. U SE DIGITALISATION R ESPONSIBLY Digitalisation brings opportunities and risks for In countries where civil society’s scope for action is outcome and impact orientation. It makes it easier restricted, communication via the internet creates for German NGOs to communicate with their new dangers for project participants. Especially partners in the Global South. They can collect and NGOs that are politically active need to weigh up pass on information about project activities more the use of digital tools for outcome and impact quickly. Virtual learning allows partners and target orientation, and should inform themselves about groups to be reached in different ways than ten secure communication channels and measures to years ago. Mobile applications (apps) can be used protect sensitive data. In particular, they should for monitoring. All project participants worldwide check data protection regulations and server can work with digitally collected information locations to protect data from misuse. simultaneously. It is foreseeable that data collection, processing But software or data analysis tools only benefit and analysis will be increasingly digital in NGOs if the users understand them, and how to the future. Algorithms will be used for these use them productively. VENRO observes a trend purposes, but how they work is sometimes not towards standardised data collection. However, the transparent. There is a danger that they could collected data is only valuable and meaningful if be based on certain assumptions which may be it has been comprehensively analysed in context. discriminatory. It is not yet fully clear what this To evaluate data in a meaningful way, a variety development means for outcome and impact of requirements must be considered. There is a orientation, or what effects digitalisation will have danger of creating “data graveyards”, since even on working methods or on project development the best software cannot comprehensively analyse and implementation. What is certain is that NGOs, data by itself. NGOs in the North and South should together with other development actors, will have therefore think about what data they really need to contend with these issues in order to counteract before deciding to acquire a particular software negative effects in project work as far as possible. package or solution. In participative approaches, NGOs and their partners can work out together how the target groups will use the digitally collected data, and how they can be included in the analysis.
From impact monitoring to impact orientation Recommendation to the German Federal The principle of “leave no-one Ministry for Economic Cooperation and behind” and requirements for Development (BMZ) and German Foreign outcome and impact orientation Office: Die Sustainable Development Goals (SDG) The United Nations Sustainable Development →→ Where data is processed and managed, privacy Goals (SDGs) are formulated in an impact- rights and human rights must be respected. oriented way in the 2030 Agenda for VENRO calls on the German Federal Ministry Sustainable Development: each of the goals for Economic Cooperation and Development is provided with indicators so that the impacts (BMZ) and German Foreign Office to enter into achieved can be measured. The overriding dialogue with German civil society on this issue. principle of the SDGs – “leave no-one The human rights risks of digital data processing behind” – is particularly important with and data management, and possible solutions for regard to outcome and impact orientation. project work, should be discussed jointly. If NGOs take this principle seriously, they must aim to have an impact on marginalised →→NGOs should be given financial support to enable groups. This changes not only the project them to utilise the potential of digitalisation in planning but also the impact monitoring, their work. because disaggregated data must be collected in order to pay special attention to impacts on marginalised communities.
From impact monitoring to impact orientation 6. TAKE ACCOUNT OF CIVIL S OCIETY’S SCOPE FOR ACTION NGOs are concerned that states around the Recommendation to the German Federal world are restricting civil society’s scope for Ministry for Economic Cooperation and action (shrinking spaces). Repressions include Development (BMZ) and German Foreign smear campaigns, arbitrary arrests and physical, Office: sexual and psychological violence. Civil society organisations are subjected to legal and →→ VENRO calls on the donor institutions to bureaucratic obstruction and harassment. Laws recognise shrinking spaces and the resulting such as anti-terrorism laws or security, internet consequences for outcome and impact oriented and media laws as well as other criminal laws are ways of working. Together with civil society, used as a pretext. solutions must be found that make it possible to continue to support politically active NGOs in Shrinking spaces make outcome and impact these contexts. orientation difficult. Joint project planning and evaluations can sometimes only be carried out in a disguised form, because staff in the partner organisations cannot speak openly about human rights or human rights violations. Many projects cannot be implemented as planned, and partner organisations increasingly have to resort to hiding projects behind other activities – especially projects that seek to achieve political change. German NGOs must assess the risk that their partners incur for themselves and others when carrying out project activities, and put people’s safety first. Rapidly changing conditions in fragile states and conflict regions also make project planning and impact monitoring more difficult, and hamper cooperation with partner organisations. NGOs are often unable to meet with local actors and target groups. Approaches such as remote monitoring make it possible to obtain information on a project’s effectiveness, even in fragile contexts. However, the safety of local partners and actors must come first. In such contexts, it is frequently impossible to record all measures, outcomes and impacts in writing in project proposals or reports.
From impact monitoring to impact orientation VENRO MEMBERS A Deutscher Paritätischer Wohlfahrtsverband Deutsch- action medeor Syrischer Verein e.V. (DSV) ADRA Deutschland DGB-Bildungswerk BUND – Nord-Süd-Netz AGIAMONDO Difäm – Deutsches Institut für Ärztliche Mission Aktion Canchanabury Don Bosco Mondo AMICA e.V. DVV International – Institut für Internationale ANDHERI-HILFE e.V. Zusammenarbeit des Deutschen Volkshochschul- Apotheker helfen e.V Verbandes Apotheker ohne Grenzen e.V. Arbeiter-Samariter-Bund Deutschland E Arbeitsgemeinschaft der Eine-Welt-Landesnetzwerke in Eine Welt Netz NRW Deutschland (agl) EIRENE – Internationaler Christlicher Friedensdienst Arbeitsgemeinschaft Entwicklungsethnologie EMA – Euro-Mediterranean Association for Cooperation arche noVa and Development Ärzte der Welt EPIZ – Entwicklungspolitisches Bildungszentrum Berlin ASW – Aktionsgemeinschaft Solidarische Welt Erlassjahr.de – Entwicklung braucht Entschuldung AT-Verband Evangelische Akademien in Deutschland (EAD) AWO International F B Fairventures Worldwide Behinderung und Entwicklungszusammenarbeit (bezev) FIAN Deutschland BONO-Direkthilfe Forum Fairer Handel BORDA e.V. FUTURO SÍ Brot für die Welt – Evangelischer Entwicklungsdienst G Bund der Deutschen Katholischen Jugend (BDKJ) Gemeinschaft Sant´Egidio Bundesvereinigung Lebenshilfe German Doctors C German Toilet Organisation CARE Deutschland e.V. Germanwatch Caritas International H Casa Alianza - Kinderhilfe Guatemala Habitat for Humanity Deutschland CHANGE e.V. Handicap International ChildFund Deutschland Help – Hilfe zur Selbsthilfe Christliche Initiative Romero HelpAge Deutschland Hilfe für Afrika e.V.* Christoffel-Blindenmission Deutschland Hoffnungszeichen / Sign of Hope D humedica Dachverband Entwicklungspolitik Baden-Württemberg I (DEAB) Indienhilfe Das Hunger Projekt INKOTA-netzwerk DED-Freundeskreis Internationaler Bund (IB) Deutsche Entwicklungshilfe für soziales Wohnungs- und Internationaler Hilfsfonds Siedlungswesen (DESWOS) International Justice Mission Deutschland Deutsche Kommission Justitia et Pax Internationaler Ländlicher Entwicklungsdienst (ILD) Deutsche Lepra- und Tuberkulosehilfe (DAHW) Internationaler Verband Westfälischer Kinderdörfer Deutsche Stiftung Weltbevölkerung (DSW) Islamic Relief Deutschland
From impact monitoring to impact orientation J Senior Experten Service (SES) JAM Deutschland Society for International Development Chapter Bonn Jambo Bukoba (SID) Johanniter-Auslandshilfe SODI – Solidaritätsdienst International SOS-Kinderdörfer weltweit* K Stiftung der Deutschen Lions KAIROS Europa Stiftung Entwicklung und Frieden (SEF) Karl Kübel Stiftung für Kind und Familie Stiftung Kinderzukunft KATE – Kontaktstelle für Umwelt und Entwicklung Stiftung Nord-Süd-Brücken Kinderhilfswerk Stiftung Global-Care* SÜDWIND – Institut für Ökonomie und Ökumene Kindernothilfe Susila Dharma – Soziale Dienste Kinderrechte Afrika KOLPING International Cooperation e.V. T Tearfund Deutschland e.V.* L Terra Tech Förderprojekte Lateinamerika-Zentrum TERRE DES FEMMES Lesben- und Schwulenverband (LSVD)* terre des hommes Deutschland Lichtbrücke Tierärzte ohne Grenzen (ToG) M TransFair Malteser International V Marie-Schlei-Verein Verband Entwicklungspolitik Niedersachsen (VEN) materra – Stiftung Frau und Gesundheit Verbund Entwicklungspolitischer Nichtregierungs- medica mondiale organisationen Brandenburgs (VENROB) medico international MISEREOR W Mission East Deutschland e.V.* W. P. Schmitz-Stiftung Missionsärztliches Institut Würzburg WEED – Weltwirtschaft, Ökologie & Entwicklung Weltfriedensdienst N Weltgebetstag der Frauen – Deutsches Komitee NETZ Bangladesch Welthaus Bielefeld Neuapostolische Kirche-karitativ e.V. Welthungerhilfe nph Kinderhilfe Lateinamerika e.V.* Weltladen-Dachverband O Weltnotwerk der KAB Deutschlands Ökumenische Initiative Eine Welt Werkhof Darmstadt OIKOS EINE WELT Werkstatt Ökonomie Opportunity International Deutschland World University Service Ora International Deutschland World Vision Deutschland OroVerde – Die Tropenwaldstiftung Z Oxfam Deutschland Zukunftsstiftung Entwicklung bei der GLS Treuhand P * Guest member Plan International Deutschland R VENRO currently has 138 members. (As of March 2020) Rhein-Donau-Stiftung S SALEM International Samhathi – Hilfe für Indien Save the Children Deutschland Senegalhilfe-Verein
From impact monitoring to impact orientation IMPRINT Publisher: Association of German Development and Humanitarian Aid NGOs (VENRO) Stresemannstr. 72 10963 Berlin Germany Phone: +49 (0)30/2 63 92 99-10 Email: sekretariat@venro org Editor: Lili Krause Final Editing: Janna Völker Translation: Richard Hewitt For their contribution we thank: Thorsten Bär, Kirsten Ehrich, Albert Eiden, Susanne Ludwig, T homas Reinhardt, Heike Scherff, Sebastian Schuster, Kirsten Wienberg Credits: Photo by Annie Spratt on Unsplash (cover), Confidence/Adobe (p. 6), Kike Arnaiz/Stocksy/Adobe (p. 11) Layout: BAR PACIFICO/ Etienne Girardet & Fabian Hickethier, Steffy Eckers Reprinting only with authorization of the Editor. Berlin, May 2020
VENRO is the umbrella organisation VENRO of development and humanitarian non-governmental organisations →→ represents the interests of (NGOs) in Germany. The association development and humanitarian aid was founded in 1995 and consists NGOs vis-à-vis the government around 140 organisations. Their backgrounds lie in independent →→ strengthens the role of NGOs and and church-related development civil society in development co- co-operation, humanitarian aid as operation and humanitarian aid well as development education, public relations and advocacy. →→ engages in advocacy for the interests of developing countries VENRO’s central goal is to construct and poorer segments of society a just globalisation, with a special emphasis on eradicating global →→ sharpens public awareness of poverty. The organisation is commited development co-operation and to implementing human rights and humanitarian issues conserving natural resources. VENRO – Association of German Development and Humanitarian Aid NGOs www.venro.org
You can also read