Evaluating Advocacy: Challenges, Methodologies and Solutions - Dr Glenn O'Neil, Owl RE 4 March 2021
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Evaluating Advocacy:
Challenges, Methodologies and Solutions
Dr Glenn O’Neil, Owl RE
4 March 2021
12
A.Challenges faced
in advocacy
evaluation
Topics B.Methods and
solutions
C.Recommended
actions"Championing or supporting a
cause or policy goal"
Gardner and Brindis (2017),
Advocacy and Policy Change Evaluation
“An intervention intended to
catalyse, stimulate or otherwise
What is seed some form of change”
ODI (2014)
advocacy? -Indirect and direct
-Private and public
-Formal and informal
3What advocacy is not?
Advocacy is not Advocacy is not having
campaigning to people represent or
change the knowledge, advocate on behalf of a
attitudes, behavior of brand or a company for
individuals – where greater awareness, e.g.
there is no link to “brand advocates”
building public support,
creating a supportive
environment or policy
change
4Tendency to measure activities rather than
outcomes
Matching advocacy ambitions to reality of
change possible
A.
The long-term non-linear nature of change
Challenges
for No “magic” advocacy evaluation tool exists
advocacy
evaluation The challenge to “prove” where you’ve
made an impact
The increased pressure for advocacy and
public affairs to show results
5Ambitions
Level of your
influence
Output Outcome Impact
61. Understand 2. Have in
what change place ways to
is desired and monitor
how change
B. 3. Select
4. Estimate
Methods appropriate
methods to
advocacy’s
influence on
evaluate
and change
change seen
solutions
5. Share and
learn from the
findings
71. Understanding what change
is desired and how
What is advocacy aiming to achieve?
Advocacy Interim
activities Outputs Outcomes Impact
outcomes
Example:
- Research - Research
Policy issue
- Coalition- produced Positive
prioritized New policy
building - Decision changes for
by decision adopted
- Meetings makers citizens
makers
reached
81. Understanding what change
is desired and how
What is advocacy aiming to achieve?
Examples:
Interim outcomes Outcomes
Media coverage Policy development
Collaboration/alignment Policy adoption
Growth of supporter base Policy revision
Recognition of issue(s) Policy blocking
Priority of issue(s) Policy implementation
Recognition of organisation Budget change
Access for organisation to advocate Practice change
91. Understanding what change is
desired and how
Theory of Change – NGO and UN advocacy on water, sanitation and hygiene (WASH) in
Health Care Facilities (HCF) - Tanzania
STIMULUS INPUTS ACTIVITES OUTPUTS OUTCOMES IMPACT
Look outside of Engagement with MoH Development of concept
Promotional events Budgets for WASH
household WASH Zanzibar note on HCF guidelines
with government and in 100 HCFs
with MoH
parliament held (FY 17-18)
Development of HCF research with
SoapBox in Zanzibar Creation of roadmap & Every health care
approach / thinking
mapping of stakeholders Stakeholder reached WASH in HC facility in Tanzania
through events integrated in has WASH to
Input into Healthy Dissemination of Zanzibar Tech. WG group set up WSDP II provide quality
Start priorities by TZ HCF research findings for guidelines Guidelines on WASH in care with
HCF produced attention to the
Unicef review of Zanzibar Engagement of the Guidelines needs of women,
assessment tools consultant for leading the implementation girls and children
process starts in HCF
Unicef research on HCF in INTERIM
mainland (7 districts) validation workshop in OUTCOMES
different zones organised WHA 2019
WAT financial support to Approval and resolution on
policy process TWG meetings for revising Launch of WASH in HCF
guidelines guidelines on WASH
Align and input from in HCF by MoH
Healthy Start global
advocacy priority Coalition building for WHA
Awareness raised
on guidelines
WASH in HCF on
WHA 2019 agenda2. Have in place ways to
monitor change
113. Select appropriate methods
to evaluate change
Methods used often in all Methods seen in all types Methods particular to
types of evaluations of evaluation but often in advocacy evaluation
advocacy evaluation
Media tracking
Document/data review
Policy tracking
Stakeholder interviews
Theory of change Policy analysis
Stakeholder surveys
Outcome mapping Media analysis
Stakeholder focus groups
Social network
Bellwether interviews
analysis
Case studies
Contribution analysis
12Has change to policies and
practices occurred?
4. Estimate What is the significance of this
advocacy’s change?
influence
on change What are the possible influences
on this change?
seen
What is the estimated influence of
your advocacy on the change seen
based on the evidence collected?
13Example – influencing policy change (fictional)
- Regional government issues a concept
Has change to policies and paper featuring ABC
practices occurred? - ABC mentioned in new regional policy
- Key national governments have issued
statements of support for ABC
What is the significance of this - High significance as first time ABC has a
profile in the policy development phase
change?
- Evidence showing critical situation for ABC
- Interest of national governments
What are the possible influences on - Coalition of actors involved
this change? - Policy process facilitated by our organisation
-Advocacy of our organisation was a key
What is the estimated influence of influence on the policy changes seen along with
your advocacy on the change seen the above interests
based on the evidence collected?
144. Estimate advocacy’s
influence on change seen
Activities
Outcomes achieved? Change seen? Significance? Your contribution?
Yes Medium High Medium
That issue 345 -research paper - First time that - Our
- Issue 345 is
is on the produced; input issue 345 is organisation
mentioned in
agenda of XYZ in consultative recognised by was main
scoping policy
commission process XYZ source of
paper; good
possibility it will commission information on
be on agenda issue 345 along
with like-
minded alliance
Etc.
155. Share and learn from the
findings
Track progress
Celebrate and
to constantly
communicate
improve
advocacy successes
advocacy
with partners /
members / sponsors
Feed findings Learn from the
into future challenges
advocacy Communicate and successes
initiatives findings using
different formats
and channels
161. Be clear and pragmatic
about what you are trying
C. to achieve
Recommended
actions for 2. Monitoring and learning
integrating must be integrated into
evaluation and strategy and
measurement in implementation i.e. as the
advocacy and advocacy is happening…
public affairs 3. “Measure what Matters”
4. Advocacy done well can
“work”!
17Available
resources
• Theory of change
• Advocacy monitoring and
evaluation methods/approaches
• Guidelines
• Communicating monitoring and
evaluation results
• “Think pieces” on advocacy
evaluation
• Examples of advocacy evaluation
reports
See: full list of resources:
https://intelligentmeasurement.net/ 18Q&A
19Contact
Contacts:
oneil@owlre.com
@glenn_oneil
glennoneil
www.owlre.com
www.intelligentmeasurement.net
20You can also read
