Translating resource revenues into effective services & infrastructure
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Translating
resource revenues
into effective
services &
infrastructure
Midterm evaluation of
the Open Contracting
Partnership’s work funded
by the BHP Foundation
Summary report
André Petheram
Scarlet George
Walter Pasquarelli
Natalia Carfi
Loren Treisman
Richard Stirling
September 2020
1 — EXECUTIVE SUMMARYTable
of Contents
3 Executive summary 20 Recommendations
3 Introduction
25 Conclusion
4 Project findings
4 Impacts 26 About
4 Intermediate outcomes 26 Oxford Insights
5 Obstacles to future 26 Open Contracting
progress Partnership
6 Our recommendations 27 Annex 1:
to OCP the development of the
7 Conclusion BHP Foundation project
8 Introduction 27 2017
10 Methodological note 27 2018
28 2019
11 Progress since 2017
30 2020
11 Impact areas
13 Outcomes identified
17 Barriers to progress
since 2017
2 — EXECUTIVE SUMMARYExecutive
summary
INTRODUCTION
In March 2020, Oxford Insights was contracted Moreover, we have traced a range of very
to conduct a midterm evaluation of the Open strong intermediate outcomes that show
Contracting Partnership (OCP)’s work under civil society organizations (CSOs), journalists,
the Translating Resource Revenues Into businesses and government officials engaging
Effective Services & Infrastructure project.1 This intensively with public contracting issues, in
is a project funded by the BHP Foundation many cases by using the open contracting
(BHPF), running from the end of 2017 until data being disclosed. There is a tangible
2022. It supports OCP’s work in an initial fifteen commitment to further and deeper reforms.
countries, with three more later added to the
project.2 It is intended to: “transform public By building on the recommendations
contracting in resource rich countries through in this report, we expect that OCP will
‘open contracting’ to promote: successfully be able to achieve its planned-
for further impacts in the remaining years
i) accessible, user-friendly open data of the Translating Resource Revenues Into
along the entire ‘deal flow’ of public Effective Services & Infrastructure project.
contracts; and
This evaluation considers OCP’s work in:
ii) better business and civic engagement
to put that data to work across
— Chile;
government.”3
— Colombia (national government);
— Argentina (federal government);
At the midpoint of the project, OCP’s
— the City of Buenos Aires;
progress towards these goals has been
— the UK (central government);
considerable.
— Afghanistan;
— Nigeria (federal government);
Firstly, OCP has noted impact in two
— Australia;
areas that clearly demonstrate how open
— Honduras;
contracting data can be put “to work across
— Mexico (federal government), and,
government” to design more effective
— Mexico City.
approaches to procurement.
We would like to acknowledge the BHP
Foundation for being a funder of this project.
1 BHP Foundation. Project summary: Translating Resource Revenues into Effective Services & Infrastructure.
https://www.bhp.com/-/media/documents/community/bhpfoundation/190507_bhpfoundation_translatingre-
sourcerevenuesintoeffectiveservicesinfrastructure.pdf?la=en
2 The initial grant document lists fifteen countries: Argentina, Canada, Chile, Colombia, Honduras, Mexico, the USA,
Ghana, Kenya, Nigeria, Sierra Leone, Tunisia, Afghanistan, Australia and the United Kingdom. Ecuador, Ethiopia,
and Malaysia were added to the project at a later date; see Open Contracting Partnership (2017). Full proposal for
the BHP Billiton Foundation. https://drive.google.com/file/d/1UoMInK8jdcINoJMyIaaM8jkCYct3yLGx/view
3 Open Contracting Partnership (2017). Full proposal for the BHP Billiton Foundation. https://drive.google.com/
file/d/1UoMInK8jdcINoJMyIaaM8jkCYct3yLGx/view, p.4.
3 — EXECUTIVE SUMMARYPROJECT FINDINGS
IMPACTS We believe that future impacts, meanwhile,
are most likely to emerge in the areas of:
OCP has seen impact in two areas
during the BHP Foundation project, both in — Healthcare and infrastructure and
Colombia: construction (that is, capital projects,
where the physical results are visible to
— Firstly, using open contracting data to the public), most consistently identified
re-design the procurement of school by interviewees as promising use cases
meals in Bogotá has seen the number for additional attention, and;
of suppliers of school meals rise from — Open Contracting Lift projects,6
12 to 55, and resulted in savings of 10- where focused theories of change
15%.4 contain clearly-defined metrics through
which OCP expects to show impact,
— Secondly, among entities using
as a result of sustained action by
Colombia’s transactional and
government and CSO partners.
transparency-focused e-procurement
system, SECOP II, competition
increased between 2018 and 2019. INTERMEDIATE OUTCOMES
The average number of bidders for
individual tenders rose by 21% over Beyond these actual and possible areas
this time. Meanwhile, more potential of impact, we noted multiple areas of
suppliers tried to offer business to progress made against OCP’s aims to open
the government, with the number of more data and “put it to work.” Across the
tenderers rising from 6571 to 7443.5 project,7 these can be summarised thus:
The procurement agency Colombia
Compra Eficiente (CCE) used 1 OCP has contributed to 16 publishers
contracting data to carefully track from BHP Foundation project countries
competition levels, while also setting opening up contracting data in a format
out guidance on increasing competition that meets the Open Contracting Data
to government agencies. Standard (OCDS), important steps
towards encouraging data use;
2 OCP has fostered a clear awareness of
the value of open contracting data within
civil society, journalism, government,
and beyond, meaning that there are,
4 Open Contracting Partnership (2018). The deals behind the meals. https://medium.com/open-contracting-stories/
the-deals-behind-the-meals-c4592e9466a2
5 Open Contracting Partnership (2020). Open for business: Colombia’s data-driven procurement reforms increase
competition. https://www.open-contracting.org/2020/07/16/open-for-business-colombias-data-driven-procure-
ment-reforms-increase-competition/.
6 See https://lift.open-contracting.org/.
7 This list does not exhaust every outcome we tracked in all locations: we identified many more that were not as
common across the project. For example, in Nigeria, there is evidence of stronger collaborative relationships
between CSOs working on open contracting, and between CSOs and government. There is also growing trust
between CSOs and government in Afghanistan.
4 — EXECUTIVE SUMMARYto some extent, stronger systems of OBSTACLES TO FUTURE
accountability within countries; PROGRESS
3 Investigations based on contracting
data that have been supported, funded, Despite these successes, a number of
or influenced by OCP have resulted in obstacles remain. The most problematic
distinct changes in government policy are that:
or personnel;
1 OCDS is often regarded as difficult to
4 Partly as a result of OCP advocating
implement and too complex for regular
for open contracting data, providing
use, and partners are not always
technical support for its disclosure, and
persuaded of its value;
persuading officials of its benefits, new
cultures of transparency are potentially 2 In some cases, there are problems
growing within certain governments; with how government entities
input data into procurement
5 OCP country managers have built
reporting systems and procurement
strong interpersonal relationships with
management information systems,
their partners, making them trusted and
meaning that the data ultimately
responsive advisors; and,
published is low quality or incomplete;
6 During the coronavirus pandemic,
3 It is not always clear what useful
CSOs and journalists have paid
“corrective action” looks like, especially
intensive attention to emergency
in terms of using data as the basis for
procurement, in some cases using
pursuing savings or managing contracts
newly-released data to conduct
more effectively; and,
investigations.
4 Political changes threaten the
development of open contracting
reforms.
5 — EXECUTIVE SUMMARYOUR RECOMMENDATIONS
TO OCP
In relation to its general work in BHPF 5 Introduce more sector-specific and
countries, we recommend that OCP should: procurement-specific expertise
to support data use within
1 Clarify the most promising specific government, so that government
and local uses for open contracting officials directly see the value of
data in each country, and promote open contracting data.
them as priorities for future impact
6 Invite partners to document and
with local partners.
then submit individual cases of
2 Collaborate with in-country governments taking specific
partners to write local theories decisions (“corrective actions”)
of change, focusing on specific because of people using open
impact areas and identifying the contracting data, allowing OCP
data that will show progress and to publicize these as appropriate.
impact (that is, set out local key Examples might include
performance indicators, or KPIs). governments resolving problems on
3 Focus on making clearer arguments infrastructure projects or canceling
for introducing the Open Contracting tenders, as a result of internal data
Data Standard (OCDS), as people use or civil society pressure.
sometimes doubt its value. 7 Prepare specific strategies for
4 Prioritize funding and supporting dealing with political disruptions
new tools and platforms through and changes of administration, so
which open contracting data that reforms are not reversed.
can be accessed by people 8 Conduct research into “cultural
without expertise in data analysis, change” within government,
so that there are more “data whereby officials become
intermediaries” in focus countries. significantly more understanding of
the value of transparency, with data
publication becoming habitual and
normalized.
6 — EXECUTIVE SUMMARYCONCLUSION
None of the obstacles we identified during CSOs consistently collaborate to improve
our evaluation are insurmountable. In most data quality and to identify problems, and
cases, OCP is already taking strong action. when government officials habitually use open
The team is widening their focus and contracting data as an invaluable part of their
investments to ensure that their partners have own decision-making.
access to high-quality data - and crucially,
information - and have the tools they need There is progress here. Governments are
to tell compelling stories about government indeed introducing new, data-based systems
contracting. And the coronavirus pandemic within their procurement processes. But
has revealed a strong global audience for supporting different use cases inside and
procurement issues, with many more actors outside of government is a delicate balancing
conducting investigations into contracting act. Every data point that a procurement
issues than we would have been able to officer willingly discloses becomes, in theory,
interview in this evaluation. raw material for a scandal uncovered by an
investigative journalist. How OCP manages
Despite these varied successes, there is still this tension across its various engagements
a risk that government leaders primarily see will be crucial to achieving impact in the
open contracting as a means by which citizens remaining years of Translating Resource
can hold them to account, and therefore as a Revenues Into Effective Services &
threat to them. Open contracting will become Infrastructure.
truly sustainable when governments and
Since March 2020, we at Oxford Insights
7 — EXECUTIVE SUMMARYIntroduction
have conducted a midterm evaluation of — improved public integrity; and/or,
the Open Contracting Partnership (OCP)’s — increased business competition.
project funded by the BHP Foundation
(BHPF) under the Translating Resource Crucially, OCP aims to demonstrate
Revenues Into Effective Services & “documented impact” in these areas when
Infrastructure project.8 The project runs the program is complete. OCP defines impact
from the end of 2017 until 2022, supporting precisely: it requires “quantitative results
OCP’s work in eighteen countries.9 It is through rigorous evaluative measurement
intended to “transform public contracting such as hard data analysis.”11
in resource rich countries through ‘open
contracting’ to promote: As the BHPF project is at its midpoint, OCP
does not expect to see the full range of
i) accessible, user-friendly open data impacts that are projected to result from
along the entire ‘deal flow’ of public its interventions. OCP’s grant agreement
contracts; and sets out an expectation of one instance of
ii) better business and civic engagement impact in year two of its BHP project - that
to put that data to work across is, before this evaluation - with one instance
government.”10 of further impact appearing in years three
and four each, and two in year five. OCP
Over the five years of the BHP Foundation demonstrated impact in Colombia in year
project, OCP expects that, when open one of the BHP project, with new and
contracting data is used by businesses, open procurement processes leading to an
governments and citizens, this will result in: increase in the number of suppliers of school
— improved service delivery; meals to the Bogotá Education Secretariat.12
— better value for money; No new impact emerged in year two. During
this evaluation, in year three of the BHP
project, OCP identified impact in a second
8 BHP Foundation. Project summary: Translating Resource Revenues into Effective Services & Infrastructure. https://
www.bhp.com/-/media/documents/community/bhpfoundation/190507_bhpfoundation_translatingresourcereve-
nuesintoeffectiveservicesinfrastructure.pdf?la=en.
9 The initial grant document lists fifteen countries: Argentina, Canada, Chile, Colombia, Honduras, Mexico, the
USA, Ghana, Kenya, Nigeria, Sierra Leone, Tunisia, Afghanistan, Australia and the United Kingdom, with Ecuador,
Ethiopia, and Malaysia added to the programme at a later date; see Open Contracting Partnership (2017). Full
proposal for the BHP Billiton Foundation. https://drive.google.com/file/d/1UoMInK8jdcINoJMyIaaM8jkCYct3yLGx/
view.
10 Open Contracting Partnership (2017). Full proposal for the BHP Billiton Foundation. https://drive.google.com/
file/d/1UoMInK8jdcINoJMyIaaM8jkCYct3yLGx/view, p.4.
11 Open Contracting Partnership (2019). 2019-2023 Strategy. https://www.open-contracting.org/wp-content/
uploads/2019/05/OCP-strategy-19digital.pdf, p.28.
12 Open Contracting Partnership (2018). The deals behind the meals. https://medium.com/open-contracting-stories/
the-deals-behind-the-meals-c4592e9466a2.
8 — INTRODUCTIONcase in Colombia, seeing an increase in
competition among suppliers using the
SECOP II procurement system.13
Beyond these impacts, this evaluation
examines what short-term outcomes have
resulted from OCP’s activities. It also asks
a) whether there is qualitative or anecdotal
evidence of additional impacts that are
already emerging and b) what areas OCP
should focus on in the future to foster and
trace impact.
In this public report, we summarise our
findings and recommendations across all
the locations we evaluated. We have also
submitted detailed country-focused reports
to OCP to help guide their decisions in
upcoming years.
13 Open Contracting Partnership (2020). Open for business: Colombia’s data-driven procurement reforms increase
competition. https://www.open-contracting.org/2020/07/16/open-for-business-colombias-data-driven-procure-
ment-reforms-increase-competition/.
9 — INTRODUCTIONMETHODOLOGICAL
NOTE
We began our evaluation in March 2020, Our main research was qualitative: we
delivering the final report on 31 August 2020. conducted desk research and over 50 semi-
As far as possible, we captured outcomes structured interviews between March and
and events as they continued to emerge August 2020. At the start of the project,
during the course of the review. The main we considered the viability of quantitative
reason for this was to include developments research (that is, analyzing numerical
in open contracting resulting from the 2020 procurement outcomes across locations,
coronavirus pandemic. While the interviews seeking to identify areas of impact), but
for our main evaluation largely concluded agreed that this would likely duplicate much
at the beginning of June 2020, some of of the ongoing work conducted by OCP’s
our findings relate to investigations ongoing own data analysts.
during August 2020. We therefore evaluated
OCP’s work during the period between Due to the ongoing 2020 coronavirus
27 November 2017 (when the BHP grant pandemic, we held interviews remotely,
agreement was signed) and 24 August 2020 having initially planned to travel to selected
(the start of the week of our final review of the countries. We sought interviewees from
report). across sectors and organizations, including
OCP country managers, OCP’s civil society
At the start of the project, we worked partners, journalists and civil servants. At
with OCP to narrow down a list of focus the very least, we wanted to ensure a good
countries from among the fifteen original balance of government and civil society
project countries, to allow in-depth analysis interviewees. A limitation of our evaluation
in the time available for our evaluation. This is the very small number of private sector
evaluation is therefore based on research interviewees that we reached.
relating to:
Finally, once we had drafted our reports,
— Chile; we shared summary versions with our
— Colombia (national government); interviewees, inviting them to add comments,
— Argentina (federal government); point out errors, and suggest new directions
— the City of Buenos Aires; for OCP, before incorporating these into our
— the UK (central government); final reports.
— Afghanistan;
— Nigeria (federal government);
— Australia;
— Honduras;
— Mexico (federal), and,
— Mexico City.
10 — INTRODUCTIONProgress
since 2017
IMPACT AREAS
OCP defines “impact” carefully, as nature and variety of suppliers. With the
measurable quantitative change in: new framework agreements in place, and
information regarding the new contracts
“indicators tracked to measure made transparently available via the SECOP
improved service delivery, better value II platform, Bogotá saw a fourfold increase
for money, improved public integrity, in the number of direct suppliers of school
or increased business competition meals, and savings at around 10-15%.15
from public spending because of open
contracting”.14 OCP was not directly involved in this reform:
it was solely due to CCE and Bogotá’s
They have identified two impacts in the BHPF education secretariat, we heard. Nevertheless,
project, both in Colombia. this was a crucial early demonstration of open
contracting’s possibilities.
1 Using open contracting data to re-
design the procurement of school 2 Among entities using Colombia’s
meals in Bogotá has seen the number transactional e-procurement system,
of suppliers of school meals rise from SECOP II, competition increased
12 to 55, and resulted in savings of between 2018 and 2019.
10-15%.
By analyzing data from SECOP II released
As reported in April 2018, the Colombian according to the Open Contracting Data
procurement agency Colombia Compra Standard (OCDS), OCP determined that
Eficiente (CCE) and Bogotá’s education between 2018 and 2019:
secretariat established two framework
agreements: one for foodstuffs and one to — the average number of bidders for open
assemble and distribute the meals. Using tenders rose by 21%;
contract data captured in Colombia’s
— more potential suppliers tried to offer
SECOP II procurement system, CCE and the
business to the government, with the
education secretariat were able to understand
number of tenderers rising from 6571 to
what minimum and maximum prices should
7443; and,
be. They also gathered information on the
— direct awards fell from 77% of all
awards to 71%.16
14 Open Contracting Partnership (2017). Full proposal for the BHP Billiton Foundation. https://drive.google.com/
file/d/1UoMInK8jdcINoJMyIaaM8jkCYct3yLGx/view, p.41.
15 Open Contracting Partnership (2018). The deals behind the meals. https://medium.com/open-contracting-stories/
the-deals-behind-the-meals-c4592e9466a2.
16 Open Contracting Partnership (2020). Open for business: Colombia’s data-driven procurement reforms increase
11 — PROGRESS SINCE 2017During this time, CCE used its contracting works projects as well as building
data to track progress against its targets, schools, for example) as especially
while also offering guidance to government promising areas.
agencies on increasing competition, and — Work in Chile to identify irregularities
providing advice to suppliers on how to in healthcare procurement has
submit strong tenders. They published already contributed to a change
monthly public reports on their progress.17 in leadership in the healthcare
buying agency CENABAST, and
There are some caveats here: only around ongoing monitoring of coronavirus
10% of national and subnational public procurement has drawn the
procurement runs through SECOP II. The attention of the National Economic
changes only appear between two years; Prosecutor’s office.18 It will be
there is insufficient data on previous years. fascinating to know whether
Moreover, emergency procurement during measurable systematic changes
2020 because of the coronavirus pandemic follow in the next two years.
will presumably push the proportion of
direct awards upwards again. OCP will — In Nigeria, Afghanistan and
need to distinguish emergency and non- Honduras there are strong cultures
emergency procurement in 2020 to continue of citizens monitoring infrastructure
to measure further changes in competition. and construction projects. Using
Nevertheless, it has worked closely with OCDS or the Open Contracting
CCE over the years of the BHPF project, for Infrastructure Data Standard
providing support on technical elements (OC4IDS) to support grassroots
of data disclosure and offering examples accountability initiatives is an
of best international practice, for example. opportunity to focus on the effects of
OCP can therefore fairly claim to have widespread data use.
contributed to this impact. — Open Contracting Lift projects
have clearly-defined metrics through
3 We identified further areas with which OCP expects to show impact.19
strong potential for future impact in An interviewee in Buenos Aires,
upcoming years. where increased SME participation in
government contracts is the goal, said
— Sector-specific impact is a strong that they believed that this trend was
possibility. Across all our interviews, indeed underway.
interviewees most commonly identified
healthcare and infrastructure
and construction (that is, capital
expenditure, including large public
competition. https://www.open-contracting.org/2020/07/16/open-for-business-colombias-data-driven-procure-
ment-reforms-increase-competition/.
17 Ibid.
18 Albert, C. and Jara, M. (2020). Los datos que analiza la FNE por posible colusión: sector público paga $500 por
mascarillas que antes costaban $13. CIPER. https://ciperchile.cl/2020/06/18/los-datos-que-analiza-la-fne-por-
posible-colusion-sector-publico-paga-500-por-mascarillas-que-antes-costaban-13/.
19 See https://lift.open-contracting.org/.
12 — PROGRESS SINCE 2017OUTCOMES
IDENTIFIED
1 OCP has contributed to 16 publishers A wide range of interviewees drawn from
from BHP Foundation project countries civil society, media organizations and
opening up contracting data in a format civil services, emphasized how the Open
that meets the Open Contracting Data Contracting Partnership has, through its
Standard (OCDS), important steps in-country workshops, international learning
towards encouraging data use. events, articles and in-person advice,
helped them understand the potential
Over the years of the BHP Foundation grant, and affordances of open contracting, and
OCP has contributed to multiple countries what specific discoveries can be made by
opening data according to OCDS. We analyzing open contracting data.
stress that this is a crucial outcome. The
existence of extensive and high-quality open Where journalists have shown interest in open
data remains an important condition for any contracting, OCP has funded collaborative
progress on data use.20 events across countries; then, when ideas for
specific investigations have emerged, OCP
Interviewees told us that OCDS offered a has offered feedback on their ideas (the Red
discrete and tangible goal that governments PALTA journalist network, for example). In
could work towards, seeing a clear reward some governments and oversight or audit
in the fact of having implemented it. OCP bodies, advice from OCP is supporting
has advocated for its introduction in decisions on what metrics to track to
discussions with senior civil servants, shown understand irregularities in procurement
its possible uses and impacts, and supported (Chile and Colombia, for example).
implementation by offering technical help
(mostly through the OCDS Helpdesk) and Some interviewees even went on to say that
advice on how to overcome political challenges. “journalists and citizens are demanding more
They have also provided support to CSOs and openness” (the City of Buenos Aires): or
private sector actors successfully campaigning that, before open contracting, “there was a
for governments to introduce OCDS.21 sense of subservience” among the public, but
that they are now more expectant of good
2 OCP has fostered a clear awareness government (Nigeria). It is not yet clear how
of the value of open contracting widespread this effect is, however.
data within civil society, journalism,
government, and beyond, meaning Accountability, in this context, refers to:
that there are, to some extent,
stronger systems of accountability — more press coverage of contracting
within countries. issues;
20 ‘Extensive and high-quality’ can broadly be taken as meaning: a large proportion of procurements disclosed in
OCDS, including a large proportion of buying entities; data covering multiple years; data on all five procurement
stages included; few missing data fields; and, few errors in data inputs.
21 The 16 publishers currently listed by OCP are: Afghanistan: National Procurement Authority; Argentina: Gobierno
de la Ciudad de Buenos Aires; Argentina: Dirección Nacional de Vialidad; Australia: Department of Finance;
Australia: New South Wales; Canada: Montreal; Chile: ChileCompra; Colombia: Colombia Compra Eficiente;
Honduras: Comisión para la Formación de la Alianza Público-Privada; Honduras: Oficina Normativa de Contratación
y Adquisiciones del Estado; Kenya: Makueni County; Mexico: Jalisco - Secretary of Planning, Administration, and
Finance; Mexico: Nuevo León (OCDS & OC4IDS); Mexico: QuienEsQuien.wiki; United Kingdom: Crown Commercial
Services; United Kingdom: Scotland - Scottish Procurement and Commercial Directorate.
13 — PROGRESS SINCE 2017— civil society organizations undertaking In Chile, for example, following a project
more targeted and informed funded by OCP to investigate competition
investigations of government in procurement, Observatorio Fiscal looked
contracting; deeper into health procurement. They noticed
— government officials understanding that multiple purchases being directed towards
open contracting data can be used GlaxoSmithKline, the former employer of the
to correct problems in procurement director of the health procurement agency,
(“internal” accountability). CENABAST. He was asked to resign after the
story became public.22
The next step is ensuring that this data use
systematically leads to governments and In the United Kingdom, Tussell, a private
oversight or audit bodies taking corrective company that tracks the UK’s OCDS data and
actions to improve procurement systems: that the EU’s Tenders Electronic Daily data, noticed
is, that the feedback loops are completed. that a contract for marine freight shipping had
There are some very good instances of been awarded to a company with no ships.
governments responding to specific issues The contract was canceled after sustained
identified by journalists or officials, but this is, public attention. OCP published a use story
broadly, not yet routine or formalized. soon after the event, demonstrating a strong
response to this kind of case.23
3 Investigations based on contracting
data that have been supported, funded 4 Partly as a result of OCP advocating
or influenced by OCP have resulted in for open contracting data, providing
distinct changes in government policy technical support for its disclosure,
or personnel. and persuading officials of its benefits,
new cultures of transparency are
CSOs, journalists and the private sector have potentially growing within certain
identified clear irregularities in procurement governments.
by using open contracting data. These cases
amount to an impressive set of examples. Some interviewees (in Afghanistan, Buenos
They clearly show how contracting data is Aires, Colombia, Honduras, and Nigeria,
being used by a range of stakeholders to for example) commented that a focus on
hold governments and officials to account, open contracting has normalized the concept
with governments then resolving inefficiencies of transparency, or created a “shift in culture”
and instances of corruption. In some cases, in government.
these investigations were directly inspired
and supported by OCP; in others, they In Afghanistan, various people said that
emerged independently of OCP but clearly the concept of releasing data is no longer
demonstrated what can be done with controversial in certain agencies or within
contracting data. the “political elite”; a remarkable shift
within a historically opaque bureaucracy.
22 Elgueta, E. (2019). Irregularidades en Cenabast finalmente hacen caer a Juan Carlos Corbeaux. El Mostrador. https://
www.elmostrador.cl/mercados/2019/06/14/el-primer-cortado-de-la-nueva-era-manalich-en-el-minsal-ministro-sa-
ca-a-juan-carlos-corbeaux-de-la-cenabast/.
23 Evans, M. (2019). How we uncovered a controversial Brexit contract using open contracting data. Open Contracting
Partnership. https://www.open-contracting.org/2019/02/25/how-we-uncovered-a-controversial-brexit-contract/.
14 — PROGRESS SINCE 2017This change began before OCP’s work future engagements - and future OCP hires
in the country, but recent focus on open - recognizing and developing this element of
contracting has possibly intensified it. One success will be crucial.
person argued that open contracting has
the potential to transform the nature of However, this observation also reflects
governance in Afghanistan, bringing an a potential vulnerability: if OCP is less
emphasis on participatory policy-making by accessible in certain countries due to
encouraging citizen oversight of government, capacity constraints, how can it ensure
and challenging officials’ “topdown” that its partners take on and develop this
assumptions. In Colombia, an interviewee persuasive and advisory interpersonal role?24
said that introducing OCDS had embedded OCP should consider that its remit for training
the principle of transparency in government; and documentation might include offering
in Buenos Aires, the understanding of the strategies for trust-building and interpersonal
importance of open contracting has strongly advocacy.
increased across agencies.
6 During the coronavirus pandemic,
The depth and precise nature of this OCP partners in civil society and the
change is worthy of further research, as it is media have paid intensive attention
potentially a significant outcome of OCP’s to emergency procurement, in some
and its partners' work. cases with direct OCP support, while
some partner governments have also
5 OCP country managers have built taken care to maintain transparency
strong interpersonal relationships with principles during the pandemic.
their partners, making them trusted
and responsive advisors. The coronavirus pandemic has demonstrated
that there is a large and diverse set of
This was a common theme, and should not users for contracting data, across OCP’s
be underestimated. Multiple interviewees BHPF focus countries. In some cases,
across various countries (including Chile, procurement agencies have moved quickly
Nigeria, and the United Kingdom) to release coronavirus-specific contracting
emphasized the ease of communicating data through dedicated portals (Colombia
with OCP country managers, making OCP a Compra Eficiente25, ONCAE in Honduras26,
regular reference-point for problem-solving and AGEOPS in Afghanistan, for example27)
and advice on specific issues. This implies and websites (Nigeria’s Bureau of Public
that much of OCP’s broad success is due to Procurement28). Other publishers have
individuals’ credibility in informal settings. For been less proactive, notably in the UK,
24 Of course, in many cases OCP partners will be taking on this role, but it will be fruitful for OCP to carefully focus
on what this involves and how to support it further.
25 Colombia Compra Eficiente (2020). Herramienta de Control Ciudadano COVID-19. https://app.powerbi.com/
view?r=eyJrIjoiMGQ5YTkwNzItZWVhMC00ZTgxLWJmYzgtODE1NWE0ZGZmNDVkIiwidCI6IjdiMDkwNDFlLTI0NTEt-
NDlkMC04Y2IxLTc5ZDVlM2Q4YzFiZSIsImMiOjR9.
26 ONCAE (2020). Observatorio de Precios para Insumos de la Emergencia. http://oncae.gob.hn/observatorio.
27 NPA (2020). AGEOPS, COVID-19. https://ageops.net/en/procurement-procedure/contract/progress/search?tag_
id=1&page_size=10&page_index=2.
28 BPP (2020). NOCOPO/COVID-19 EMERGENCY PROCUREMENT. https://nocopo.bpp.gov.ng/ContractReportCovid19.aspx.
15 — PROGRESS SINCE 2017where ministries have been criticized for not and II procurement systems for it to be
following requirements that contract notices usable. In Nigeria, though, the Bureau of
for direct awards be published 30 days after Public Procurement (BPP)’s simple list of
the award.29 coronavirus-related contracts has been used
by investigative journalists from Dataphyte
OCP has arguably contributed to and the International Center for Investigative
procurement agencies’ steadfastness on Reporting to identify unregistered companies
transparency principles during the crisis. receiving government contracts.31
For example, Colombia Compra Eficiente’s
guidance to procuring entities directly Investigations into coronavirus-related
refers to OCP advice on the importance contracting have prompted governments
of transparency during emergency and oversight agencies to take action on
procurement.30 In other cases, OCP has, over irregularities. In Argentina, La Nación Data
years, consistently encouraged procurement reported on suspiciously high prices being
agencies to disclose data. Coronavirus paid for foodstuffs during the pandemic,32
contracting transparency can be understood leading to 15 officials’ being dismissed from
as a spillover effect of this advocacy. the Ministry of Social Development.33 In Chile,
the Investigative Journalism Center (CIPER)
The coronavirus-specific data released has documented the government paying CLP$500
not always been directly useful for data users, per unit for masks that used to cost $12.80
however. In Colombia, we heard from one per unit. The National Economic Prosecutor’s
investigative journalist that it was easiest office is opening a preliminary investigation.34
to build their own contracts database by
manually inputting information from PDFs These cases show how, during the
of contracts. While CCE’s coronavirus pandemic, the feedback loops between
transparency portal did help them identify data disclosure, data use and authorities’
what contracts they wanted to examine, reactions have accelerated. It is possible that
they could not download data from it directly. coronavirus has strengthened these feedback
Moreover, there were too many errors in loops, and deepened the global market for
the data downloaded from the SECOP I contracting data use.
29 Kinder, T. and Plimmer, G. (2020). UK government paid £1.7bn to private groups for coronavirus contracts. Financial
Times. https://www.ft.com/content/7fe7c2d5-24df-431b-9149-50417fa0236a.
30 Colombia Compra Eficiente (2020). Guía de Transparencia en la Contratación Estatal Durante la Pandemia del
Covid-19. https://www.colombiacompra.gov.co/sites/cce_public/files/cce_documentos/v5_guia_de_transparencia_
en_la_contratacion_en_la_pandemia_covid-19.pdf, pp.3-5
31 ICIR. (2020). INVESTIGATION: COVID-19 response projects worth N534.98 million awarded to unverified contractors.
https://www.icirnigeria.org/investigation-covid-19-response-projects-worth-n534-98-million-awarded-to-unveri-
fied-contractors.
32 Cabot, D. (2020). Total, paga el Estado. La ineficiencia en las compras que genera sobreprecios y corrupción. La
Nación Data. https://www.lanacion.com.ar/economia/total-paga-estado-la-ineficiencia-compras-genera-nid2353052.
33 Pallaro, B. (2020). Compras del Estado. Entre la falta de transparencia y la ausencia de controles en tiempo real. La
Nación Data. https://www.lanacion.com.ar/politica/compras-coronavirus-un-sistema-peligroso-deficiente-nid2360123.
34 Albert, C. and Jara, M. (2020). Los datos que analiza la FNE por posible colusión: sector público paga $500 por
mascarillas que antes costaban $13. CIPER. https://ciperchile.cl/2020/06/18/los-datos-que-analiza-la-fne-por-
posible-colusion-sector-publico-paga-500-por-mascarillas-que-antes-costaban-13/.
16 — PROGRESS SINCE 2017Barriers to
progress since 2017
1 OCDS is often regarded as difficult This would help resolve a further issue,
to implement and too complex for wherein OCDS is introduced into
regular use, and partners are not environments where data is already
always persuaded of its value. relatively open. In Chile and Colombia, the
additional added value of an OCDS API over
This was a common refrain during our the data that is available through the Chilean
interviews. Many people said that the nested government’s Data Warehouse (to which
JSON format was off-puttingly complex for CSOs have access if they sign a license) or
many potential users in civil society and the SECOP II platform in Colombia is often
government, and stressed the need for unclear. In these places, open contracting
simpler and quicker ways to analyze the investigations tend not to use OCDS data.
large-scale data available through OCDS. More broadly, for government officials across
One interviewee captured the point neatly: countries, OCDS implementation can be
the analyst “who only knows about Excel” seen as an expensive and difficult project,
is excluded from using much OCDS data. where the benefits are not clear compared
The problem is exacerbated in places to the costs.
with relatively low data literacy, such as
Afghanistan, Colombia, Honduras, and Finally, one interviewee questioned the
in some agencies in the UK. While OCP purpose of the standard when the quality of
does advocate for data to be disclosed in data feeding into it is low in the first place
CSV format, these releases are often not (that is, through the whole procurement
in simple tabular structures and remain management information system). They
challenging for many users. referred to inconsistent units being used
and misclassifications of goods making
During interviews, the OCDS Helpdesk analysis very hard. This person - and other
emphasized that the purpose of OCDS, and interviewees - worried that OCDS was being
the JSON structure in particular, is not to implemented so that senior civil servants
be accessible to all users; rather, it is to act and ministers could show that they had
as a basis for the creation of intermediary ticked a “checkbox” of transparency, but
analytical tools. There is a risk that this then ignore the serious question of how to
message is being lost. Before and during ensure minimum data quality standards.
OCDS implementation projects, OCP may To avoid this in the future, our interviewee
need to be more careful about sustaining strongly recommended that OCP design
a clear narrative about what the standard a more visible way of exerting pressure on
should be used for. Outlining clear use cases, governments over the issue of data quality.
specific to OCDS, will help overcome this, as This is discussed further below.
we recommend below.
17 — BARRIERS TO PROGRESS SINCE 20172 In some cases, there are problems suggested that OCP survey its partners
with government entities inputting asking them to name the five things they
data into procurement reporting need for open contracting data to be
systems and procurement usable, and build this into a “minimum
management information systems, quality standard” that should apply to any
meaning that the data ultimately contracting data that the government
published is low quality or incomplete. releases. This should be understood as a
quality mark rather than a data standard,
This has been an issue in Nigeria, designed to publicly hold governments to
Afghanistan, Colombia, and Chile, we account on data quality. OCP publicizing
heard. While there is an established process governments’ performance against this could
by which contracting authorities should help prevent governments from implementing
be uploading data on contracts for it to be the structure of OCDS but ignoring data
disclosed in a central location like AGEOPS quality, this person emphasized.
in Afghanistan or NOCOPO in Nigeria,
procurement officers often do not follow Broadly, focused attention on this issue
this properly. This means that the data will benefit OCP; it will certainly arise in
disclosed is frequently incomplete or low future engagements. At the least, it is worth
quality. An interviewee in Chile stressed that considering a short project asking what kinds
data quality is the single largest restriction of sanctions and/or persuasive approaches
on their ability to analyze contracting data. have been most useful for compliance, in the
For example, units of measurement and experience of OCP’s partners.
quantities are entered very inconsistently,
meaning that it is very difficult to understand 3 It is not always clear what useful
how much of a particular item is being “corrective action” looks like, especially
purchased in aggregate. in terms of using data as the basis
for pursuing savings or managing
Explanations for this have varied. “The contracts more effectively.
incentives are very low” to enter high-quality
and complete data into the system, we What happens next when you have clear,
heard in Chile. In Nigeria, we heard that data-based analysis in front of you?
individual ministries claim that their internet Suggesting sanctions or punitive action, or
connection isn’t good enough to upload the reversal of the problem which has been
data, or that they lack the training to do so. In identified, might often be the result. This was
both Afghanistan and Nigeria, interviewees the case for CENABAST in Chile, following
reflected that procurement officers had the dismissal of its director after Observatorio
been “afraid” of what disclosing data would Fiscal’s investigation into low competition in
reveal publicly. In Afghanistan, however, health procurement. But open contracting
the situation has apparently improved as data can be used more strategically by
procurement officers have become more procurement agencies, and this is not always
used to the idea of data transparency. clear to them.
One person said that OCP should develop For example, understanding the relative
more visible ways to hold governments to size of suppliers is the first step towards
account for data quality in general. They understanding how a government might
18 — BARRIERS TO PROGRESS SINCE 2017exert commercial leverage over these Elections and political turmoil have presented
suppliers in pursuit of savings. Equally, themselves as serious obstacles to data
identifying that departments are disparately disclosure, and even moments for regression.
buying similar goods and services at a range For example, changes of government in
of prices can justify a cross-governmental Mexico and Argentina, and elections in
sector-specific strategy. Afghanistan and Nigeria have arguably
distracted decision-makers from open data
One person, reflecting on the priorities of new and procurement reform as priorities.
governments in Latin America, provocatively
suggested that OCP should remove words Where civil servants are political appointees,
like “transparency”, “accountability” and and therefore change along with
“openness” from their campaigning lexicon. governments, the rupture can be severe.
The language of efficiency will appeal to This is the case in Argentina, for example;
contemporary decision-makers, they argued. there are no official handovers between
Where they have huge debts and deficits teams, and institutional memory can be very
following the coronavirus crisis, this will be swiftly lost. The case is similar in Mexico.
yet more powerful. Another interviewee, in OCP does plan for governmental transitions,
Argentina, also referred to the need to make having contributed to a transition memo
the “storytelling” around OCDS stronger, in Mexico, and also supported introducing
with a focus on “saving money” and making legal mandates for open contracting so that
“government smarter”, to win the political requirements to publish data are consistent
argument inside government. Of course, between administrations. Nevertheless, the
many such framings have potentially divisive vulnerability remains. OCP might consider
political connotations and must be chosen whether it can go even further in identifying
carefully to have an appeal across party and how to create consistency between regimes.
ideological lines. For example, this could include establishing
a common organizational strategy for
A similar reflection focuses on the examples handling political change or maintaining
OCP uses to make the case for open documentation that can be quickly
contracting. Interviewees in the UK and disseminated with new officials.
Australia - both countries that perceive
themselves as having low levels of corruption OCP has been broadly successful at
- suggested that using examples of countries handling political difficulties in the UK,
with higher corruption rates and recent though in this case there is a permanent
institutional change could be counter- civil service. Here, the team has maintained
productive. In Whitehall and Canberra, relationships with people at a range of
government officials can point to their long levels in the civil service, and thereby
institutional stability and apparently low ensured a continued base of support within
corruption rates to suggest that the context is the government until procurement reform
simply different. becomes a priority amongst the leadership.
They have also maintained close attention
4 Political changes threaten the to deepening capacity among civil society
development of open contracting and journalists.
reforms.
19 — BARRIERS TO PROGRESS SINCE 2017Recommendations
1 Clarify the most promising specific in procurement was cited by interviewees
and local uses for open contracting as especially helpful. Other specific focus
data in each country, and promote areas could include citizen monitoring, data
them as priorities for future impact publishing, SME inclusion, and empowering
with local partners. women-owned businesses, we heard.
Broadly, we suggest that there is further 2 Collaborate with in-country partners
work to be done to demonstrate the to write local theories of change,
various purposes and affordances of open focusing on specific impact areas
contracting data use, especially when it and identifying the data that will
involves using OCDS data. This is often seen show progress and impact (that
as complex and hard to use. is, set out local key performance
indicators, or KPIs).
Telling stories about data use (likely based
on the work of data intermediaries, as With use cases and impact targets chosen,
recommended below) will be more focused a shared understanding of these goals
and tangible if centering on specific use across civil society and government will
cases. Interviewees in Australia, for example, provide a consistent focus for stakeholders’
powerfully made this point. The examples of future activities. Co-creation and review from
Observatorio Fiscal on health procurement partners will ensure that the goals are realistic
in Chile, on school meal provision in and appropriate, and will help determine the
Colombia, and on infrastructure monitoring in level of investment needed to achieve them.
Honduras, exemplify it well, as will the stories
emerging from OCP’s Lift projects. This approach, involving more regular
tracking of intermediate indicators at the
However, some countries lack such specific country level, allows for more agility and
“flagship” cases, and use remains relatively flexibility as OCP moves towards impact:
general. Across our interviews, healthcare, OCP will be able to adapt its approach
infrastructure and construction were regularly according to a continual review of what is
cited as sectors where open contracting working and what is not. Building in scope
could realize its potential. It might support to quickly identify unexpected changes,
a more commercially-minded approach manifested by single “corrective action”
to government buying, some people cases, for example, can also allow for
said. Of course, the pursuit of corruption rapid changes of emphases and effort to
was also a common theme. OCP’s Latin capitalize on these.
American training on identifying red flags
20 — RECOMMENDATIONSWe understand that OCP is moving more example, understanding exactly how
closely towards a more use-case-focused investigations rely on OCDS data, how OCDS
monitoring approach, especially in the specifically has made these investigations
project design for their Lift projects. This is more effective and/or easier, and which other
the right direction to take, and also provides audiences will find this narrative particularly
an opportunity to experiment with different appealing. It will also involve using the
methods in different countries. For example, language of civil servants: creating business
OCP might consider the merits of the “most cases and showing the likely return on
significant change” evaluation methodology. investment of OCDS, for example.
This is useful for monitoring people’s
perceptions of change and reacting to Finally, OCP should keep in mind the
unexpected outcomes in complex situations.35 possibilities that OCDS introduction may
ultimately not be the best way to disclose
3 Focus on making clearer arguments contracting data in certain cases. This will
for introducing the Open Contracting continually encourage a lively argument
Data Standard (OCDS), as people for its adoption, or ensure its judicious
sometimes doubt its value. application. One person argued for the
development of an “OCDS-lite”: in the
Our evaluation has suggested that the format of a single table with just a few
specific added value of introducing OCDS crucial fields, so that implementation would
is not always clear to partners, especially be quicker and use simpler.36
when there are quicker and easier routes
to analyzing contracting information. One 4 OCP should prioritize funding and
interviewee said that the time taken to supporting new tools and platforms
implement OCDS can be very unappealing to by which open contracting data
senior government officials. can be accessed by people without
expertise in data analysis, so that
Re-emphasizing the argument that it is a there are more “data intermediaries”
starting point for data intermediaries, who in focus countries.
can go on to create new tools, products, and
services, will be important. In some cases, OCP should not assume that new OCDS use
this can involve local GovTech companies, cases will emerge organically: they must be
from whom governments might purchase carefully supported. In Chile, Observatorio
licenses to use OCDS-based management Fiscal’s Public Buying Observatory provides
information systems; OCP might even offer a very strong example of OCP’s ability to do
investment capital. In Latin America, the this (though it was not based on OCDS data).
many CivicTech CSOs may also be helpful.
We suggest that there are three potential
In part, this is an issue of storytelling elements to such work. Firstly, converting
appropriate to the audience at hand: for data to (or releasing data in) simplified
35 See https://www.betterevaluation.org/en/node/5280 for theories of change and https://www.betterevaluation.org/
en/plan/approach/most_significant_change for most significant change.
36 OCP’s “Usability checklist” and tabular data collection spreadsheet are steps towards this. But our interviewee was
considering an even simpler format: https://docs.google.com/spreadsheets/d/1nG7e52E1CXOXoUjz6pimW4Z7er9u3D-
JSs98QKdJJioE/edit#gid=110864222 and https://www.open-contracting.org/resources/data-collection-spreadsheet/.
21 — RECOMMENDATIONSformats like CSV. Secondly, supporting medicine purchases? Could civic monitoring
intermediaries to create intermediary tools: of infrastructure projects run alongside the
this is often more time- and resource- reform of major project management within
intensive than expected, potentially requiring government? Open contracting data can be
grant funding. Thirdly, and crucially, promoting used as the basis for category strategies and
the new tools’ benefits and training end- significant savings, and government officials
users on their use. In our own experience, need to see this more clearly. This will also
this whole process requires concerted effort help OCP develop more context-specific
over significant periods of time to work, with examples of successful reforms, diversifying
the final phase requiring a sustained focus the perceived narrative focus away from
on marketing intermediary platforms to the corruption cases.
relevant user groups.
Targeting different user groups with different
One successful example beyond this advice formats and use cases will also likely
evaluation is OpenUp in South Africa: a civic be fruitful here. For example, government
tech organization that creates public tools officials may respond more strongly to private
based on open government data.37 Beyond business cases, white papers, and even slide
their development of intermediary tools, their decks, rather than public blogs. Meanwhile,
success lies in the close relationships that journalists will be more keen to access
they have established with a wide range of training on translating potentially complex
end-users. That is, they have forged links technical issues into compelling public stories
between citizens, civil society, government (OCP has been notably strong on this).
officials and journalists. This enables them
to directly support the use of data in their 6 Invite partners to document and
campaigns and news stories, for example. then submit individual cases of
governments taking specific
5 Introduce more sector-specific and decisions because of people using
procurement-specific expertise to open contracting data, allowing OCP
support data use within government, to publicize these as appropriate.
so that government officials directly Examples might include governments
see the value of open contracting data. resolving problems on infrastructure
projects or canceling tenders, as
A focus on specific sectors and use cases a result of internal data use or civil
will also allow OCP to deepen its work, by society pressure.
bringing in category-specific expertise that
can show the advantage open contracting There are multiple cases of journalists and
can offer to government officials and civil society organizations investigating
ministers. If there are price variations in specific issues that they have discovered by
medicine costs, for example, can OCP offer assessing contracting data. It is also possible
its partners access to people with experience that these kinds of results are occurring
of planning and negotiating large-scale without OCP knowing about them. While
37 https://openup.org.za/; CODE in Nigeria provide another good example, at https://www.connecteddevelopment.
org. See also Treisman, L. (2014). Access to information: bridging the digital divide in Africa. The Guardian. https://
www.theguardian.com/global-development-professionals-network/2014/jan/24/digital-divide-access-to-informa-
tion-africa.
22 — RECOMMENDATIONSYou can also read
