Audits for Accountability: Evidence from Municipal By-Elections in South Africa - OSF
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Audits for Accountability:
Evidence from Municipal By-Elections in South Africa
April 18, 2019
Daniel Berliner* and Joachim Wehner†
London School of Economics and Political Science
Abstract:
Theories of retrospective accountability assume that voters punish poor governance and reward
improvements, yet empirical evidence remains mixed. We extend this research to a new context,
assessing the impact of audit information on electoral performance in South African
municipalities. Our novel identification strategy focuses on by-elections triggered by deaths in
office of local councilors and compares those taking place before and after audit results are
announced each year. We find that timely audit information affects the vote share of the
responsible party, with voters rewarding improvements and punishing poor performance by
about 5 percentage points. Individual-level survey evidence offers similar results. Our findings
broaden the scope conditions of prior work by documenting accountability effects in a strongly
party-centered setting. We emphasize the importance of information disseminated at full scale –
which is transmitted through regular media and organizational channels, increases salience, and
ensures common knowledge – and from a credible source.
Word Count: 11,631
*
Assistant Professor of Political Science and Public Policy, Department of Government, London School
of Economics and Political Science, d.berliner@lse.ac.uk.
†
Associate Professor in Public Policy, Department of Government, London School of Economics and
Political Science, j.h.wehner@lse.ac.uk.
For helpful comments and assistance, we would like to thank Jim Alt, Ben Ansell, Thomas Bräuninger,
Christian Breunig, Natalia Bueno, Hugh Cole, David Doyle, Andy Eggers, Aaron Erlich, Alice Evans,
Karen Ferree, Omar García Ponce, Mark Hallerberg, Robin Harding, David Heald, Macartan Humphreys,
Ryan Jablonski, Eckhard Janeba, Horacio Larreguy, Bob Mattes, Zeev Maoz, Nishendra Moodley,
Aneesh Raghunandan, Philip Rathgeb, Anselm Rink, Petra Schleiter, Kathryn Schneider, Collette Schulz-
Herzenberg, Daniel Sturm, Christian Traxler, Martin J. Williams, and Lauren Young. Trishna Kurian
provided outstanding research assistance. Versions of this paper were presented at the 2017 MPSA, EPSA
and APSA annual meetings, the City of Cape Town, and seminars at Hertie, Konstanz, LSE, Mannheim,
Oxford, Stellenbosch, and UC Davis. We would like to thank staff at the Auditor-General and at the
Electoral Commission, especially Stuart Murphy, for data and patiently answering our queries. Our
research into news coverage was possible due to database access in the National Library of South Africa.
We are indebted to Washeelah Kapery and Reza Omar for the data from the South African Citizen
Survey. All interpretations and conclusions are entirely our own and should not be attributed to any of the
above individuals or institutions. We gratefully acknowledge financial support for this project from the
Suntory and Toyota International Centres for Economics and Related Disciplines (STICERD) and the
Department of Government at LSE.
1
At the heart of democracy is the idea that voters can select politicians to run government and
deselect them when they prefer an alternative (Przeworski et al. 1999, Besley 2007, Bovens et al.
2014). One of the preconditions for this seemingly simple idea to work is that voters
retrospectively reward or punish politicians for their performance in office, which requires
sufficient information to evaluate incumbents. Yet empirical evidence to support this assumption
has been decidedly mixed (Healy and Malhotra 2013, de Vries and Solaz 2017). One major
initiative evaluating the effect of government performance information on voting behavior
through seven coordinated field experiments in six countries found that the effects of the studied
interventions were “largely null” (Dunning et al. 2018: 15). This evidence may lead some to
conclude that the potential of information and accountability initiatives to improve the quality of
government has been overstated.
We contribute new evidence to the contrary, finding strong effects of local government
performance information on voter behavior in South African by-elections, and propose a
perspective that helps to reconcile seemingly disparate findings from earlier studies. We
highlight the particular importance of scale as a key consideration in how information shapes
electoral accountability. Citizens may be more likely to respond in ways consistent with
principal-agent models of electoral accountability in contexts where performance information
originates from a familiar and credible source, is transmitted via regular media and
organizational channels, receives nationwide attention and amplification, and leads to common
knowledge across citizens in disparate locales. Information at full scale may thus have electoral
effects beyond those captured by researcher-manipulated experiments introducing localized
interventions.
One promising approach to studying such “real-world” information effects has been to focus on
audits of local governments, particularly where these are produced by credible organizations and
receive substantial public attention. Yet this approach has relied on rare institutional features for
convincing identification strategies, which has limited the number of settings examined in
empirical studies. Aside from Brazil’s randomized municipal audits (e.g., Ferraz and Finan 2008,
Brollo 2010), few other contexts have been analyzed. In both Mexico (Larreguy et al. 2014) and
Puerto Rico (Bobonis et al. 2016), scholars have studied the electoral impact of municipal audits
2
that were irregularly (but not randomly) timed so as to be released before elections in some
localities and afterwards in others. However, we know of no studies to date examining the
electoral impacts of both annual and universal local government audits. The accumulation of
knowledge requires extending such research to new contexts.
We do exactly this, studying audit information at the municipal government level in South
Africa. As every municipality is audited in every year, and local government elections are held
all at once every five years, we adopt a novel approach for comparison. Our solution is to focus
on by-elections, and to compare those where citizens do, and do not, have access to timely and
attributable information about local government mismanagement. By-elections are relatively
frequent – over 100 per year on average – and are held to replace councilors who have died,
resigned, or been removed from office, and they take place within a rigid statutory timeframe. As
resignations and removals from office may be strategically timed, we focus solely on by-
elections triggered by deaths in office (e.g., Jones and Olken 2005).
Analyzing by-elections triggered by the death of a councilor over the period 2008 to 2015, we
assess the impact on governing party vote shares of the Auditor-General’s annual release of
municipal audit results. This annual report receives a high level of attention in the South African
media, and its findings are often used by political actors. Comparing by-elections held in the
periods after each report’s release date, with those held during the rest of each year, allows us to
isolate the effect of audit information from the effect of underlying governance conditions
themselves. Our identification strategy thus relies on a natural experiment created by the
confluence of by-election timing and audit timing. Our qualitative assessment rules out the
potential for manipulation of their relative timing, and we find no quantitative evidence of
imbalance on observable covariates or of strategic sorting of by-election timing in our sample.
We find that voters punish deteriorations and reward improvements in audit performance, with
effects on the vote share for the mayor’s party equivalent to roughly 5 percentage points.
Interestingly, we find that information conveying unchanged audit performance also leads to a
similar decline in vote share. This suggests that voters respond negatively to new information on
stagnation as well, consistent with the widespread frustration with the poor status quo that has
3
led to sometimes violent “service delivery protests” (Alexander 2010). We detect very similar
patterns with individual-level evidence from a survey in the field before and after the release of
the 2016-17 municipal audit report. Comparing the periods before and after the release date,
evaluations of mayoral performance improved in municipalities where the new information
demonstrated positive change and vice versa.
Our work extends existing research on the effects of government performance information on
voter behavior in crucial ways. First, our evidence is from a new context where information is
published by a credible (and politically independent) official source, transmitted through regular
media and organizational channels, and receives nationwide attention. One benefit of our setup is
that it allows us to examine real-world general equilibrium effects of information (Glennerster
and Takavarasha 2013: 57-59). Studies on real-world electoral impacts of local government
audits to date have been limited to contexts where either a random (Ferraz and Finan 2008) or
non-random (Larreguy et al. 2014) selection of municipalities are audited each year, or where all
municipalities are eventually audited but with staggered timing (Bobonis et al. 2016). Second,
our empirical approach avoids potential concerns associated with researchers or external actors
manipulating information in the context of elections (Cronin-Furman and Lake 2018, Johnson
2018).
Finally, extant work on the electoral effects of audits has examined systems with directly elected
mayors (Ferraz and Finan 2008, Larreguy et al. 2014, Bobonis et al. 2016). Likewise, field
experiments of the electoral impacts of performance information have focused on candidate-
centric political settings (Dunning et al. 2018). As Grossman and Michelitch (2018: 295) caution,
it is not clear what this implies for systems where electoral accountability relies on political
parties. While the case we examine involves the direct election of local politicians at the ward
level, very few voters know their identity.1 Moreover, the information treatment is about the
performance of a particular political party in charge of the municipality rather than an individual
seeking reelection – after all, the incumbent is deceased. Only Larreguy et al. (2014) study
1
Only 14 percent of South Africans could name their local councilor in 2006, the third-lowest ratio
among 17 African countries (Mattes 2008: 120).
4
electoral impacts of audits in a system without reelection, due to term limits. In addition, South
African mayors are not directly elected but chosen by local councils based on party majorities.
By contributing evidence on the electoral effects of audits in a strongly party-centered political
system, we help to clarify the scope conditions for this body of research.
Information and Accountability at Scale
Theories of retrospective accountability suggest that voters look back to punish or reward
politicians for their performance in office (Fiorina 1981, Przeworski et al. 1999). Building on
these theories, many donors, policymakers, and civil society organizations advocate information-
based interventions as ways to promote good governance and combat corruption (Khagram et al.
2013, Kosack and Fung 2014, World Bank 2016). Yet both theoretical and empirical questions
remain unresolved, leading to a large and active literature seeking to assess the extent to (and
circumstances under) which voters reward or punish politicians based on the level of corruption
or quality of governance whilst in office (de Vries and Solaz 2017).
Empirical results from this literature appear inconsistent. A number of experimental studies
documents null results of information dissemination treatments across different contexts
(Humphreys and Weinstein 2012, Lieberman et al. 2014, Dunning et al. 2018). A field
experiment conducted in Mexico found that information on corruption led voters to disengage
and depressed turnout (Chong et al. 2015). Other studies suggest that the electoral response to
corruption information is limited if voters and politicians share partisan ties (Anduiza et al. 2013)
and that voters are less likely to punish corruption when they benefited from it (Fernández-
Vázquez et al. 2016). Yet some results are supportive of theories of retrospective voting. In a
survey experiment in Brazil, Winters and Weitz-Shapiro (2013) find evidence that voters are
willing to react to specific, credible, and accessible information on corruption by punishing
politicians at the polls. Boas et al. (2018), too, detect willingness to punish corruption in a survey
experiment in Brazil, but this does not translate into electoral effects in a field experiment.
Costas-Pérez et al. (2012) examine corruption scandals in Spain and find the strongest electoral
punishment where the press provides extensive information on judicial charges. Perhaps the most
5convincing evidence that voters punish poor and reward good performance is from natural
experiments involving audits of local authorities conducted by public bodies (Ferraz and Finan
2008, Bobonis et al. 2016).
How should we interpret and systematize this varied set of findings? One notable pattern among
empirical studies of information and electoral accountability relates to the difference between
experimental and observational studies. Field experiments have, for good reasons, emerged as
the gold standard of internal validity and causal inference in political science and economics
(Gerber and Green 2012, Glennerster and Takavarasha 2013). In the literature noted above, field
experiments contribute a disproportionately large share of studies reporting null results. One
reason for this is likely to be the impressive commitment of the involved scholars to demanding
standards of research transparency, including pre-analysis plans, to combat publication bias
(Franco et al. 2014, Dunning et al. 2018). In other words, if research transparency is highest
when field experiments are involved, null results are less likely to disappear in the file drawer
and we are more likely to get to know about them.
Yet there could be other reasons. Specifically, we emphasize the continued importance of
observational studies with the potential to capture the effects of information at full scale. These
can capture dynamics that may not be activated in studies at smaller scale, where information
treatments are delivered across only a subset of localities within a larger political system, and
usually through irregular or unfamiliar channels. Experimentally manipulated information-based
interventions at the locality level may not always be appropriate where the true treatment effect
of interest pertains to nationwide counterfactual settings with and without additional information
available. This is because information at a system-wide scale sets in motion social processes that
may not be present in smaller-scale studies. Such “scale effects” can result in effects from small-
scale studies not holding when an intervention is scaled up (Banerjee and Duflo 2009:167-169,
Deaton and Cartwright 2018: 15-16). Yet the reverse is possible, too, and especially so in the
case of information-based interventions. In particular, the dissemination of information at full
scale is likely to affect its transmission, salience, and common knowledge.
6First, government performance information at full scale reaches citizens through transmission
channels that are regular and familiar: usually through news media or political organizations.
Information that arrives through irregular or unfamiliar channels may be less credible or less
likely to be understood. In the extreme, citizens may distrust or ignore information disseminated
by unfamiliar researchers or non-governmental organizations. When it takes time to build trust
and to demonstrate the value of new information, one-off interventions may not capture effects
that might materialize in mature transmission environments.
Second, government performance information at full scale is more likely to be salient to citizens.
There is only limited space on the public agenda of media attention and political discourse. As
such, it matters when performance information explicitly takes up space on this agenda, at the
expense of other topics. Citizens may be more likely to take notice of such information in the
first place, to place it in an electorally-relevant context, and to give it more salience vis-à-vis
other considerations (McCombs and Shaw 1972). Further, nationwide media and political
attention can have amplifying effects, non-linearly increasing the salience of information.
Third, government performance information at full scale can also become common knowledge
through the recognition that other citizens in other locales have access to similar information.
Common knowledge may be particularly important in shaping electoral accountability, given the
well-known collective action problems inherent in anti-corruption efforts (Persson et al. 2013).
Gottlieb (2016) sought to vary common knowledge based on the proportion of each community
receiving the information treatment, and Arias et al. (2017) using a loudspeaker announcing the
delivery of leaflets in treated communities. However, common knowledge across geographic
jurisdictions is distinct from that within communities.
These features are crucial to real-world dynamics of political information that might spur voters
to reward strong performance or punish weak performance. Yet it is difficult for smaller-scale
experimental studies (despite their clear inferential benefits) to adequately incorporate them, and
researcher manipulation of elections at larger scales is generally both impossible and unethical.
These features, instead, point towards the importance of supplementing experimental work by
also observationally studying the effects of large-scale government performance information in
7contexts where specific institutional circumstances create as-if-random variation. However, the
rarity of such conditions is reflected in the continued paucity of such research since the
publication of Ferraz and Finan’s (2008) now-classic study.2 Our goal is to extend this research
by developing a novel identification strategy suitable for new contexts.
Beyond transmission, salience, and common knowledge, other aspects merit discussion. The
source of information can be particularly important for its potential electoral impacts. Botero et
al. (2015) find that participants in a survey experiment in Colombia responded more to
corruption allegations presented by the leading (and non-partisan) national newspaper, than by a
non-governmental organization or the judiciary. Weitz-Shapiro and Winters (2016) show that
voters discern between more and less credible information sources, and that a country’s supreme
audit institution can be a highly trusted source of impartial and reliable information on
government performance. However, it is feasible for smaller-scale experimental studies to
address this, for example by clearly sourcing information from existing official audits (Chong et
al. 2015, Buntaine et al. 2018).
There are also potential large-scale dynamics that suggest dampened effects of government
performance information, even in a context where smaller scale field experiments had identified
positive treatment effects. These might include negative spillovers, differential effects that cancel
each other out in the aggregate, or general equilibrium adjustments by politicians (Banerjee and
Duflo 2009, Glennerster and Takavarasha 2013: 57-59).
These possibilities further highlight the importance of studying the electoral effects of
information at system-wide scale. In order to do this, we focus on information from independent
2
Notable exceptions are Larreguy et al. (2014) on Mexico and Bobonis et al. (2016) on Puerto Rico.
Another valuable line of research leverages the incidence of scandals to study voter responses to
corruption (e.g., Costas-Pérez et al. 2012, Eggers 2014, Fernández-Vázquez et al. 2016, Larcinese and
Sircar 2017). However, it is often more difficult for such studies to fully address concerns over selection-
into-treatment, to discern the effects of information from the effects of scandal itself, or to consider
positive as well as negative information. Some of these difficulties also apply to an earlier literature on
economic voting (e.g., Duch and Stevenson 2008).
8financial audits of local authorities. A surprisingly small number of studies assess the electoral
impacts of audits in a rigorous way (de Renzio and Wehner 2017). Ferraz and Finan (2008)
examine how the public release of audit reports on federally transferred funds affects the
reelection prospects of mayors in Brazil. They exploit that some municipalities were randomly
audited prior to elections and some afterwards. Where pre-election audits revealed violations
associated with corruption, their publication significantly reduced reelection probabilities,
especially in municipalities with local radio stations (see also Brollo 2010, Ferraz and Finan
2011, Avis et al. 2018). For Mexico, Larreguy et al. (2014) show that local media amplify the
effect of audit results published ahead of mayoral elections. Bobonis et al. (2016) find that
mayors in Puerto Rico are less corrupt when audits are scheduled prior to elections, thereby
boosting their probability of reelection, but this decrease is not sustained in later audits. These
studies suggest that audits can reduce mismanagement when they are regularly undertaken and
timed to facilitate accountability in elections. This is a particularly important avenue of study to
better understand how institutional design can generate incentives for good governance.
The South African Context
We examine the electoral effects of audit information in a new context, South African local
government. Following the country’s transition to democracy, the 1996 Constitution charged
municipalities with providing essential basic services, such as electricity, water and sanitation,
and local planning and development functions (Kroth et al. 2016). Following a restructuring of
local governments in 2000, Parliament passed new statutory frameworks to improve municipal
financial management and service delivery. Yet, progress remained disappointing. In 2009, the
newly-elected government of President Jacob Zuma of the African National Congress (ANC)
launched a high-profile and ambitious turnaround initiative, “Operation Clean Audit 2014.”3 It
promised “clean audits” – certifying flawless financial and other reporting and full compliance
with legislation – for all provincial and local governments within five years (Appendix Table A1
3
This followed an earlier initiative, “Project Consolidate,” launched by President Thabo Mbeki in 2004.
9provides full definitions of audit outcomes). At the end of the period, the target was missed by a
wide margin (Appendix Table A2, Powell et al. 2014).
The audit process is set out in the Municipal Finance Management Act of 2003 (sections 126 and
127). Within two months after the end of the financial year on June 30, the accounting officer of
a municipality has to submit financial statements to the Auditor-General, the country’s
independent auditor. The Auditor-General has three months upon receipt of the statements to
complete an audit and to submit the results to the municipality. In other words, audit results for
the financial statements of a municipality should be finalized by the end of November. No later
than seven months after the end of a financial year, the mayor of a municipality must table an
annual report including the outcome of the audit in the municipal council. In practice, not all
municipalities observe these deadlines and audits receive little attention until months later.
South Africa’s 1996 Constitution establishes the Auditor-General as one of six institutions meant
to “strengthen constitutional democracy” (section 181). The Auditor-General has to audit and
report on the accounts, financial statements, and financial management of national, provincial,
and municipal governments (section 188). All audit reports must be made public. Following the
reform of local government, the Auditor-General started to publish increasingly detailed reports
on local management and finances, and to highlight “unauthorized”, “irregular”, and “fruitless
and wasteful” expenditure. Since 2009, the Auditor-General has publicly released audit results
for all municipalities pertaining to the financial year ending roughly one year prior, on a date
falling in the winter of each year – usually June or July.
The findings from municipal audits are widely reported in the national and local media,
including newspapers,4 radio, television,5 and the internet.6 The Auditor-General holds a series of
4
E.g., “All eyes on auditor-general’s local government audit”, Sunday Independent, August 11, 2013.
5
E.g., “Auditor-general's report shows irregular expenditure in municipalities”, SABC News, June 1,
2016, available at https://www.youtube.com/watch?v=kV33m9j6Ylw (accessed April 19, 2018).
6
Eyewitness News, an independent news publisher, established a “Municipal Audits” website including a
database of audit findings and related news (Appendix Figure A2).
10events to publicize the results and to maximize their local salience.7 Figure 1 plots newspaper
articles that mention terms narrowly indicative of municipal audit results (Appendix Figure A1
provides an overlay version of this plot). The data represent seven-day rolling sums of the
articles detected by the search and the grey shaded areas indicate 90-day periods following the
release of the Auditor-General’s report. The seven highest article counts all immediately follow
winter release dates. The absence of these mid-year spikes before the introduction of this report
in 2009 shows how it has heightened media awareness of local audits. In addition, municipalities
and political parties prominently advertise when they receive a “clean audit.”8
7
E.g., “A-G’s roadshow paints a bleak picture”, Diamond Fields Advertiser, March 30, 2012. The article
explains: “More than half of all Northern Cape municipalities will probably receive a disclaimer of
opinion from the Auditor-General for the 2010-11 financial year… Opposition parties have called for
heads to roll.”
8
E.g., on April 8, 2016, Ekurhuleni Metropolitan Municipality placed a half-page advert in a national
weekly newspaper, the Mail & Guardian, to publicize the upcoming “State of the City Address” by the
executive mayor (Appendix Figure A3). In capital letters, it boasted with a “CLEAN AUDIT 2014/2015”
and added: “The City of Ekurhuleni is proud to achieve another Clean Audit.” Local government
elections were scheduled to take place later that year, and the governing ANC was under pressure to
defend its majority in the municipality. On its website, the opposition Democratic Alliance (DA) has used
clean audits in municipalities where it governs as independent verification of its good governance
credentials (Appendix Figure A4).
11Figure 1: Media attention to municipal audits spikes after the release of the Auditor-General’s report
Number of Articles (Seven−Day Rolling Sum)
70
60
50
40
30
20
10
0
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017
Year
Note: The grey shaded areas indicate 90-day periods following the release of an Auditor-General’s summary report on municipal audits. The number of articles
was calculated as the seven-day rolling sum of newspaper articles extracted from Sabinet’s SA Media database over the period January 29-31, 2018, using the
following search query: (municipal OR municipality OR "local government") AND (audit OR "auditor general" OR "irregular expenditure" OR "unauthorised
expenditure" OR "fruitless and wasteful expenditure"). At the time of data extraction, the database covered 38 mainstream South African media publications with
national or regional distribution. A change in Sabinet’s collection procedure resulted in a gap in coverage from late 2014 to early 2015, visible as the flat line
over that period.
12In 2018, the South African Citizen Survey asked a representative sample of South Africans about
municipal audits and the Auditor-General. According to the weighted responses for March and
April 2018, a surprisingly large number of respondents, 24 percent, claimed they had heard,
seen, or read about the audit results of the municipal audit carried out for their local municipality,
while 25 percent claimed to have heard, seen, or read about reports of unauthorized, irregular, or
fruitless and wasteful spending by their municipality (with 35 percent having heard of either). As
part of our analysis below, we test whether the release of the Auditor-General’s report affects
these responses. Moreover, excluding those who said they did not know, 56 percent had “a lot
of” or “some” trust in the Auditor-General, against 42 percent for their mayor. A massive 59
percent of respondents said that information about corruption and financial mismanagement in
their municipality was “very” important, and 25 percent “somewhat” important, in deciding
which political party they will vote for in a municipal election or by-election, while merely 16
percent said it was either “not too” or “not at all” important.
The audit process produces two potential annual “treatments” by which information is made
publicly available: once when the audit outcome information is released locally, and once when
it is released nationally. In practice, however, the local releases are not tightly clustered around
the January deadline and often are neglected altogether. Well-run municipalities compile high-
quality accounts and submit them for audit swiftly, with publication months prior to the deadline,
while others fail to do so on time or at all. In interviews we conducted, auditors, politicians, and
municipal officials emphasized that local audit results receive systematically heightened and
concentrated media and public attention following the later, nationwide release.
Municipal elections are held every five years. The first elections to the restructured municipal
councils took place in late 2000, followed by regular elections in 2006, 2011, and 2016. While
national and provincial elections involve only closed party lists, the Municipal Structures Act of
1998 provides for a mixed-member proportional representation system (section 22). Half of the
seats on a municipal council are elected under plurality rule in single-member wards and the
other half based on proportional representation using party lists, with the overall allocation of
seats proportional to each party’s percentage of the total vote. Our focus is on ward seats, for
which by-elections are held to fill vacancies. Municipal management is under the control of
13mayors, who are not directly elected but rather chosen by a majority on the local council, in
some cases (roughly 15% immediately following the 2011 local government elections) requiring
a coalition where no party holds a majority of seats.
The Municipal Structures Act sets out specific triggers and strict timeframes for by-elections
(section 25). These have to occur when the Electoral Commission (IEC) does not declare the
result of an election, a court sets aside an election result, a council is dissolved, or a vacancy in a
ward occurs. A by-election has to be held within 90 days of the triggering event.11 Ward by-
elections are triggered when a councilor resigns, is no longer qualified,12 is removed from office
due to a violation of the Code of Conduct for Councilors, changes partisan affiliation, or dies.
Research Design
South Africa’s Auditor-General audits every municipality in each year. This creates a challenge
for evaluating the electoral effects of this information, as regular elections are also held for all
municipalities at the same time, every five years. If we were to compare the performance of the
mayor’s party in local elections between municipalities with good and poor audit results, we
would have no control group where citizens received no information. Our solution is to focus on
by-elections (Appendix Table A3). From 2007 to 2015, there were a total of 949 by-elections in
209 municipalities against a total of 234 local and metropolitan councils as of 2011. These by-
elections take place irregularly throughout the year and the IEC groups several on the same day,
generally every few weeks. The average by-election date saw elections for new councilors in
about ten wards around the country, and roughly ten such dates were held each year on average
during the period we examine.13
11
Strictly speaking, a by-election has to be held within 90 days of the IEC receiving notice. In only a
handful of cases municipal managers appear to have attempted to delay notification for strategic reasons.
12
Membership is regulated in the 1996 Constitution (section 158).
13
Since a Constitutional Court ruling in 2016 clustering of by-elections has become more difficult, since
the Court required the IEC to improve registered voter data. According to electoral officials this translated
into a higher number of by-election timetables, with fewer ward by-elections per timetable, and a higher
14We leverage this setting to compare by-elections held shortly after the annual release of
municipal audit results – when information closely related to corruption and financial
mismanagement is available, prominent, and timely – with by-elections held during the rest of
the year. While the total number of by-elections is very small when we examine a narrower
window just before and after the release dates, we also pursue this approach in a robustness
check. As illustrated in Figure 2, our “treatment” of interest is the presence of timely and salient
audit information at the time of a by-election. Our primary treatment window is the 90 days
immediately following each audit release. We vary this period to validate that recency matters.
Figure 2: Treatment assignment of by-elections
Note: By-elections E are triggered by the death of an incumbent ward councilor and occur at
time t throughout the year. Here, the third and fourth by-elections are “treated” as they fall into
the period of x days following the release of the Auditor-General’s municipal audit report.
To support a causal interpretation of our results, our analysis is based on by-elections triggered
by the death of an incumbent councilor who was elected to represent a particular ward. Other
triggers involve far greater potential for strategic timing or reflect local conditions (Jones and
average number of days to fill a ward vacancy of around 85 days, against about 80 days prior to the ruling
(email communication).
15Olken 2005). Our identifying assumption is thus that our information treatments can be
considered “as-if” random, exogenous to by-election outcomes or the underlying differences in
the location or context of by-elections. In other words, the timing of the by-elections in our
sample relative to the national release of audit information has to be random. The remainder of
this section probes this assumption.
Given that our observations are triggered by either deaths or by human decisions, and the
potential for concerns over the timing of both by-elections and the audit release date, we consider
several possible threats to this identification strategy. Dunning (2012) suggests a focus on three
factors in evaluating the plausibility of an as-if random natural-experimental treatment:
information, incentives, and capacities. In this case, municipal politicians do indeed know their
audit results before the national release date. They also plausibly have incentives to manipulate
timing: The governing party of a municipality would presumably prefer to face by-elections
ahead of the release date if the results will reflect poorly on their performance, and after the
release date if the results will reflect positively.
The matter of capacities becomes far more complicated, however. First, a by-election is triggered
by some incident or human action: A councilor dies in office, chooses to resign, is promoted to
higher office, or is terminated or expelled by some other actor – the council as a whole, their
political party, or a provincial authority. Deaths in office are presumably randomly timed, but
other causes of by-elections might be more or less likely depending on other factors, such as the
quality of municipal management or the intensity of political competition. Of greater concern
would be potential strategic timing of resignation, termination, or removal from office, in order
to manipulate whether by-elections fall either before or after the release date. As local councils
already know their audit results ahead of the national release date, a governing party might prefer
a by-election to be held when they can gain maximum advantage from this information.
While individual councilors have the capacity to strategically time their resignation, and other
actors have the capacity to strategically time expulsions or terminations, several other factors
complicate this possibility. First, the timing of by-elections is governed by national election law
and is largely out of the hands of individual municipal governments. By-elections are required to
16be held within three months of the triggering event and grouped by the IEC in clusters as noted
above. An individual councilor thus cannot predict when the by-election to replace them would
be held based purely on the date of their resignation, as this would also depend on how many
other by-elections get triggered around the country, and how the IEC decides to group these.
While the IEC theoretically could attempt to manipulate a timetable (on which several by-
elections are held around the country) so that elections fall just before or after the audit release
date, this would require overcoming a coordination problem between the diverse local needs of
the ruling ANC in different municipalities. In one municipality the local ANC might prefer the
by-election be held before audit information is available (either because of their own poor
performance if they govern, or the good performance of the governing party if another party
governs), while in another they might prefer the by-election be held after the audit information is
available. Municipalities with by-elections are likely to fall into different provinces, so any
collusion to time by-elections strategically would require coordination across different provincial
authorities with their own interests and incentives. Moreover, the 90-day deadline allows very
little slack (footnote 13). Taken together, these conflicting incentives, coordination challenges,
and the tight deadline all make it exceedingly difficult, arguably impossible, for the IEC to act in
this way – even if its independence were too weak to resist political pressure to attempt this.
Another possibility is that the Auditor-General’s national release date itself might be strategically
manipulated for political gain, particularly by the nationally-governing ANC. Might the ANC
pressure the Auditor-General to delay the release by a few weeks until after a series of
competitive by-elections had passed? We do not think this is plausible, given the independence
of the Auditor-General and the same coordination problem across different local conditions
discussed above regarding the IEC. In practice, the annual release date for the municipal audit
report is set by the Auditor-General in an internal planning meeting held in September or
October, before work on the report commences in December. A project plan gets drafted on the
basis of the release date the Auditor-General chooses at this meeting. This decision is thus taken
months before the dates of any relevant by-elections are known. Moreover, this also means there
is no scope for any actor to attempt to strategically manipulate the outcomes of individual audits,
which are finalized before a by-election is triggered.
17Additionally, we can observe a most-likely case for political pressure to succeed, in which the
Auditor-General nonetheless did not move the release date. The 2016 municipal elections,
originally scheduled for February, had to be delayed until August due to a Constitutional Court
ruling about updates to the voters roll. These elections, in which every local council would be
reelected, were of extreme national importance and ultimately delivered the largest electoral
blow yet to the ANC’s hold on power (Olver 2017). However, while the elections were
originally scheduled to take place before the national audit release in June, the court ruling meant
that they would instead take place shortly afterwards. If ever there were a time for the ANC to
pressure the Auditor-General to delay the audit release date, this was it. That the national audit
release instead took place as scheduled, attracting substantial news coverage in the context of the
upcoming elections, was testament to the Auditor-General’s independence and shows the limited
potential for such manipulation.14
It is equally unlikely that local politicians or the ANC can manipulate media coverage of the
Auditor-General’s report, for example by delaying the transmission of this information to voters
or suppressing specific results. South Africa has a vibrant and diverse media landscape. The
Press Freedom Index produced by Reporters Without Borders acknowledges threats to media
independence but ranks the country similarly to established democracies such as the United
States and the United Kingdom.15 Figure 1 confirms that media transmission of the audit
outcomes follows rapidly on the release date of the Auditor-General’s report.
We thus expect that the primary threat to our research design comes from strategic timing of
individual decisions to resign or to terminate or expel councilors, although the impact of these
will be blunted by the limited capacity for individuals to manipulate the timing of by-election
14
As City Press noted on June 5, 2016: “The release on Wednesday of the Auditor-General’s report on
municipalities could be the worst timing for political parties trying to win in the local government
elections.”
15
In the 2016 edition, South Africa ranked 39 out of 180 countries, comparable to the United Kingdom
(38) and the United States (41). Available at https://rsf.org/en/ranking/2016 (accessed December 10,
2018).
18dates themselves. For by-elections triggered by deaths, the possibility of manipulation is most
remote. To assess this further, we examined balance between by-elections inside of 90-day
windows following each regular audit release date, and all other by-elections outside those
windows, on several ward- and municipality-level covariates (Appendix Table A4). Across all
by-elections, some imbalanced covariates suggest a potential for strategic timing of by-elections
around audit release dates. For by-elections due to death only, we see fewer imbalances and only
two are statistically significant at the 10 percent level – roughly what could be expected by
chance. This is consistent with our expectation that deaths in office are randomly timed relative
to the release dates of the audit report.
While most deaths in office are due to natural causes or accidents, the relative timing of which
are random, this might conceivably not be the case for murders. One investigation into allegedly
“political” murders of local councilors, politicians, or officials over a period of more than one
year linked these to specific local corruption issues, jobs, business interests, disputes involving
taxi operators, robbery, or unfortunate cases of mistaken identity.16 While exact motives are
difficult to pin down, not a single murder of a ward councilor was linked to the publication of
audit results. Hence, any potential for the strategic timing of deaths in relation to the Auditor-
General’s national release date appears highly implausible. Most of these allegedly “political”
murders took place in the province of KwaZulu-Natal, and in supplementary results discussed
later we check whether by-elections in any particular province affect our findings.
We also conduct McCrary (2008) sorting tests, using the time to or from the nearest national
audit release date as the running variable (and excluding the years in which no release took
place). This test finds evidence of sorting (with more by-elections ahead of the release date)
across all by-elections, but no evidence of sorting across by-elections due to deaths only
(Appendix Figure A5).
In the case of by-elections due to death, we thus have strong reasons to believe we can treat the
availability and prominence of information in the window following the national audit release as
16
See “Op-Ed: In Search of a Political Murder”, Daily Maverick, October 20, 2016.
19a treatment that is “as-if” random with regard to potentially confounding factors. In particular,
while a direct relationship between audit outcomes and voter behavior would conflate the effect
of audit information with any indirect effects of the underlying quality of municipal government,
a focus on by-elections held after the audit release allows us to isolate the effect of the
information itself.
Data and Specification
We combine data on elections and by-elections from South Africa’s IEC and on audits from the
Auditor-General of South Africa, supplemented with municipal-level information collected from
Statistics South Africa and other sources.
Our main outcome of interest is the vote share of the party that governs the municipality, but we
also examine voter turnout. Turnout is relatively direct – does the information available about the
quality of municipal government spur more or fewer registered voters to turn out to vote in by-
elections? This question is of particular importance given that turnout is much lower in by-
elections than in regular local elections (across all by-elections, the average ward saw turnout
drop from 54.7 percent in the previous regular local election to 38.3 percent in the by-election).
Voter turnout is observed for all by-elections, except in the 8.5 percent of by-elections which
were uncontested. In these cases, only one party fielded a candidate, and no actual by-election
was held. As these situations offer no voter behavior to study, we must treat them as missing.17
Assessing who citizens vote for in by-elections, however, raises some thorny issues regarding
attribution. Unlike in mayoral elections in Brazil, mayors in South Africa are indirectly elected.
The incumbent party in any given ward is thus not necessarily the party responsible for
municipal government (although, given the dominance of the ANC, this is the case in the
majority of observations). Further, as we examine by-elections to replace deceased councilors,
17
We found no differences in the proportion of uncontested by-elections inside and outside the post-
release window.
20incumbents are not on the ballot. Audit information is thus attributable not to the incumbent
party for a particular ward council seat, but rather to the party of the mayor who governed the
municipality at the time to which the audit pertains, and it is this party that citizens would either
punish or reward for weak or strong performance. We thus measure the “relevant” or
“responsible” party’s vote share in each ward, even if this party is not the incumbent party in that
ward. For observations with coalition councils we researched the party affiliation of the
individual mayor at the close of the fiscal year to which the audit applied (as no centralized
source of data on the nature of specific council coalitions yet exists in South Africa).
Attribution is most straightforward where a municipality was run by a single party that held a
majority of council seats throughout the relevant fiscal year to which the audit applies and where
this party is still in charge at the time of the by-election. Conversely, barriers to attribution may
arise when, during the relevant fiscal year covered by the audit, a municipality was governed by
coalition or had multiple mayors from different parties, or when the mayor’s party has changed
in the interim period since the end of the relevant fiscal year. In 77 percent of by-elections
triggered by death, none of the aforementioned barriers to attribution were present. One of our
robustness checks assesses whether these barriers affect the results.
Our main independent variables are an indicator for a by-election held in the 90 days following a
national audit release date, a measure of audit performance, and the interaction between these.
National audit release dates are the date the Auditor-General presents his report on municipal
financial management to the public. Audit outcomes fall into one of several categories defined in
Appendix Table A1. In some cases, later audit reports update previous years’ findings with new
information, particularly where outstanding information is submitted by municipalities at a later
date. However, we use only the contemporaneous audit outcomes as known and publicized at the
time of each national release date. The Auditor-General employs a system of color-coding for
reports, in part to improve the ease of interpretation of audit results and their use by citizens
(Fung et al. 2007). Clean audits are identified with green, unqualified with yellow, qualified with
purple, adverse with pink, disclaimer with red, and outstanding with grey (Appendix Figure A6).
While this terminology may be confusing, it features regularly in South African media coverage,
as audit outcomes are reported several times a year for various entities (including, at other times
21of the year, national and provincial departments). Moreover, to aid communication, the reports
use a traffic light system and arrows to categorize movements in performance over the prior year
as either improved, unchanged, or regressed (e.g., Auditor-General of South Africa 2016: 19).
This approach abstracts from the underlying technical categories by focusing on the trend in
performance, which is more intuitively understandable.
Our coding of audit outcomes is based on changes over time. These contain new and salient
information that has the potential to enable voters to reward the governing party for
improvements, or to punish deteriorations. Using the official hierarchy of audit outcomes
described above, we adopt the Auditor-General’s trichotomous representation of movements
over time, with values of one for an improvement, negative one for a decline, and zero for the
same audit outcome as in the previous financial year. This also resembles “good news” and “bad
news” information treatments in experimental work meant to prompt voters to update their prior
beliefs (Dunning et al. 2018). In our sample, 13 percent of observations involve deteriorations in
audit performance, while 26 percent are improvements.
Our main model is as follows:
DVote sharew,m,t = a + b1DOutcomem,t + b2Windowt + b3(DOutcome x Window)m,t + ew,m,t
where the change in Vote share is the vote share in a by-election in ward w of municipality m at
time t of the party of the mayor of the municipality who was in power at the end of the fiscal
year to which the audit applies, minus the vote share of that party for that ward in the last
countrywide regular municipal election. The change in Outcome is our trichotomous measure
and captures the latest year-on-year change in the audit outcome for municipality m that should
be available at time t. We assume that a municipal audit has been finalized according to the
statutory timeline within five months after the end of a fiscal year, meaning that the information
referenced by Outcome updates every December 1. Window is an indicator of whether a by-
election falls within the post-release window, so that it is treated with timely audit information.
We set the post-release window to x days following the nationwide release of the Auditor-
General’s municipal report, where x is 30, 60, 90 or 120, with 90 as our default. We cluster
22standard errors at the by-election date level, the unit of “as-if-random” treatment assignment, and
report robustness checks with alternatives. To gain precision, we also account for the relevant
initial vote share (in the last regular election) and allow for province and year-specific shocks.
An alternate model dispenses with the assumption of linearity and examines deteriorations and
improvements in audit performance separately. In addition, we report a range of robustness
checks below. The Data Appendix contains variable definitions and data sources.
We are interested in the coefficient on Window, and how it is conditioned by audit performance.
Specifically, retrospective accountability leads us to expect the vote share of the responsible
party to get a boost from timely news revealing improved audit performance, and for it to drop
following news of a deterioration. It is less obvious what impact we should expect when voters
receive timely information that an audit result is the same as in the previous year. The period we
examine was characterized by deep and widespread discontent with the dismal state of local
governance, including a wave of “service delivery protests” across municipalities that reached
“insurrectionary proportions” in some cases (Alexander 2010: 37), and the failure of the vast
majority of municipalities to achieve “clean” audits (Appendix Table A2). Where stagnation
implies the continuation of a poor status quo, it may get punished as well.18
18
E.g. “Service Delivery Protests in South Africa”, CCTV News, February 8, 2014, available at
https://www.youtube.com/watch?v=guNv1-bNeOE (accessed September 26, 2018). In this example,
community members explain their protest with a lack of progress. Says one: “There is no change at all!”
23Results
We first evaluate the average effect of a by-election falling into the post-release window, by
omitting the interaction term (column 1 of Table 1). The coefficient on Window is negative but
falls short of significance at standard levels. This is to be expected, as the impact of the
information treatment is likely to depend on the nature of the news that it conveys. Next, we run
the basic interactive model specified above (column 2). The coefficient on Window now captures
the impact of a by-election being treated with timely information conveying stagnation in audit
performance, estimated at more than -3 percentage points. The coefficient on the interaction term
shows a difference of 5 percentage points when an audit conveys improved performance. The
joint effects calculations at the bottom of the table – the sum of the two coefficients – yields an
overall positive impact of timely news of an improved audit outcome, although the estimate is
imprecise. Conversely, our coding implies an equivalent difference in the opposite direction for a
decline in audit outcome, yielding an overall reduction in the relevant party’s vote share by more
than 8 percentage points. This pattern strengthens in size and significance once we account for
province and year-specific shocks (column 3) and the initial vote share of the relevant party in
that ward in the last regular municipal election (column 4).
Finally, our baseline model assumes that positive and negative news effects are equal in
magnitude. Splitting the audit change variable into two separate indicators (column 5) shows that
timely information on deteriorations is punished, and improvements rewarded, by about 5
percentage points, but the effect of deteriorations is not significantly different from that of
stagnation. While the number of deteriorations in our dataset may not be enough to distinguish
these two effects, this result is consistent with a context where new information on a lack of
improvement also constitutes “bad news” and voters respond to it just as negatively as they do to
information on worsened performance. Figure 3 visualizes the effects reported in columns 4 and
5. Overall, we detect sizable impacts of highly-credible audit information on these as-if-
randomly-timed by-elections.
Prior work suggests that recency bias shapes the effect of audits on election outcomes, in that
voters give more weight to recent information than to older information (Bobonis et al. 2016).
24You can also read
