Pre-Analysis Plan (Amendment 4): Citizen Monitoring of Urban Waterways in Jiangsu, China* - Egap
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Pre-Analysis Plan (Amendment 4):
Citizen Monitoring of Urban Waterways in Jiangsu, China*†‡
Mark T. BUNTAINE1 and Bing ZHANG2
1
University of California, Santa Barbara, buntaine@bren.ucsb.edu
2
Nanjing University, zhangb@nju.edu.cn
September 4, 2019
*
July 9, 2019 update: We pre-register this amended design after the collection of baseline data, assignment of treatment, imple-
mentation of the treatment, and initial access to the first round of data collected on water quality. This pre-analysis plan is a minor
amendment that clarifies how we will use survey-based data not yet collected in several places. It does not modify any planned analyses
related to water quality (M1), the only outcome data collected at the time this amendment is filed. It also includes a note about the
interpretation of meta-analysis results prior to the authors having any access to information about results from this study or other
studies in the initiative (section 11.7).
†
August 9, 2019 update: As compared to Amendment 2, this version of the pre-analysis plan makes minor corrections and clarifications
to covariate descriptions encountered while working with code and mock data, particularly a typo in Table 2 that omitted a covariate
from one line only (E13) and variable definitions for rainfall (C1 & C2). We also provide an update on the our use of the authority
surveys based on response rates.
‡
September 3, 2019 update: As compared to Amendment 3, this version of the pre-analysis plan clarifies the procedures that our
team will use to produce meta-analysis results based on conversations with the steering committee. We file this amendment prior to
conducting any analysis with outcome data. At the time of filing this amendment, we have access to survey-based outcome data, but
we have not yet received the second endline water quality measures for H1.
1
Contents
1 Motivation and Background 4
1.1 Implementing Partner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2 Theory 6
2.1 Agency Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2 User Self-Regulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3 Sample 7
3.1 Waterways . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.2 Survey Respondents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
4 Citizen Monitoring 10
4.1 Recruitment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4.2 Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
4.3 Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
4.4 Incentives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
4.5 Encouragement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
4.6 Replacement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
5 Interventions 13
G
5.1 Treatment 1: Disclosure of Citizen Monitoring to Government (D ) . . . . . . . . . . . 13
P
5.2 Treatment 2: Disclosure of Citizen Monitoring to the Public (D ) . . . . . . . . . . . . 14
5.3 Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
6 Randomization Strategy 15
6.1 Blocking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
7 Hypotheses 16
7.1 Primary Hypotheses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
7.2 Hypotheses on Intermediate Outcomes / Mediators . . . . . . . . . . . . . . . . . . . . . 16
7.3 Hypotheses on Heterogeneous Treatment Effects . . . . . . . . . . . . . . . . . . . . . . 17
8 Measurement 17
8.1 Overview of Data Collection Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
8.2 Primary Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
8.2.1 Water Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
8.2.2 Waste Accumulation in Waterways . . . . . . . . . . . . . . . . . . . . . . . . . . 19
8.2.3 Welfare . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
8.3 Mediators & Intermediate Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
8.3.1 Citizen Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
28.3.2 Citizen Knowledge, Attitudes, and Perceptions . . . . . . . . . . . . . . . . . . . 21
8.3.3 Authority Knowledge and Actions . . . . . . . . . . . . . . . . . . . . . . . . . . 22
8.4 Moderators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
8.4.1 Neighborhood-Level Income . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
8.5 Baseline water quality severity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
8.6 Covariates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
8.6.1 Environmental / Meteorological Measures . . . . . . . . . . . . . . . . . . . . . . 23
8.6.2 Individual-Level Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
9 Analysis 24
9.1 Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
9.1.1 Waterway Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
9.1.2 Individual-Level Survey Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . 25
9.1.3 Estimation Contingencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
9.1.4 Estimating Heterogeneous Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
9.1.5 Estimating Mediation Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
9.2 Covariates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
9.3 Ordinal Scales . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
9.4 Sample Restrictions and Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
9.5 Cost Effectiveness and Return on Investment . . . . . . . . . . . . . . . . . . . . . . . . 29
9.6 Attrition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
9.7 Multiple Comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
9.8 Failure to Treat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
9.9 Missing Covariate Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
9.10 Interference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
9.11 Manipulation check . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
10 Qualitative Data and Analysis 31
11 Meta-Analysis 33
11.1 Meta-analysis Hypotheses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
11.1.1 Primary Hypotheses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
11.1.2 Hypotheses on Moderators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
11.1.3 Hypotheses on Intermediate Outcomes . . . . . . . . . . . . . . . . . . . . . . . . 33
11.2 Meta-analysis Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
11.3 Estimators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
11.4 Moderators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
11.5 Match between Project Analysis and Meta-Analysis . . . . . . . . . . . . . . . . . . . . 34
11.6 Notes on Project Contribution to Meta-Analysis . . . . . . . . . . . . . . . . . . . . . . 38
11.7 Theory and Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
31 Motivation and Background
Water pollution is one of the most pressing issues in China. Due to massive industrial wastewater
discharges and the extensive use of agricultural fertilizer, approximately 70 percent of China’s rivers
and lakes are deemed unsafe for human consumption (World Bank, 2007). The Ministry of Water
Resources in China recently released an investigative report claiming that more than 80 percent of
the mainland’s groundwater has been polluted as well (Buckley and Piao, 2016). Water pollution in
China causes significant human health effects, such as digestive cancers (Ebenstein, 2012). In the global
context, there is no place where water pollution affects as many people to such a great extent as China
(Liu and Yang, 2012).
In Jiangsu Province, these challenges are pronounced. Jiangsu is one of the most economically-
developed provinces in China and has a relatively high population density. Having undergone rapid
urbanization in recent decades, small urban waterways in Jiangsu province generally suffer from poor
water quality. These waterways, which are encountered by urban residents as they go about their
lives, are polluted by municipal sewage and garbage. Unlike larger watersheds and rivers, which are
subjected to automated government monitoring, small urban waterways are not well-monitored and
have not received much attention from water resource managers in local Environmental Protection
Bureaus.
Environmental protection has been prioritized at the highest levels of government in China. Envi-
ronmental governance in China has been largely based on command-and-control regulation, inherited
from China’s previously planned economy and its hierarchical political system (Carter and Mol, 2013).
In China’s centralized political system, the central government can create yardstick competition
among local officials by rewarding or punishing them based on relative performance (Zhang, 2006).
The top-down target responsibility system, which holds local officials to account for reaching targets,
is one of the major institutional drivers of this competition. At the center of this governance process
is the nationwide Five-Year Plan that sets overall goals, establishes priorities and allocates targets
to sub-regions. The reduction of pollution now plays an important role in China’s Five-Year Plan.
Local governments have greater authority and resources to invest in both economic development and
in environmental protection (Jin and Zou, 2005), which has opened space for local governments to be
more responsive to concerns expressed by citizens about environmental quality.
There is optimism that citizen involvement in the management of pollution can help to improve
environmental quality in China (Li, Liu and Li, 2012). Both central and local leaders realize that envi-
ronmental degradation causes significant discontent with government and is potentially destabilizing.
As a consequence, efforts are shifting away from simply implementing top-down targets and toward
being directly responsive to the environmental concerns expressed by citizens (Carter and Mol, 2013).
Responsiveness to the concerns expressed by citizens might build trust in government among citizens
and enhance political stability. Small urban waterways are an excellent setting to test whether this
new style of bottom-up, responsive environmental governance leads to better environmental quality.
Unlike the management of industrial pollution in large watersheds, small urban waterways have
not been made a priority within Five-Year Plans. Management action by government therefore may
4depend more on bottom-up pressure on municipal officials driven by public concern. In addition,
cleaning up the urban waterways of Jiangsu requires that local governments better understand the
location of pollution so that they can remediate pollution effectively, such as reconfiguring waste and
storm water systems, undertaking campaigns against household pollution, installing filters in drainage
systems, or restoring natural riparian zones to improve water quality. Additionally, behavioral changes
that result in less littering and dumping from residents can also improve water quality, reinforcing the
importance of bottom-up awareness and action from citizens.
The major challenge of improving water quality in small urban waterways, like many other places
in the world, is that they are currently managed as an open-access resource. There is neither regular
monitoring of this collective resource, nor are there strong social or legal rules to prevent non-point
source pollution of urban waterways. Because of the non-point nature of pollution in urban waterways
and the lower priority they have received in government plans, the recovery of waterways might be
boosted by creating credible citizen monitoring of water quality in waterways that reveals public de-
mand for remediation. Since 2013, the Jiangsu Provincial government has planned to clean-up urban
waterways. Social monitoring by citizens may help overcome information problems that result in poor
water quality and reveal citizen demand for remediation.
1.1 Implementing Partner
The Mochou Ecological Environmental Protection Association (MEEPA) is an environmental
NGO, which was launched by Professor Yaodong Li to protect the environment of Mochou Lake and
the surrounding areas. MEEPA has a people-centered approach to realizing a harmonious coexistence
with nature. Its mission is to eliminate human-caused contamination of the environment by harness-
ing social cooperation and changing human behavior. Currently, MEEPA is committed to promoting
environmental protection at the local level, carrying out environmental education for teenagers and en-
hancing environmental supervision and management by local governments, with special attention paid
to safeguarding the environmental rights and interests of the public. MEEPA has also worked with
media outlets to expose the violations of polluting enterprises and has been involved in the supervision
of the resulting remedial activities by enterprises.
In terms of urban waterways, MEEPA has implemented its Citizen Manager of River (CMR)
project since January 2015. In particular, MEEPA collaborated with the Environmental Protection
Department of Jiangsu Province and selected 10 waterways in Nanjing as pilots to track and monitor
the recovery of water quality. MEEPA recruited Citizen Managers for the 10 rivers and then organized
experts to provide trainings on the responsibilities and activities of CMRs for monitoring water quality.
For each waterway, the Citizen Manager sends monitoring data on water quality to the MEEPA regu-
larly. MEEPA organizes the data into summary reports and submits them to the relevant government
departments. Above all, MEEPA has generated significant and sustained collective action, facilitated
public participation in river protection, and encouraged public participation in efforts to reduce water
pollution through public, round table discussions that bring together citizens and public officials. The
10 rivers that have been part of existing MEEPA monitoring are not included in our sample, but serve
5as the template for our field experiment.
2 Theory
We build our theoretical expectations about how citizen monitoring of urban waterways might
improve water quality from two strategic interactions: one between the central government and local
governments, and one between users (i.e., residents who are potential polluters) of urban waterways.
The central and local governments interact strategically in a principal-agent relationship, while residents
near waterways face a collective action problem that limits the self-regulation of pollution and dumping.
We expect that community monitoring and information dissemination will alter the nature of these
interactions to improve the quality of urban waterways.
2.1 Agency Problems
Consider a local government agency that has a mandate to address pollution, such as the municipal
and county Environmental Protection Bureaus as in the present study. This agency has a primary
incentive to appear successful to higher levels of government. It receives goals related to the remediation
of pollution from higher levels of government and its officials are rewarded for making progress toward
those goals. The higher level of government sets goals based on visible environmental problems that
cause or are thought to cause public discontent. The management of public discontent is the primary
objective of higher levels of government. Thus, in the absence of other pressures, local governments
will use available resources in ways that will most visibly achieve or make progress toward goals set by
higher levels of government.
The local government faces two problems in attempting to meet general goals related to the
remediation of pollution. First, there are many remediation activities that might usefully meet general
goals, but significant information problems exist about the kinds of pollution that are most concerning
to citizens and possible to address, including basic information on the location and severity of local
pollution. More information can thus be useful for optimizing the portfolio of activities that local
governments take to reach goals on pollution remediation to the extent that it makes problems more
identifiable and solvable. Second, goals from higher levels of government are based on managing
discontent caused by environmental problems. Thus, the local government will prioritize activities that
specifically respond to explicit citizen demand for remediation. To the extent that citizen monitoring
provides a strong signal of local demand that higher levels of government will be able to observe, local
governments will have strong reasons to respond to these demands.
Citizen monitoring might solve both of these problems: it provides information on highly localized
and tractable problems that could enter into the portfolio of activities that local governments use to
show progress to higher levels of government. Indeed, finding specific and tangible opportunities for
remediation are likely to be prized by local governments that face overwhelming but diffuse problems
related to environmental management. Citizen monitoring might provide information about specific
areas and solutions that are possible to achieve in relatively short time periods. Second, if information
6about environmental problems becomes public knowledge and thus the basis for public discussion
and discontent, the priority that local governments will attach to addressing such problems identified
through citizen monitoring should be greater, since maintaining a low level of discontent will result in
reward and more discretion from higher levels of government. Discontent among politically powerful
groups, like those who enjoy higher incomes, is likely to be particularly salient for local officials. To the
extent that citizen demands for remediation are observable to higher levels of government, local officials
should also be more likely to respond to demand for remediation. While it is possible that a response
to information and citizen attention to pollution might pull resources away from other activities, we
believe this is less of an issue because local environmental protection bureaus are not significantly
resource constrained and information about tangible and solvable problems is scarce.
2.2 User Self-Regulation
In terms of user self-regulation – the decision by individuals to limit their littering and/or dumping
in waterways – we consider the relevant decision to be a function of the norms in each local setting.
An individual incurs fewer costs of disposal and greater personal convenience if she throws litter in
the waterway and thus will only change behavior if she expects some sort of social or official cost to
dumping or littering. We expect that citizen monitoring might raise such costs in several ways. First,
it makes salient prescriptive norms about what is desired by other members of the community, given
the effort that is exerted by monitors to manage pollution in local waterways. Monitoring should raise
expectations about both detection and social punishment for the violation of this prescriptive norms.
Second, because monitors have official vests and specialized equipment, citizen monitoring may raise the
beliefs about the likelihood of being caught and punished for dumping or littering. Finally, community
monitoring may given communities something to rally around and talk about, creating group identity
or pride about living in a clean environment. People are more likely to comply with social norms
when behaviors are compatible with their group identity and citizen monitoring as envisioned by our
implementing partner attempts to foster an environmentally-friendly group identity. For all of these
reasons, we expect that monitoring of water quality by citizens will decrease the amount of littering
and dumping around waterways.
3 Sample
3.1 Waterways
The units randomized in our study are 160 urban waterways in Jiangsu province. These waterways
are located in urban and semi-urban areas of 9 large and mid-sized cities, from a total of 13 cities
in Jiangsu province. Most of these waterways are close to local communities and can be seen and
smelled by residents on a daily basis. The urban waterways in our sample are typically upstream,
micro-watersheds that feed into larger rivers downstream. In addition, most waterways are located in
different counties within single municipal jurisdictions. As a consequence, the waterways eligible to
be included in our sample are hydrologically unconnected and thus unaffected by pollution control or
7community action in other locations, which avoids concerns about direct interference between units in
our study. To rule out interference and to assist with randomization and analysis, we implemented two
rules to produce the final sample of waterways from the 206 waterways eligible for inclusion:
1. We remove from the sample any waterway that is hydrologically connected directly from another
waterway in the sample, based on an analysis of proximity between waterways. We attempted
to remove the downstream of hydrologically connected waterways based on the likely direction
of flow, but mapping is not always of sufficient resolution to ensure that we have removed all
downstream waterways instead of upstream waterways. We confirmed a lack of hydrological
connection between the remaining waterways during our baseline water quality measurements.
2. We remove from the sample all waterways that are in cities that contain less than eight waterways,
to allow for city-level fixed effects in our analysis. The result is a sample with waterways in 9
cities.1
This process results in 166 waterways, which we use to form our experimental sample. We com-
pleted the base survey and baseline water quality measurement for each of these waterways prior to
assigning treatment. Based on mismatches between the location data from the environmental moni-
toring company and the enumeration teams, we were left with a total of 160 waterways that form our
final experimental sample.
Note that we failed to find any residences within 2 km of two of the waterways at baseline, leaving
only 158 waterways with a baseline resident survey. Nonetheless, these waterways were still monitored
and information was disseminated according to their original treatment assignments. We will attempt
to complete endline measurement at these two sites and will use them as part of the analysis for all
measures we are able to collect at endline.
3.2 Survey Respondents
We will collect data through a repeated cross-section of residents who live near each of the water-
ways and observe their attitudes about pollution control and support for the remediation of pollution,
with the expectation that only the public dissemination treatment will increase positive attitudes, be-
cause it will reveal the demand for better water quality among fellow citizens and change perceptions
of descriptive and prescriptive norms as described in our theory section above. Our enumeration team
walked near each waterway and approached every 4th person with an invitation to take a 10-minute
survey at baseline (see 11.7), after confirming that they live within 2km of the point and are willing
to be contacted again by mobile phone at endline after one year. We offered to conduct the survey
in a non-public place when requested. While we originally designed the survey as a panel with the
intention to recontact these individuals, pilot testing at endline indicated a much lower response rate
than we had originally anticipated ( 25%), so we moved the design to a repeated cross-section, with
waterway-average values from the baseline used as covariates.
1
Surprisingly, our baseline water quality measures do not show evidence of city-level clustering of water quality, so we changed our
randomization strategy and implement blocking based on pre-treatment water quality measures.
8Figure 1: Rivers eligible for inclusion in the sample.
94 Citizen Monitoring
For citizen monitoring to be effective, monitors must understand the monitoring protocol and
have incentives to truthfully and consistently monitor waterways as planned. For the purpose of this
study, monitoring is the collection and dissemination of information about the status of the resource.
Since 2015, MEEPA has created a volunteer base that monitors the water quality of urban waterways
based on simple chemical analyses that can be carried out by trained volunteers, including assessing
odor, taking pH readings from waterways, and measuring Chemical Oxygen Demand (which provides a
measure of organic pollutants in waterways). Under the existing program, data from citizen monitoring
are not disclosed or disseminated systematically by MEEPA. The purpose of the proposed treatments
is to understand how information gained by governments and/or the public from citizen monitoring can
support the development of remedial activities by government agencies and encourage more positive
attitudes about pollution control and less littering and dumping by the public.
4.1 Recruitment
Monitors will be recruited using MEEPA’s existing membership and through its followers on
WeChat and Weibo networks. MEEPA will compile lists of people who are living within walking
distance or a short travel time from each of the waterways in our sample. To the extent that the
existing network of approximately 1,000 individuals who have expressed an interest in getting involved
in monitoring do not cover all of the waterways assigned to treatment, MEEPA will send out staff or
students to recruit volunteers in person for each waterway. MEEPA maintains a local office in most of
the cities in sample, which will organize this kind of recruitment effort if necessary.
During recruitment, every volunteer is asked to register online and so they must have access
to a mobile phone with an ability to communicate by WeChat. They will be asked to indicate which
waterway they would like to monitor, as well as information about their ability to do this job. Monitors
will be screened to ensure that they meet four basic characteristics required for successful, accurate, and
consistent monitoring of waterways for at least one year based on an interview with MEEPA managers.
Preference will be given to people who have expressed a long-standing interest in being involved with
social monitoring. If there are too many monitors, MEEPA managers will select monitors who first
expressed interest in becoming river guardians and who meet the basic qualifications.
1. Current affiliation: the volunteer should not have conflicts of interest, such as employment as
a government official or be employed by a developer in the area.
2. Time available: The volunteer should prove they have enough time to conduct monitoring,
especially that they do not have extensive plans for travel. The monitor should also confirm
that they do not have any major health issues that would affect their ability to offer consistent
monitoring.
3. Health: The volunteer should be not younger than 18 years old and the volunteer should be in
good enough health to move around near waterways without risk of injury.
104. Motivation: The volunteer should make a stated commitment to consistently monitor the wa-
terway assigned every other week for at least one year (study period).
4.2 Training
The goal of training is to ensure that monitors fully comprehend the measurement protocol and are
sufficiently motivated to complete it accurately and consistently. MEEPA will train three volunteers
for each waterway (80 waterways will be assigned to citizen monitoring) to conduct twice-monthly
monitoring of water quality. We use multiple monitors to decrease sampling variation and to reveal
persistent demand from volunteer monitors for better environmental quality. Volunteers will monitor
water quality by sense and a simple test kit. In particular, they will monitor water quality by measuring
or observing color, smell, cleanliness, pH, dissolved oxygen (DO), chemical oxygen demand (COD),
ammonia, nitrogen, and phosphorus content according to existing protocols.
The monitors will be trained according to the existing protocols of MEEPA, which includes a
normative component and a technical component. The normative component focuses on the importance
of having “river guardians” who have a passion for environmental quality and for human and ecological
well-being. During the training, all volunteers will trained to take measurements of water quality
using a pre-specified test card. In accordance with MetaPAP guidelines, the training will include the
following components:
1. Verification of monitors eligibility at training sessions.
2. Background information on the purpose and content of monitoring.
3. Motivational material emphasizing the responsibilities of the role.
4. Conceptual training on the methods employed to measure water quality.
5. An opportunity to complete the full monitoring protocol both in the presence of trainers and
independently, with a chance to ask questions and receive feedback.
6. Training on how to submit the information to MEEPA for further dissemination, including a trial
demonstration and an opportunity to complete the process independently, ask questions, and
receive feedback.
4.3 Monitoring
Each trained monitor will be provided materials to monitor the waterway in their neighborhood
on a twice-monthly basis and instructed on how to report the resulting information to MEEPA though
WeChat by taking a photograph of their water quality monitoring card and submitting it to MEEPA
and the research team via WeChat. All of original monitoring documents will be sent to MEEPA at
the end of project, with digital copies used to compile the data for reports and to check on compliance
with the protocol. All monitors are asked to collect data at same monitoring site, which MEEPA and
the NJU research team will make available in the form of maps. All data will be double checked by
11MEEPA to ensure completeness and follow-up will occur for any monitors that show indications of
non-compliance with the protocol. Every monitor will ask to start monitoring during the same week,
even though the training of monitors will be staggered by city because of staff availability. MEEPA will
alert all monitors on Monday of the weeks when they should begin and continue taking measurements.
During the course of implementation, MEEPA and staff at Nanjing University checked all of that
all of the submitted data was reasonable given the data collection protocols being used. All volunteer
monitors took pictures of their reporting sheet at the end of each session and sent that information to
MEEPA and Nanjing University by WeChat. The quality controls were reporting checks rather than a
quality checks. For instance, some monitors who submitted values for pH that were more precise than
the measurement equipment allowed were contacted for corrections. Likewise, errors in the placement
of decimal happens less than 1 percent of the time, but were corrected as part of the validation process.
The actual measurements of the monitors were never cross-checked in the field, provided that plausible
values were submitted as part of the semi-monthly reporting.
4.4 Incentives
Most of the monitors will voluntarily join MEEPA and this project, so we expect that they will
have high levels of intrinsic motivation. We will still provide a “lunch stipend” of $2 per monitor
per monitoring session. The goal of the incentive is to avoid non-compliance with the monitoring
assignment at the zone-level and foster social connection among the monitors in each of the waterways
in the sample.
4.5 Encouragement
MEEPA will also create an online WeChat group for online communication between monitors;
reminding monitors of their responsibilities and sending encouragement regularly to all monitors. Be-
cause all recruited monitors will be part of this ongoing encouragement and reminder network, we
ensure that the encouragement is standardized across waterways in the study. Quarterly we will send
all monitors copies of the reports that we send to the government, so that they can see that they
are contributing to actual information that is disseminated, which is likely to promote more sustained
engagement. Monitors will also be alerted to the public dissemination of results over the WeChat
and Weibo platforms. On a regular basis, MEEPA will disseminate by WeChat encouragements to
monitors and reminders about the environmental values that they are pursuing. This information will
be managed in a uniform, but adaptive ways to all monitors by MEEPA, as context and local events
dictate.
4.6 Replacement
We will replace any monitor who explicitly drops out of the program or who does not provide a
monitoring report for a period of four weeks. We may hold a monthly “replacement” group training
session if this becomes a major issue, but will conduct individual-level training to the extent that it
12occurs only sporadically. Based on MEEPA’s existing experience organizing the activities of monitors,
we do not expect high rates of attrition from monitoring.
5 Interventions
5.1 Treatment 1: Disclosure of Citizen Monitoring to Government (DG )
Intervention. MEEPA will submit a summary report to local government offices on a quarterly basis
based on the citizen monitoring information for the waterways in this experimental condition. The
summary report will be a well-formatted document containing all measurements and any trends in the
measurements detected over time by citizen monitoring for each waterway. This report will be delivered
by express mail with a confirmed signature for receipt to the Environmental Protection Bureau, the
Housing and Urban-Rural Development Bureau, and the chief executive office (e.g., governor, mayor) at
each of the province, city, and county level. We will note to each of these offices that a copy of the same
report is sent to the all of the other offices mentioned. The report will highlight citizen-reporting as
the data source, which might increase the perceived demand for stronger environmental management
among government officials. In Nanjing City, the report will also be sent to a special office for the
recovery of waterways and rivers.
Justification. Monitoring is critical to improving the management of open-access resources, as it
creates the information necessary to create institutions and enforce rules. External monitoring and
evaluation (e.g. third-party evaluation, local communities), unlike internal evaluation, is considered to
be a more independent, objective and professional way of government performance evaluation (Conley-
Tyler et al., 2005). Theoretically, the external monitoring could improve government performance
by enhancing official attention (Baekgaard, 2015; Chingos, Henderson and West, 2012; James and
Moseley, 2014). Several observational research studies suggest that rankings, ratings, and performance
targets can influence government actions (Hood, 2006; Ho, 2005; Boyne and Chen, 2006; Kelley and
Simmons, 2015). Nyqvist, de Walque and Svensson (2017) also find that community-based monitoring
on primary health care can improve health care delivery and health outcome by providing information.
Governments often find it efficient to rely on the citizen monitoring to alert them to problems, not
only because it limits resource expenditures, but also because it allows governments to take credit
for addressing citizen concerns (McCubbins and Schwartz, 1984). Related to pollution control, past
research has generated some positive evidence that social monitoring contributes to improved environ-
mental management of collective resources, yet the mechanisms by which information generates these
outcomes is not always clear (Conrad and Hilchey, 2011). One reason why governments might be
positively disposed to act on information provided by volunteer monitors is that the very provision of
this information indicates a public demand for stronger collective management and the development of
collective action potential in the area of pollution control (van Rooij et al., 2013; Johnson, 2014). Given
that the Chinese government is clearly interested in managing areas of public concern where collective
action potential is emerging and where citizen demand indicates room for improvement by governments
13(Lorentzen, 2014), the provision of social monitoring information may both solve monitoring problems
and raise the saliences of addressing the pollution of small urban waterways for officials.
5.2 Treatment 2: Disclosure of Citizen Monitoring to the Public (DP )
Planned Intervention. MEEPA will create a social media feed of the citizen monitoring data using
the very popular WeChat platform, with more than 900,000,000 active users. In collaboration with
Nanjing University, MEEPA will establish a WeChat group for every waterway in the sample and
have monitors post QR codes near waterways that will allow residents to join the groups to discuss the
results of the citizen monitoring activities in their waterway. On a quarterly basis, MEEPA information
collected as part of the citizen monitoring platform about the waterways assigned to this condition will
be posted. Reinforcing reminders about the program and interpretation of information released as part
of prior reports will occur at least bi-weekly to keep engagement in the feeds by residents active. In
addition, MEEPA will also put up posters that contain summary reports of water quality near the
waterways that are part of this treatment condition on a quarterly basis. Previous work has shown
that social media feeds about environmental issues generate intense public attention and often spur
public action (Kay, Zhao and Sui, 2015). For example, as of 2015 the BlueSky app that publishes data
air quality and firm compliance with environmental rules had more than 3.1 million active users across
China (Liu, 2015). Local WeChat groups often grow quickly based on interest in a topic. In general,
social media is the primary way that citizens gain access to environmental information in China, so
this treatment allows us to investigate emerging modes of environmental governance.
Actual Intervention. MEEPA created a WeChat group for every waterway in the sample and posted
QR codes near waterways that will allow residents to join the groups to discuss the results of the citizen
monitoring activities in their waterway. However, the rate of uptake was extremely low (the majority
of waterways has zero joins after the first set of posters were posted). As a consequence, MEEPA
decided not to spend time maintaining these groups and instead focused on putting up posters that
contained summary reports of water quality rankings near the waterways on a quarterly basis. Thus,
the treatment as delivered can be understood as the quarterly posting of posters in prominent places
around each waterway.
Justification. Making information available to the public about the quality of urban waterways
might spur remedial action by changing public behavior and creating a response from government to
public attention and demands for action. Social media, both in western countries and China, has led
political actors and administrative institutions to provide better public goods or services in response to
public mobilization (Sobaci, 2016). It is common for Chinese netizens to express public opinion in the
digital sphere and push governments to make changes. According to the survey by The Governance
Project at Stanford University, 90 percent of Chinese officials agreed that Public opinion can cut
short the career prospects of government officials (Distelhorst, Fu and Hou, 2016). There has been a
great deal of research that deals with the role of public knowledge about and attention to pollution in
14creating responses to shared natural resources, both in China (e.g., Tan, 2014) and in other contexts
(e.g., Tietenberg and Wheeler, 2001). In many cases, public knowledge and attention are critical to a
response from government.
However, residents of developing countries face a number of barriers to accessing quality informa-
tion that can be the basis for mobilization or petitions to government. Misinformation may be more
persistent in developing countries because of a lack of rules mandating the provision of environmen-
tal information. Under these conditions, private willingness-to-pay or mobilization can be distorted in
ways that are not welfare enhancing (Greenstone and Jack, 2015). For example, Jalan and Somanathan
(2008) provide Delhi residents with information about the quality of their tap water and find a signif-
icant change in expenditures on drinking water following the treatment. The accessibility of quality
information can also affect how people form opinions about public policy and mobilize to pressure gov-
ernments for environmental management. Easier access to environmental information enables people
in China to lodge environmental complaints (Dong et al., 2011). Environmental information disclosure
increases citizen’s readiness to participate in environmental activism (Liu et al., 2010). There is some
evidence that environmental activism and expressed public opinion are used by the central government
to monitor the performance of local governments (Distelhorst, Fu and Hou, 2016).
We specifically hypothesize that informed residents will demand better environmental management
from municipal government and that municipal governments will respond to this demand. This is an
important hypothesis for testing in this context, because the most prominent attempts to organize
around environmental issues in China are all based on the disclosure of environmental information.
When information about pollution comes from fellow citizens, the target audiences may be more likely
to take action and change attitudes because of the creation of new social norms and the increased
potential for collective action. In turn, a more mobilized public may be better able to generate action
from public managers to reduce the pollution of urban waterways, given that addressing environmental
issues is both centrally mandated and important for political stability.
5.3 Control
In the control condition, citizen monitors will not be active and no information will be transmitted
to local governments from MEEPA or the research team. We will rely on measurements of a profes-
sional environmental monitoring company and surveys implemented by our research team from Nanjing
University to assess differences between these waterways and waterways assigned to treatment.
6 Randomization Strategy
Our partner MEEPA will organize citizen monitoring for 80 waterways in Jiangsu province. Our
treatments are bundled monitor plus dissemination treatments, since it does not make sense to monitor
without a plan to use or otherwise transmit the information in some way. Because we expect dissemi-
nation to the relevant government agency and dissemination to the public to produce additive or even
have a positive interaction effect, we will cross the treatment arms such that dissemination of each type
15will occur for 60 waterways, yielding the following design:
• Monitoring / Dissemination to Government & Public (DG & DP , n=40)
• Monitoring / Dissemination to Government (DG , n=20)
• Monitoring / Dissemination to Public (DP , n=20)
• Control (n=80)
6.1 Blocking
We block our experimental sample by ordering the waterways based on the pre-treatment index
value of water quality and completely assigning treatment within blocks of eight. We will use complete
assignment with equal probability of treatment for all units within each block.
7 Hypotheses
7.1 Primary Hypotheses
We have primary hypotheses about water quality and resident welfare:
Water Quality
CHINA-H1: Community monitoring will result in improved water quality.
CHINA-H2: Community monitoring will reduce littering and dumping near waterways.
Welfare
CHINA-H3: Community monitoring will increase property values within 500m of waterways.
(If H1 & H2 have null results, then we will not test this hypothesis)
CHINA-H4: Community monitoring will increase the non-market value of living near waterways.
(If H1 & H2 have null results, then we will not test this hypothesis)
7.2 Hypotheses on Intermediate Outcomes / Mediators
We have a number of hypotheses on intermediate outcomes that we expect to be on the causal
pathway to improved environmental quality and welfare of residents:
CHINA-I1: Community monitoring increases scrutiny of the management authority.
CHINA-I2: Community monitoring increases information to the management authority.
16CHINA-I3: Community monitoring increases actions by the management authority to manage pollution in
waterways.
CHINA-I4: Community monitoring increases information to users about prescriptive norms.
CHINA-I5: Community monitoring increases information to users about descriptive norms.
CHINA-I6: Community monitoring increases public participation in the management of the waterway.
CHINA-I7: Community monitoring increases demand for the remediation of pollution in waterways.
CHINA-MED1: Satisfaction with management of pollution in waterways depends on the change in water quality.
7.3 Hypotheses on Heterogeneous Treatment Effects
CHINA-C1: Citizen monitoring will improve water quality and reduce littering/dumping most for waterways
with the most severe pollution at baseline.
CHINA-C2: Citizen monitoring will improve water quality and reduce littering/dumping most for waterways
in areas with higher levels of economic development (e.g., average income, average real estate
prices).
8 Measurement
8.1 Overview of Data Collection Methods
We will employ undergraduate student-enumerators from Nanjing University to conduct two
rounds of surveys, at baseline and endline. We will recruit 50 local resident based on a random-walk
protocol around every waterway for the baseline survey. During the baseline survey, we will collected
phone numbers of those surveyed with a plan for recontact. The recontact rate was too low to use
this strategy for endline. The endline survey will be administered to a new sample of 50 residents near
each waterway, for a repeated cross-section design. We will also use a number of meteorological and
observational measures from secondary sources as control and outcome variables.
8.2 Primary Outcomes
8.2.1 Water Quality
M1 (Water Quality Index) We will contract a third-party environmental monitoring company
to evaluate the water quality using certified equipment and methods at the baseline and twice at
the endline of project for each of the waterways in the sample. We planned to take the endline
sample one year after the baseline samples, to limit seasonal effects on the measures. However, due
to contract delays, the measurements will actually be taken two years after baseline. The third-party
environmental monitoring company will have passed China Meteorology Accreditation. The water
quality indicators are mainly selected based on those used for Guidelines for Black and Odorous
17Urban Waterways by the Ministry of Housing and Urban-Rural Development of China. According the
Guidelines, the indicators we use to measure water quality are transparency, dissolved oxygen (DO),
Oxidation Reduction Potential (OPR), ammonia, chemical oxygen demand (COD), phosphorous, and
total nitrogen (TN).
Table 1: Water quality indicators with index weightings
Indicator Grade IV Standard Index Weight
Transparency 25* 0.15
Dissolved oxygen (DO) 3 0.2
Oxidation Reduction Potential 50* 0.2
(ORP)
Ammonia 1.5 0.1
Chemical Oxygen Demand (COD) 30 0.15
Phosphorous 0.3 0.1
Total Nitrogen (TN) 1.5 0.1
* Based on the Guidelines for Black and Odorous Urban Waterways
For the purpose of hypothesis testing, we will form the seven indicators of water quality into a
single index. Weights were chosen by the research team in proportion to rough importance of grading
waterways as part of the Guidelines for Black and Odorous Urban Waterways. In particular, we will
create a value for each indicator that corresponds to compliance with grade IV water quality standards
for that indicator, which is corresponds to the targets that EPB staff are attempting to achieve in
Jiangsu province at the time that our project was designed. Each value that is >1 will indicate
pollution that exceeds the standard or a measure of a favorable indicator that is less than the standard
andIn the case of ORP, which can take on both positive and negative values, both the measurement
and the scale will be adjusted by +400 mV to ensure the consistency of the resulting index.
We will then take a weighted average of Pi such that the final index value I is:
∑
i
w i Pi (3)
i=1
The environmental monitoring company will take at least one sample per waterway. The water
samples will be collected at the bridges which are identified in our baseline survey. All of waterways
will be monitored in the same period (less than 3 days). The detail requirement of storage and delivery
of water samples can be found in Table 5. The standard testing method will be applied to test the
water quality indicators (see Table 6).
According the to Shanghai water quality index measurement, which uses the same formula, we can
roughly classify the resulting index values to have the following meanings:
• I ≤ 0.8 (Qualified water source): The functional standards are water quality are met, even if
individual items exceed the relevant standards.
• 0.8 < I ≤ 1 (Basically qualified water source): Water function has not been severely damaged,
even though a few items are exceeded.
• 1 < I ≤ 2 (Polluted water source): Contaminated water where many of the individual items do
not meet the standards, requiring remediation.
• I > 2 (Heavily polluted water source): Severely contaminated water that is unlikely to meet
functional standards for use.
8.2.2 Waste Accumulation in Waterways
M2 (Waterway Waste Accumulation) We will walk a transect of each waterway and identify in
general the total amount of trash floating in the waterway according to a 5-point scale, with an example
photograph provided for each rating within the scale. We will ask the environmental monitoring
company to record photographs of the waterway for coding at both at baseline and both endline water
quality measurements, with student enumerators recording the same value at baseline and the survey
endline. We will use the endline photographs from student enumerators as the primary measure, as
they will be specifically trained in taking a photograph with a good perspective. Coding will be done
by enumerators blinded to treatment assignment. This measure will be used to assess the amount of
littering and dumping from the neighborhoods that surround the waterways in our sample.
8.2.3 Welfare
M3 (Property Values) We will test whether there are differential changes in housing prices between
treatment and control within a 500m buffer of each river. We will test for an effect and continue to
monitoring this outcome over several years, conditional on estimating a positive treatment effect of
19Figure 2: Measurement of the total amount of trash floating in the waterway according to a 5-point
scale.
monitoring on water quality. In particular, we will scrap a real-estate website for the price per square
meter of every residence that has been sold in the buffer in the three-month immediately prior to
treatment (August-October 2017) and again in the same three-month period in future years (the first
test for August-October 2019). The price per square meter indicates the average value of housing near
each waterway in the sample.
We will collect housing prices from the online website (FANG.COM), which is China s largest real
estate platform having the most coverage and presenting information on both new and resale housing.
First, we use data capture tools to collect all the house information in a certain city, including the name,
address, price, area, orientation, release and update times, and surrounding facilities (e.g. hospital,
school, shopping mall, bus stop, subway station, etc.). Second, we convert the address to a geographic
coordinate for each unit. Third, we calculate the straight-line distance from each river to each house
using the information of latitude and longitude. Finally, we keep the houses whose distances to the
river are less than 500m. Note that FANG.com data is only available for the five largest cities in our
sample, so waterways in these cities will comprise our effective sample for this analysis.
8.3 Mediators & Intermediate Outcomes
8.3.1 Citizen Actions
Self-reported actions. The survey instrument will measure self-reported citizen actions with respect
to pollution in waterways. The self-reported behaviors measured will include both private actions and
appeals to public authorities for action.
B14a. How often do you walk along the waterway closest to you?
B14b. How many times in a month do you typically walk along the river?
B15. If you see somebody throwing garbage or polluting the waterway, you will:
B16. How many of the following actions have you taken during the past month? (list experiment)
20B19a. Did you contact any public officials to express specific concern about the quality of the waterway
in your neighborhood during the past year?
B20. How often in the past month have you discussed the pollution of the waterway nearest to your
home with other residents of your neighborhood who are not members of your household?
B23b. If yes, do you join in the WeChat group?
8.3.2 Citizen Knowledge, Attitudes, and Perceptions
The survey instrument will include items on citizens willingness to pay for improving water
quality and their perceptions about severity of water quality problems. We will use these items to
assess whether exposure to citizen monitoring affects causes residents to change their perceptions of
descriptive and prescriptive norms, update their knowledge about the status of the waterway in their
area, and increase the level of importance that they ascribe to its quality. The items in this category
are (see Appendix 2 for responses values for these items):
B9. How has water pollution in the waterway closest to you changed in the last year?
B10. How polluted is the waterway closest to you?
B11. In our country, rivers are classified into six categories based on water quality. Which category do
you think the river closest to you falls into?
B12. Does the pollution of the waterway closest to your home have an impact on your life?
B13. How do you think waterway pollution affects the price of houses nearby?
B17. How much attention and effort does the city government make to manage pollution in the water-
way closest to your home?
B18. How satisfied are you with the management of pollution in the waterway closest to your home by
the municipal government?
B22. We have heard that the Government intends to set up a watershed management fund for this
waterway through voluntary contributions and then to carry out relevant projects under the
supervision of the committee of the community residents, such as laying pipelines, Interception,
do not pollute the city river. How many Yuan are you willing to support this month? If you
indicate willingness below, if this fund is established, we will contact you for your donation?
At endline, we will send an invitation to join the MEEPA WeChat group to all of the survey
subjects and we will use as a behavioral outcome the proportion who choose to join the group based
on the invitation (see survey instrument, item E24).
In terms of the tests for knowledge by citizens, we will take the outcome to be the absolute value
of the deviation between the perceived water quality and the actual water quality as measured at
endline, using the deviation at baseline as a control variable. In terms of the trend, we will take the
21deviation between the perceived trend and the actual trend (if any) as the outcome variable. We will
count ”much better” or ”much worse” as a change of two grades, while using ”a little better” and ”a
little worse” as a change in one grade in the measured direction. Our hypothesis is that the deviation
from the actual values will be lower in the waterways that are assigned to the monitoring treatment,
as compared to the waterways that are assigned to control.
8.3.3 Authority Knowledge and Actions
We will conduct an endline survey with all of the government officials who are eligible to receive
the quarterly reports created based on the data from citizen monitoring. This survey will focus on
perceptions of the management authority about the level of pollution in each waterway, the level of
public concern about water pollution for each waterway, and the perceived likelihood that the public
living near each waterway is seeking action from government to address pollution in the waterway. Our
draft items for this survey, which will be finalized around the time of the endline survey:
A1. How would you rate the quality of water in «Waterway Name»? [classification I-VI]
A2. How concerned about water pollution is the average individual living near «Waterway Name»?
A3. How many times have members of the public from «Waterway Name» expressed concern to
government offices in the last year?
A4. Is the public living near «Waterway Name» paying attention to what the management authority
is doing to address water quality?
A5. What will you do if there is complain or concern about water pollution by local resident?
A6. Are you aware of any monitoring by volunteers for «Waterway Name»?
A7. Are there any directives from upper-level government to address pollution in «Waterway Name»
based on citizen monitoring or complaints?
A8. What you have done during the previous year to address water quality in «Waterway Name»?
A9. How much funds have been invested for the remediation of water quality in «Waterway Name»?
A10. How have you obtained information about the quality of «Waterway Name» in the past year?
Our planned survey of municipal and county officials is the most difficult part of our data collection
strategy and the part that is least likely to succeed owing to non-response. While we will endeavor
to ask officials to answer each of these questions to each of the waterways in the sample that are part
of their jurisdiction, it is possible that we will not have enough responses for a meaningful analysis.
If this is the case, we will use any of the data that we receive to inform the design of more focused
and intensive interviews with a limited number of officials, which we will use as qualitative evidence of
whether monitoring affected perceptions, knowledge, or actions of officials. We will seek information
from official at each of the offices that are eligible to receive reports if they had been assigned to
22You can also read
