Reaiming RE-AIM: Using the Model to Plan, Implement, and Evaluate the Effects of Environmental Change Approaches to Enhancing Population Health
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
FRAMING HEALTH MATTERS
Reaiming RE-AIM: Using the Model to Plan, Implement, and
Evaluate the Effects of Environmental Change Approaches to
Enhancing Population Health
Diane K. King, PhD, Russell E. Glasgow, PhD, and Bonnie Leeman-Castillo, PhD
the issues and recommendations are also ap-
The RE-AIM (reach, effectiveness, adoption, implementation, maintenance)
plicable to social environment interventions,
framework, which provides a practical means of evaluating health interventions,
has primarily been used in studies focused on changing individual behaviors. and intended or unintended social conse-
Given the importance of the built environment in promoting health, using RE- quences of interventions are included within
AIM to evaluate environmental approaches is logical. We discussed the benefits the RE-AIM model. Our specific goals are to
and challenges of applying RE-AIM to evaluate built environment strategies and provide a rationale for using RE-AIM to plan
recommended modest adaptations to the model. We then applied the revised and evaluate built environment changes that
model to 2 prototypical built environment strategies aimed at promoting promote health behavior, discuss definitions
healthful eating and active living. We offered recommendations for using RE- and measures of the dimensions of RE-AIM
AIM to plan and implement strategies that maximize reach and sustainability, and propose adaptations to them, illustrate
and provided summary measures that public health professionals, communities,
applications of the dimensions through exam-
and researchers can use in evaluating built environment interventions. (Am J
ples of built environment changes, and estab-
Public Health. 2010;100:2076–2084. doi:10.2105/AJPH.2009.190959)
lish practical RE-AIM summary measures
for built environment interventions.
The RE-AIM (reach, effectiveness, adoption, to evaluate programmatic and policy9 interven- ROLE OF THE BUILT ENVIRONMENT
implementation, maintenance) framework1 was tions addressing a wide range of health condi- IN PUBLIC HEALTH
developed to enhance the impact of health pro- tions (e.g., diabetes, obesity, and hypertension)10–12
motion interventions by evaluating the dimen- and health behaviors (e.g., physical activity, di- The increased understanding among behav-
sions considered most relevant to real-world etary behaviors, and smoking).13–15 ioral scientists, public health practitioners, and
implementation, such as the capacity to reach Despite RE-AIM’s efficacy as a public health planning experts of the built environment’s role
underserved populations and to be adopted planning and evaluation framework, it has not in promoting healthy behavior and reducing
within diverse settings.2,3 Briefly, the reach been formally applied to interventions target- health risks (e.g., pollution, inactivity, acci-
dimension of the framework refers to the per- ing the social or built (i.e., manmade features of dents)19 offers an opportunity to use a transdisci-
centage and characteristics of individuals receiv- the environment that provide the settings for plinary approach to addressing major risk factors
ing the intervention; effectiveness refers to the human activity)16,17 environment. As public associated with many of the leading causes of
impact of the intervention, including anticipated health continues to expand its focus beyond death (e.g., cancer, respiratory and heart diseases,
as well as unanticipated outcomes; adoption surveillance and epidemiology to address root unintentional injuries).20 Furthermore, because
concerns the percentage and representativeness factors affecting community health, we need emphasizing the physical location where individ-
of settings that adopt the intervention; imple- models that help frame the planning and imple- uals encounter an intervention will influence
mentation refers to the consistency and cost of mentation of multilevel health interventions and which populations are reached, how often they
delivering the intervention; and maintenance re- guide comprehensive evaluations of the pro- are reached, and whether the environmental
fers to long-term sustainability at both the setting cesses, effects, and outcomes18 associated with change has a positive, neutral, or negative effect
and individual levels (see http://www.re-aim.org such interventions. Holistic evaluations of (e.g., does transit-oriented development21 in-
for more information about the framework.).1,4,5 changes in public spaces (e.g., changes in trans- crease a community’s access to desirable retail
The RE-AIM model was intended to guide portation and land use) are critical given the services or lead to gentrification and displace-
planning and evaluation of evidence-based in- complexity of such changes and their strong ment of low-income residents?22), strategic selec-
terventions6,7 that address the different levels potential to positively affect social capital and tion of the location for the built environment
of the socioecological model, such as those that cohesion or to exacerbate social and health change during the planning stage is critical.
target individual health behavior change by inequities. As a result of public concern about obesity
increasing intrapersonal, organizational, and Here we focus on applying RE-AIM to built and health disparities and the inequitable
community resource support.8 It has been used environment interventions, although many of burden of chronic diseases, especially in poor
2076 | Framing Health Matters | Peer Reviewed | King et al. American Journal of Public Health | November 2010, Vol 100, No. 11FRAMING HEALTH MATTERS
neighborhoods,23,24 there is social pressure on 3. What agencies are responsible for imple- Continuing with the active transportation
policymakers to address local inequities related menting the change? example, assessing effectiveness may require
to accessing healthful food and safe physical 4. What agencies are responsible for main- measuring whether there are different effects
activity venues.19,25–28 Recommendations for taining the change? across different subgroups9 (e.g., did the in-
evidence-based environmental changes priori- 5. What funding needs to be secured to im- stallation of bike lanes, sidewalks, and destina-
tized by the Centers for Disease Control and plement and maintain the change? tions increase active transportation or reduce the
Prevention in a recent report addressing obe- number of car trips among those who will most
sity29 included improved geographic availability benefit, and were there any unintended nega-
of full-service grocery stores30,31 and farmers’ ADAPTING RE-AIM FOR BUILT tive outcomes,42 including social justice issues?).
markets32 to ensure communities’ access to ENVIRONMENT INTERVENTIONS Thus, measuring whether a built environment
healthful, affordable foods. The report also rec- change results in health behavior changes among
ommended increased residential access to Applying RE-AIM to evaluate built environ- the members of the target population may re-
nearby (i.e., within a half-mile) public outdoor ment changes is not straightforward for several quire data collection methods that have origins in
recreational facilities33 and improved infrastruc- reasons. Each RE-AIM dimension, although urban planning and marketing research (as
ture (e.g., bike lanes, sidewalks) for active trans- conceptually the same as originally defined by opposed to health research methods, wherein
port.34–37 Glasgow et al.,1 requires assessment indicators known participants volunteer for programs).43
Successful implementation of such projects different from those used for evaluating pro- Some methods that have been used with
could be aided by the use of a planning and grams or treatments (Table 1). success in such instances include systematic
evaluation framework that explicitly requires For example, reach (absolute number, per- observational approaches such as behavior
identification of the target population, as well as centage, and representativeness of those af- mapping, in which the number and character-
appropriate settings, institutions, and partners, fected by the environmental change) is chal- istics of people using the space and the way
with the goal of increasing the probability that lenging to calculate when considering potential they use the space are sampled and recorded at
the project will maximize access and health and actual users of public space. To paraphrase various times44; telephone or door-to-door sur-
outcomes in a sustainable way. The RE-AIM a line from the movie Field of Dreams (Univer- veys in which household data are gathered45;
framework fits well given its attention to the sal Pictures, 1989), ‘‘If you build it, will they and street-intercept survey techniques to collect
representativeness of both participants and come?’’ is the reach question relevant for built data from potential users.46 Street-intercept sur-
settings or, in the present case, the interven- environment interventions. For example, if veys have been shown to be more successful
tion’s geographic location and the agents in- a neighborhood makes environmental im- than are telephone interviews in capturing a rep-
volved.1 Application of the model requires provements such as sidewalk and bike lane resentative sample of the target population
knowledge of or collection of data on the target additions and traffic calming initiatives (e.g., stop within specific geographic boundaries, especially
population and the potential settings and orga- signs, curb extenders) to increase active (i.e., in the case of low-income and culturally diverse
nizations (e.g., clinics, worksites, schools) that can pedestrian and bike) transportation, who is populations, among whom face-to-face methods
implement the intervention. Defining and speci- being reached? Identifying the target population may promote trust.
fying target populations and institutional that could potentially use the sidewalks and bike Adoption from the RE-AIM perspective has
‘‘adopters’’ is less clear, however, in the planning lanes—in this case, residents of the neighbor- traditionally been defined with respect to the
of built environment interventions, as described hood where improvements were made—and settings (e.g., worksites, clinics, schools) in
subsequently. then capturing who actually uses them requires which programmatic or policy changes take
Jilcott et al.9 offered recommendations for collecting data on the target population and then place. Similar to reach at the individual level,
applying RE-AIM to health policies, and their conducting observational or survey research adoption can be used to evaluate the charac-
initial set of background questions provided before and after installation. teristics of institutions or organizations that
a useful starting point for considering the com- In instances in which geographic boundaries adopt or decline the intervention and whether
plexities of built environment interventions for the designated target population are not those that adopt it are representative of all
intended to change behavior and, by so clearly defined, researchers will often use buffer eligible or invited institutions or organizations.
doing, improve health. These questions zones, or circular areas, around the specific Identification of potential adopters is less
(reworded to fit built environment issues) are geographic location approximating the catch- concrete in the case of changes in the built
as follows: ment area for expected users.41 The size of the environment, in which key adopters may
buffer zone may vary according to the ubiquity change over the course of the project.
1. Whose health behaviors and health are to be of the destination (e.g., coffee shop vs specialty The process of changing public spaces to
improved? food store), its importance to a community’s daily promote health behaviors involves a geo-
2. What stakeholders need to be included in life, and the location of the intended users (target graphic component (i.e., where the project is
the planning process, and what agencies are population). Thus, a coffee shop’s buffer zone built influences the populations that are
responsible for approving the environmen- may be a few blocks, and a specialty food store’s reached) as well as a multiagency component
tal change? buffer zone may be the entire city. (i.e., specific agencies have the authority to
November 2010, Vol 100, No. 11 | American Journal of Public Health King et al. | Peer Reviewed | Framing Health Matters | 2077FRAMING HEALTH MATTERS
TABLE 1—Definitions, Challenges, and Metrics of Applying RE-AIM to Built Environment Projects
RE-AIM Dimension Definition Questions and Challenges Built Environment–Specific Metricsa
Reach No. of people and percentage of the target If space is redesigned to improve accessibility Estimate number of people reached based on the population
population affected and the extent to which to the community, how can it be calculated living or working within a specific distance of the change,
the individuals reached are representative whether people who live and work nearby visit? observe and describe visitors at varied times and days
and include those most at risk. of the week, conduct intercept surveys to determine
whether people visiting are from the surrounding
neighborhoods, use setting-level proxies such as sales
receipt volumes or daily usage, and track changes in reach
over time.
Effectiveness A measure of effects on health behaviors, Users of the space may demonstrate both Observe and map population behaviors (both positive and
including positive, negative, and desirable and undesirable behaviors, so negative) occurring before and after the environmental
unanticipated consequences. does the change produce equal effects change, document how robust or consistent the outcomes
across subgroups? are across key subgroups, assess any unanticipated
consequences (including both positive and negative
behaviors), enlist adopters/agencies to help describe and
quantify behaviors (e.g., food purchases, park attendance)
before and after the change, identify public data sources
(e.g., crime or accident data) that can be used to quantify
changes.
Adoption (inclusion No. and percentage of settings participating, Because settings do not ‘‘adopt’’ built Assess the representativeness of those making
and approval) and the extent to which the settings environment changes, who are the adopters decisions with regard to selection of the setting and
selected are representative of settings (e.g., target population, business owners, design of the change; assess the inclusion of those
that the target population will city council)? needed to approve the project (city council,
use or visit. neighborhood association), implement the change
(public works), and maintain the space (parks and
recreation, police); planning stage—calculate the
percentage of key stakeholders involved; implementation
stage (qualitative)—evaluate whether the agency or group
approving the change is viewed positively, and if it is able
to maintain the change?
Implementation Level of adherence to implementation When the environmental project is completed, Are standards or guidelines for implementing or
(installation) principles or guidelines, the extent to does it meet established design principles or installing the built environment change followed or
which all versus selected elements are plans for attracting visitors from the target only partially implemented? Do barriers or
implemented, and the cost. population? Are the ongoing costs deterrents to use remain (address via intercept
sustainable? surveys and observations)? Where and when is the
cost of change incurred? Who pays? Document
other changes needed to support the project (e.g.,
law enforcement or traffic engineering).
Maintenance Individual level—individuals continue to What agencies or groups are in a position to Individual level—the long-term impact on health behaviors as
(sustainability) exhibit the desired health behavior changes monitor individual behavior and setting novelty erodes (6 months or more after installation).
Setting level—change is maintained and maintenance over time? Is there a policy or Setting level—the approving/enforcing agency
deterioration or development of new barriers program in place that will support ongoing continues to provide upkeep and necessary support
to use is prevented or mitigated. improvements and maintenance to sustain use (e.g., lighting, police patrol), and budget and staff
and address evolving issues not initially are allocated each year to ensure that space is
anticipated? Are there adequate resources maintained.
and plans for covering ongoing
maintenance costs?
Note. Resources for evaluating built environment interventions include Active Living Research,38 the King County Food and Fitness Initiative,39 and the Project for Public Spaces.40
a
Can also be used to assess change over time in each dimension.
2078 | Framing Health Matters | Peer Reviewed | King et al. American Journal of Public Health | November 2010, Vol 100, No. 11FRAMING HEALTH MATTERS approve the change being made and the know- maintenance often requires the involvement of establishing ways to frequently report progress how and resources to implement and maintain community members and public entities to and celebrate the achievement of milestones are it over time). Policymakers, planners, traffic preserve the quality of the space and to prevent important for projects that may require months engineers, law enforcement personnel, resi- other changes that create obstacles and di- or even years to complete.58 dents, and other stakeholders should be in- minish its use by the target population. To demonstrate how the revised RE-AIM volved in site selection as well as the design and Economic pressures on communities, framework can be applied to built environment evaluation of the project.47 Diffusion of the changes in neighborhood demographics, crime, interventions, we described 2 exemplars based intervention to other settings may be less im- and upkeep are factors that influence contin- on composites of actual community strategies portant than ensuring widespread community ued use of public spaces. Thus, the RE-AIM employed in Colorado during the past 3 years access and attending to unforeseen conse- framework needs to be modified to include (see http://www.livewellcolorado.org). These quences for adjacent property owners (e.g., diverse indicators such as factors that influence exemplars are also summarized in Table 2. The a park that increases the number of cars parked construction and maintenance costs.56 Engag- example strategies have been endorsed by the on adjacent streets). ing citizen groups, law enforcement officials, and Centers for Disease Control and Prevention The implementation dimension of RE-AIM local government representatives at the initiation as ways to combat obesity29 and are repre- has traditionally been used to examine the of the project; providing a mechanism for col- sentative of built environment strategies now consistency with which an intervention is de- lecting systematic cost (e.g., annual upkeep) and being implemented by communities across the livered and the cost of such delivery. Although usage data; and creating long-term plans to country. The first strategy, ‘‘farmers’ market,’’ the evidence supporting built environment monitor environmental or social changes that addresses barriers related to fruit and vegetable changes is increasing,48–51 there has been a lack may threaten continued appropriate use of the access and consumption. The second strategy, of research on the ways in which study findings space will help ensure its maintenance. ‘‘complete streets,’’ encourages active transport. are translated and adapted to ensure that such changes produce desired improvements in health USING RE-AIM TO DESIGN AND PLAN Farmers’ Market behaviors. In addition, the agents involved in SUSTAINABLE ENVIRONMENTAL A coalition was formed to address obesity implementing changes may vary according to the CHANGES issues in a low-income community. Plans were stage of implementation, and not all of these drafted for an evening farmers’ market that agents may be well versed in best practices One advantage of built environment inter- would be situated in a centrally located church related to health promotion. ventions is that they can influence the behavior parking lot to address the lack of a grocery store The complexity of implementing wide-scale of large and diverse segments of the popula- within the predominantly Latino neighborhood. changes in public spaces was underscored tion. In addition, once built, such projects are The coalition defined the denominator for when the Americans with Disabilities Act (104 likely to be sustained, although maintenance calculating reach as the estimated number of Stat 327) was implemented in 1990.52 Cities will be required to retain their intended use. households within 1 mile (1.6 km) of the pro- needed to evaluate and communicate compli- Because construction costs for built environ- posed market site, given that households beyond ance standards across agencies and organizations ment changes can be high, careful planning that 1 mile tended to be composed of non-Latino with differing goals and priorities (e.g., govern- includes the intended users as well as those Caucasians, a group that was not the primary ment, planners, engineers, transportation, law who will need to approve, construct, and focus of the market. Plans to track customers enforcement, and people with disabilities). Pub- maintain the environmental change is essential. included observing the number of visitors to the lished best practices for changing public spaces, Each dimension of the RE-AIM framework market (i.e., to estimate the numerator for reach) including design principles and research-based can be used as a blueprint for planning (Table and using vendor sales information to determine evaluation guidelines, are available23,44,53–55; 2). We recommend planning for evaluations of the volume of fruit and vegetable purchases. however, there is a need for assessment of the the intervention from the start,7 including To ensure that the farmers’ market would be degree of fidelity to these principles, how they identifying metrics readily available from public approved and would appeal to the target are translated in diverse situations, and the sub- sources (e.g., crime and accident statistics), and population, the following partners were in- sequent effects on health behaviors when they identifying means by which behaviors can be cluded in the planning process: the neighbor- are only partially followed. tracked routinely and efficiently (e.g., store and hood association, the police department, the Maintenance includes assessment at both the restaurant register receipts, electronic benefit parent–teacher organization, local family individual level (i.e., are desired health behav- transfer machines at farmers’ markets that allow farmers and ranchers, the church priest, and iors sustained?) and the setting level (i.e., do use of food stamps, and routine customer sur- a nearby Latino social organization. The adopting institutions integrate the intervention veys). Training community groups in the use of implementation was assessed in both quanti- into regular practices and provide staffing and qualitative methods, such as systematic observa- tative and qualitative terms. Quantitative data budgetary support?). In the case of built envi- tion and walkability audit tools44,57 may en- included number of vendors per week and ronment interventions, individual maintenance courage the involvement of the community in variety of fruits and vegetables offered. Qualita- implies continued use of the space by a high maintaining the environmental change (Table 1). tive data included information gathered from percentage of the target population. Setting Finally, identifying milestones up front and a focus group of community members formed to November 2010, Vol 100, No. 11 | American Journal of Public Health King et al. | Peer Reviewed | Framing Health Matters | 2079
FRAMING HEALTH MATTERS
TABLE 2—Application of RE-AIM to the 2 Example Built Environment Strategies
RE-AIM Dimension Planning Stage Farmers’ Market Complete Streets
Reach Identify target population whose health
or health behavior could benefit.
Numerator Postimplementation observation of no. of Postimplementation observation of no. of visitors
shoppers at various times and days and per day arriving at the retail district at various
assessment of their demographics (age, times and assessment of their demographics
gender, race). (age, gender, race).
Denominator All residents residing within 1 mi of the market. All residents within 3 mi of the surrounding retail
district.
Effectiveness Identify desired health or behavioral outcomes Average number and percentage of market Average number of visitors per day who walk, bike,
and estimate probability that target customers per day who purchase fruits and or take public transportation to commute to
population will engage in those behaviors if vegetables, changes over time with regard to the retail district; increases in observed foot
the environmental change is made (with volume of fruit and vegetable sales (as a proxy and bike traffic; increase in public
consideration of cultural norms, convenience, for direct measurement of consumer eating transportation volume to the revitalized retail
and alternatives). behaviors), increased perceived access to district; decreases in observed motor vehicle
fruits and vegetables among patrons of the traffic to destinations within the revitalized
market (market survey), and data collected via district; and decreased accidents involving
intercept surveys of residents living within 1 mi pedestrians or bicyclists and cars.
of the market.
Adoption Identify and include key stakeholders to ensure
(inclusion/approval) that the project is designed to fit the target
population and that all the organizations
needed to approve the project (city council,
neighborhood association), implement the
change (public works), and maintain the space
(parks and recreation, police) are involved.
Numerator Planning stage—residents/target population are Planning stage—agencies, organizations, and
included to allow an understanding of residents/target population are involved in
preferences and to address potential barriers. planning the project.
Approval stage—agencies and organizations approve Approval stage—agencies and organizations
use of the space for a farmers’ market. necessary for approving the project see it through.
Implementation stage—vendors accept Implementation stage—extent to which project is
invitation to sell their produce at the market. approved as planned and resources are
approved to support implementation of the change.
Denominator Planning stage—comprehensive list of agencies Planning stage—comprehensive list of agencies
and organizations are invited to participate in and organizations are invited to participate in
establishing the farmers’ market. the design of the revitalization project.
Approval stage—the goal is to ensure that the Approval stage—the goal is to ensure that the
correct agencies and stakeholders are correct agencies and stakeholders are involved.
involved.
Implementation Identify standards and guidelines, including Postimplementation—the extent to which setting Planning stage—setting for project addresses
universal design and smart growth principles; for farmers’ market addresses transportation, transportation, type of retail, pricing, and other
perform health impact assessment to minimize pricing, and other barriers to food access for barriers for both target population and retailers.
unintended adverse consequences; anticipate target population; the variety of fruits and Implementation stage—necessary supports and
barriers and delays that might affect costs and vegetables and local foods are consistently resources are consistently provided (e.g., clean-up,
timelines; and provide a forum for airing available; linguistic and culturally competent police patrols, lighting).
community concerns and unanticipated backlash. customer service in place; and that food is
handled safely.
Continued
2080 | Framing Health Matters | Peer Reviewed | King et al. American Journal of Public Health | November 2010, Vol 100, No. 11FRAMING HEALTH MATTERS
TABLE 2—Continued
Maintenance Identify and include in planning all agencies
or groups that may be in a position to
monitor usage of the space and its
maintenance over time.
Individual Postimplementation—customers continue to Postimplementation—customers continue to visit
purchase or increase demand for fruits central retail district via foot, bike, or public
and vegetables. transport.
Setting Postimplementation—farmers’ market vendors Postimplementation—retail establishments continue to
continue to offer fresh fruits and vegetables thrive and attract local customers, and annual
and increase or maintain profits. budget and staff resources continue to be allocated
to maintaining the environmental change.
help provide an understanding of food needs, personnel, and traffic engineering personnel), the criteria met. Scores can then be averaged
pricing, and the optimal location for the market. representatives of businesses (chamber of across all dimensions. In Table 3, the complete
Maintenance plans were not discussed, although commerce, grocery stores, and restaurants), streets and farmers’ market examples just de-
a potential future need to relocate the market and resident groups (bicycle organizations, scribed are used to assign a score to each RE-
was raised as a result of concerns about liability seniors groups, and neighborhood associa- AIM dimension, as well as a summary score
from the church and complaints about increased tions). The coalition charged with implement- that allows for comparison between these 2
traffic from some of the neighbors. ing the project assessed fidelity to smart growth very different projects.
principles by evaluating the city’s master plan We calculated reach by observing and
Complete Streets and recommending ways to adapt it to meet counting the number of visitors to either the
This strategy applied smart growth princi- land use guidelines. Maintenance plans in- farmers’ market or the revitalized retail district
ples54 related to land use planning and transit- cluded ongoing tracking of perceived barriers and dividing this value by the number of people
oriented development to revitalize a city’s central and business satisfaction and profitability; this residing in the predesignated geographic area.
retail district and encourage commerce in a his- information was collected through town hall We calculated effectiveness as the proportion of
toric low-income area. Plans called for the meetings hosted by the coalition and the city visitors engaged in the desired health behavior
surrounding street network to be retrofitted council. The ultimate goal was to add language (i.e., purchasing fruit and vegetables or actively
according to complete streets guidelines, which to the city’s master plan to ensure application commuting to the retail district).
promote roadway designs that increase safety of smart growth and complete streets principles Adoption was calculated as the percentage
and accessibility for users (e.g., bicyclists, pedes- to all future land use projects (Table 2). of invited agencies and individuals participat-
trians, transit users, and motorists) of all ages and ing in the planning and approval process (in-
abilities. Complete streets designs typically in- QUANTIFYING RE-AIM: THE cluding those involved in implementing and
clude sidewalks wide enough to accommodate BOTTOM LINE maintaining the change). We rated implemen-
wheelchair users, bike lanes, and traffic calming tation using an anchored scale based on the
elements (e.g., reduced speed limits). Because Given the diversity of environmental ap- extent to which implementation deviated from
public transportation improvements associated proaches and the potential cost and time preestablished criteria (e.g., for the farmers’
with the revitalized space served residents within commitment associated with projects that in- market, adherence to the planned number of
3 miles, the target population was defined by the volve changes to the built environment, one vendors and the diversity and cost of food and,
city as those living within a 3-mile buffer. A practical use of RE-AIM is to compare projects for the complete streets example, adherence
desired behavioral outcome was an increase in whose target populations and target behaviors to established design guidelines).
active transportation behaviors among individ- differ. Three straightforward ways of provid- Finally, we estimated maintenance using
uals commuting to the revitalized district. ing a ‘‘bottom-line’’ summary score are poten- a similarly constructed anchored scale based
Bicyclist, pedestrian, and transportation data tially appropriate. on the likelihood that the environmental
were assessed through periodic observations change (and resulting reach and effective-
and intercept surveys conducted within the First Approach ness) would be sustained (and measured sub-
district. Adopters included in the planning, First, scores on each RE-AIM dimension can sequently via periodic observations). The
approval, and design of the project were gov- be set as 0.0 to 1.0 (or 0% to 100%), reflect- summary scores for the 2 projects were close
ernment officials (city manager, public works ing an estimated proportion (or percentage) of (0.47 and 0.53) despite wide variation on the
November 2010, Vol 100, No. 11 | American Journal of Public Health King et al. | Peer Reviewed | Framing Health Matters | 2081FRAMING HEALTH MATTERS
TABLE 3—Scores on RE-AIM Dimensions for Farmers’ Market and Complete Streets Built Environment Strategies
RE-AIM Dimension Farmers’ Market Complete Streets
Reach
Numerator Observed average no. of daily shoppers (100) Observed average no. of daily visitors (2000)
Denominator Residents within a 1-mi buffer of the market (1000) Residents within a 3-mi buffer of the district (7000)
Scorea 0.10 (100/1000) 0.28 (2000/7000)
Effectiveness
Description Average no. of customers per day who purchase fruits and Average no. of visitors per day who walk, bike, or take public
vegetables (60) transportation to commute to the retail district (100)
Scorea 0.60 (60/100) 0.05 (100/2000)
Adoption (inclusion/approval)
Numerator No. of agencies and organizations accepting the invitation No. of agencies and organizations accepting the invitation and
and participating (9) participating (20)
Denominator Total no. of agencies and organizations invited to participate Total no. of agencies and organizations invited to participate in
in establishing the farmers’ market (10) establishing the revitalized district (25)
Scorea 0.90 (9/10) 0.80 (20/25)
Implementation
Description The community identified a location for the market on a side The project addressed public transportation, sidewalks, and bike
street that can be closed off to traffic and is adjacent to lanes between low-income neighborhoods and the redeveloped
retail and restaurants, which will increase visibility and be space; the community was unsuccessful in attracting a grocery
mutually beneficial to the market and the adjacent store to address a major need; and barriers to implementing
businesses traffic calming measures are being addressed
Scoreb 0.75 0.50
Maintenance (projected)
Description No plans have been discussed for sustaining the farmers’ Commitment to continuously improve the district by adding green
market as a permanent structure areas and expanding the pedestrian and biking infrastructure
has been written into the city’s 10-year budget and master plan
Scorec 0.0 1.0
RE-AIM summary score (average across dimensions) 0.47 0.53
a
Scores range from 0.0 to 1.0 (or 0% to 100%), reflecting an estimated proportion (or percentage) of the criteria met.
b
Score is a subjective rating ranging from 0.0 (no criteria met) to 1.0 (all criteria met), of how closely the actual implementation matched the planned criteria.
c
Score is a subjective rating ranging from 0.0 (unlikely) to 1.0 (very likely), of the likelihood that built environment changes (and resulting reach and effectiveness) will be sustained.
separate dimensions. This suggests that impor- participation of adopters with the authority to behaviors. One way to address such situations
tant information, such as whether a sustain- approve the project and the likelihood that it would be to parse out the costs most directly
ability plan has been discussed, may be ob- will be maintained). related to the targeted health behavior, such
scured if summary scores alone are used. as the costs of walking and biking infrastructure
Third Approach improvements.
Second Approach A third and related index recommended by In general, the efficiency index method may
A second approach is to form a composite Glasgow4 is the ‘‘efficiency index,’’ in which the be best suited to projects (e.g., community
score by multiplying the 0.0 to 1.0 reach score cost of the built intervention is divided by the gardens, trails, or playgrounds) in which the
by the 0.0 to 1.0 effectiveness score (R · E). The R· E metric. Including cost information may direct costs of implementation and mainte-
scores for the 2 examples (0.06 and 0.014) appeal to decision makers and investors tasked nance are closely related to the R · E score. The
mask large differences in effectiveness (0.60 with allocating scarce resources. However, esti- issue of who collects, analyzes, and summarizes
and 0.05; Table 3). Although the R · E score is mating true costs may not be practical for large, these data for decision makers is a complex
relatively simple to calculate (because it elimi- multifaceted infrastructure changes, particularly one whose detailed discussion is beyond the
nates the less straightforward adoption, imple- given that large capital investments may be scope of this article. We recommend that a
mentation, and maintenance ratings), it re- offset by civic and social benefits (e.g., neutral party, such as a state health represen-
moves those aspects of RE-AIM that are most increased commerce and jobs, traffic and crime tative or an independent evaluation firm, con-
likely to affect reach and sustainability (i.e., the safety), in addition to improved health duct these analyses.
2082 | Framing Health Matters | Peer Reviewed | King et al. American Journal of Public Health | November 2010, Vol 100, No. 11FRAMING HEALTH MATTERS
Presenting RE-AIM data in a way that means of many community organizations. CO 80237-8066 (e-mail: diane.king@kp.org). Reprints
can be ordered at http://www.ajph.org by clicking on the
resonates with the general public is another Recruiting individuals and measuring their
‘‘Reprints/Eprints’’ link.
complex issue. Use of graphic representations, health and health behaviors longitudinally is This article was accepted May 18, 2010.
such as charts that illustrate the relative often not an option. However, changes in
strength of each dimension, may better facili- behavior can be adequately captured by com- Contributors
tate communication and decision making than munity volunteers trained in using qualitative D. K. King designed and developed the content, exam-
ples, and discussion, and made the final edits. R. E.
use of numerical scores.59 techniques such as systematic observation and
Glasgow provided major conceptual input with respect to
behavior mapping.60 adapting and quantifying the RE-AIM dimensions. B.
DISCUSSION AND IMPLICATIONS In addition, if diverse partners are involved Leeman-Castillo developed the exemplars and provided
overall editorial input. All authors made substantial
in the design and implementation phases, data
contributions to the content of the article and reviewed
The 5 RE-AIM dimensions, with some mod- already collected for other purposes (e.g., sales the final version.
ification of definitions, seem to be applicable to receipts, crime and accident statistics) can also
built environment interventions and provide be used to quantify reach and effectiveness. Acknowledgments
added value given their usefulness in antic- Even if it is not possible to measure all aspects This work was supported in part by the Kaiser Perma-
nente Colorado Community Benefit Department and the
ipating impact, planning for sustainability, and of the RE-AIM framework for a given built
National Institute of Diabetes and Digestive and Kidney
addressing unexpected or adverse conse- environment intervention, consideration of all Diseases (grant 2 R01 DK035524-21).
quences. The greatest modification with respect dimensions in the planning stage, including We thank Jessica Osbourne from the Colorado Phys-
ical Activity and Nutrition Program for providing valu-
to both planning and evaluation was associated qualitative assessments of relevant metrics (e.g.,
able input on evaluating active community environments
with the adoption dimension. Because built the characteristics of who is, and who is not, and LiveWell Colorado for inspiring the exemplars.
environment interventions do not involve ac- participating and benefiting61), can enhance the
ceptance by a specified set of institutions or success of the intervention. Human Participant Protection
organizations such as schools or worksites, A follow-up question that emerges from this Because of the conceptual nature of this study, no
protocol approval was needed.
identifying the participation and characteristics application of RE-AIM is whether a particular
of adopters is less central than is identifying and RE-AIM dimension should be weighted more
References
including those with the authority to approve heavily than others or whether a summary score
1. Glasgow RE, Vogt TM, Boles SM. Evaluating the
the project and those involved in its implemen- can suffice. The answer to this question depends public health impact of health promotion interventions:
tation, enforcement, or maintenance. Although on the situation. As our examples showed, the RE-AIM framework. Am J Public Health. 1999;
89(9):1322–1327.
the specific adopters may change as the project comparing an average summary score across
moves from planning and design to implemen- RE-AIM dimensions may obscure important 2. Green LW, Glasgow RE. Evaluating the relevance,
generalization, and applicability of research: issues in
tation, anticipating and including all critical elements such as inclusion of key stakeholders or external validity and translation methodology. Eval
stakeholders and end users during the planning plans for maintaining the change. Also, a sum- Health Prof. 2006;29(1):126–153.
stage will reduce the likelihood of costly delays, mary score is not meaningful in and of itself 3. Program Evaluation: A Variety of Rigorous Methods
revisions, or cancellations. because it has no referents or norms. Comparing Can Help Identify Effective Interventions. Washington, DC:
US Government Accountability Office; 2009.
An advantage of using RE-AIM is that it ties the scores for each dimension across strategies
4. Glasgow RE. RE-AIMing research for application:
together key concepts that can be used in both may be the most useful method and may also be ways to improve evidence for family practice. J Am Board
planning and evaluating built environment easier to communicate visually to constituents. Fam Pract. 2006;19(1):11–19.
projects. The model can be applied to various With an increasing number of communities 5. Glasgow RE, Klesges LM, Dzewaltowski DA, Bull SS,
scenarios to compare and make decisions re- using socioecological approaches, including Estabrooks P. The future of health behavior change research:
what is needed to improve translation of research into health
garding how a proposed project’s location policies, programs, and environmental changes,
promotion practice? Ann Behav Med. 2004;27(1):3–12.
affects reach, which agencies and organizations to promote health behavior, the ability to apply
6. Glasgow RE, Green LW, Klesges LM, et al. External
need to be brought to the table, and the relative a single framework such as RE-AIM across validity: we need to do more. Ann Behav Med. 2006;
costs of different project scenarios. A disad- different types of interventions is advanta- 31(2):105–108.
vantage of using RE-AIM is its conceptual geous.5,9 We propose furthering the frame- 7. Klesges LM, Estabrooks PA, Glasgow RE, Dzewaltowski
nature; that is, the framework does not provide work’s usefulness by adapting and applying the 5 D. Beginning with the application in mind: designing and
planning health behavior change interventions to enhance
guidelines on what specific data to collect, how RE-AIM criteria to the planning of built envi- dissemination. Ann Behav Med. 2005;29(2):66–75.
to collect these data, or how to monitor ‘‘ex- ronment interventions with maximal reach and
8. McLeroy KR, Bibeau D, Steckler A, Glanz K. An
posure’’ to the project and its impact on effectiveness. j ecological perspective on health promotion programs.
behaviors over time. Thus, community groups Health Educ Q. 1988;15(4):351–377.
may find it challenging to address all 5 di- About the Authors 9. Jilcott S, Ammerman C, Sommers J, Glasgow RE.
Applying the RE-AIM framework to assess the public
mensions of RE-AIM in a practical manner. The authors are with the Institute for Health Research,
health impact of policy change. Ann Behav Med. 2007;
On a related note, measurement recom- Kaiser Permanente Colorado, Denver.
34(2):105–114.
Correspondence should be sent to Diane K. King, PhD,
mendations for built environment interven- Institute for Health Research, Legacy Highlands, Suite 300, 10. Dunton GF, Lagloire R, Robertson T. Using the RE-
tions may demand expertise that is beyond the Kaiser Permanente Colorado, PO Box 378066, Denver, AIM framework to evaluate the statewide dissemination
November 2010, Vol 100, No. 11 | American Journal of Public Health King et al. | Peer Reviewed | Framing Health Matters | 2083FRAMING HEALTH MATTERS
of a school-based physical activity and nutrition curricu- 27. Miles R. Neighborhood disorder, perceived safety, 45. Duncan SC, Strycker LA, Duncan TE, Chaumeton
lum: ‘‘Exercise Your Options’’. Am J Health Promot. and readiness to encourage use of local playgrounds. Am NR. Telephone recruitment of a random stratified youth
2009;23(4):229–232. J Prev Med. 2008;34(4):275–281. sample for a physical activity study. J Sport Exerc Psychol.
11. Farris RP, Will JC, Khavjou O, Finkelstein EA. 28. Casey AA, Elliott M, Glanz K, et al. Impact of the 2004;18(3):353–368.
Beyond effectiveness: evaluating the public health impact food environment and physical activity environment on 46. Miller KW, Wilder LB, Stillman FA, Becker DM. The
of the WISEWOMAN program. Am J Public Health. behaviors and weight status in rural U.S. communities. feasibility of a street-intercept survey method in an
2007;97(4):641–647. Prev Med. 2008;47(6):600–604. African-American community. Am J Public Health. 1997;
12. Glasgow RE, Nelson CC, Strycker LA, King DK. 87(4):655–658.
29. Khan LK, Sobush K, Keener D, et al. Recommended
Using RE-AIM metrics to evaluate diabetes self-manage- community strategies and measurements to prevent 47. Wright DS, Anderson LA, Brownson RC, et al.
ment support interventions. Am J Prev Med. 2006; obesity in the United States. MMWR Recomm Rep. 2009; Engaging partners to initiate evaluation efforts: tactics
30(1):67–73. 58(RR07):1–26. used and lessons learned from the Prevention Research
13. Glasgow RE, Klesges LM, Dzewaltowski DA, Estabrooks Centers Program. Prev Chronic Dis. 2008;5(1):A21.
30. Shaffer AG. The Persistence of L.A.’s Grocery Gap:
PA, Vogt TM. Evaluating the overall impact of health The Need for a New Food Policy and Approach to Market 48. Aytur SA, Rodriguez DA, Evenson KR, Catellier DJ,
promotion programs: using the RE-AIM framework to Development. Los Angeles, CA: Center for Food and Rosamond WD. Promoting active community environ-
form summary measures for decision making involving Justice, Urban and Environmental Policy Institute, Occi- ments through land use and transportation planning. Am
complex issues. Health Educ Res. 2006;21(5):688–694. dental College; 2002. J Health Promot. 2007;21(suppl 4):397–407.
14. Glasgow RE, Gaglio B, Estabrooks PA, et al. Long- 31. Larson NI, Story MT, Nelson MC. Neighborhood 49. Floyd MF, Crespo CJ, Sallis JF. Active living research
term results of a smoking reduction program. Med Care. environments: disparities in access to healthy foods in the in diverse and disadvantaged communities stimulating
2009;47(1):115–120. US. Am J Prev Med. 2009;36(1):74–81. dialogue and policy solutions. Am J Prev Med. 2008;
15. Glasgow RE, Nutting P, Toobert DJ, et al. Effects 34(4):271–274.
32. Kunkel ME, Luccia B, Moore AC. Evaluation of the
of a brief computer-assisted diabetes self-management South Carolina seniors farmers’ market nutrition educa- 50. Saelens BE, Sallis JF, Black JB, Chen D. Neighborhood-
intervention on dietary, biological, and quality-of-life tion program. J Am Diet Assoc. 2003;103(7):880–883. based differences in physical activity: an environment scale
outcomes. Chronic Illn. 2006;2(1):27–38. evaluation. Am J Public Health. 2003;93(9):1552–1558.
33. Kahn EB, Ramsey LT, Brownson RC, et al. The
16. Lopez-Zetina J, Lee H, Friis R. The link between effectiveness of interventions to increase physical activ- 51. Reynolds KD, Wolch J, Byrne J, et al. Trail charac-
obesity and the built environment. Evidence from an ity: a systematic review. Am J Prev Med. 2002;22(suppl teristics as correlates of urban trail use. Am J Health
ecological analysis of obesity and vehicle miles of travel 4):73–107. Promot. 2007;21(suppl 4):335–345.
in California. Health Place. 2006;12(4):656–664. 52. US Dept of Justice. The ADA and city governments:
34. Dill JC, Carr T. Bicycle commuting and facilities in
17. Brownson RC, Chang JJ, Eyler AA, et al. Measuring major US cities: if you build them, commuters will use common problems. Available at: http://www.ada.gov/
the environment for friendliness toward physical activity: them. Transp Res Rec. 2003;1829:116–123. comprob.htm. Accessed July 19, 2010.
a comparison of the reliability of 3 questionnaires. Am 53. Boehmer TK, Luke DA, Haire-Joshu DL, Bates HS,
J Public Health. 2004;94(3):473–483. 35. Saelens BE, Sallis JF, Frank LD. Environmental
correlates of walking and cycling: findings from the Brownson RC. Preventing childhood obesity through
18. Dannenberg AL, Bhatia R, Cole BL, et al. Growing transportation, urban design, and planning literatures. state policy: predictors of bill enactment. Am J Prev Med.
the field of health impact assessment in the United States: Ann Behav Med. 2003;25(2):80–91. 2008;34(4):333–340.
an agenda for research and practice. Am J Public 54. National Center for Appropriate Technology. Principles
Health. 2006;96(2):262–270. 36. Frank LD, Schmid TL, Sallis JF, Chapman J, Saelens
of smart growth. Available at: http://www.smartgrowth.
BE. Linking objectively measured physical activity with
19. Dannenberg AL, Jackson RJ, Frumkin H, et al. The org/about/principles/default.asp. Accessed July 19,
objectively measured urban form: findings from SMAR-
impact of community design and land-use choices on 2010.
TRAQ. Am J Prev Med. 2005;28(suppl 2):117–125.
public health: a scientific research agenda. Am J Public 55. Transportation for a New Era: Growing More Sus-
Health. 2003;93(9):1500–1508. 37. Staunton CE, Hubsmith D, Kallins W. Promoting safe
tainable Communities. Washington, DC: Urban Land In-
walking and biking to school: the Marin County success
20. Perdue WC, Stone LA, Gostin LO. The built envi- stitute; 2009.
story. Am J Public Health. 2003;93(9):1431–1434.
ronment and its relationship to the public’s health: the legal 56. Wang G, Macera CA, Scudder-Soucie B, et al. Cost
framework. Am J Public Health. 2003;93(9):1390–1394. 38. Active Living Research. Tools and measures. Avail-
analysis of the built environment: the case of bike and
able at: http://www.activelivingresearch.org/resource
21. Strategic Alliance. Enact strategy: transit-oriented de- pedestrian trials in Lincoln, Neb. Am J Public Health.
search/toolsandmeasures. Accessed July 19, 2010.
velopment. Available at: http://eatbettermovemore.org/ 2004;94(4):549–553.
SA/enact/neighborhood/car_dependence.php. Accessed 39. King County Food and Fitness Initiative. Compen-
57. Pikora TJ, Bull FC, Jamrozik K, Knuiman M, Giles-
July 19, 2010. dium of assessment and audit tools. Available at:
Corti B, Donovan RJ. Developing a reliable audit in-
http://king.wsu.edu/foodandfarms/documents/KCFFIe
22. Policy Link. Transit oriented development. Avail- strument to measure the physical environment for
valuationtools4-15-09.pdf. Accessed July 19, 2010.
able at: http://www.policylink.org/site/c.lkIXLbMNJrE/ physical activity. Am J Prev Med. 2002;23(3):187–194.
b.5137373/k.E65E/Transit_Oriented_Development. 40. Project for Public Spaces. Understanding park use.
58. Mitchell RE, Florin P, Stevenson JF. Supporting
htm. Accessed July 19, 2010. Available at: http://www.pps.org/parks. Accessed July
community-based prevention and health promotion ini-
19, 2010.
23. Active Living and Social Equity: Creating Healthy tiatives: developing effective technical assistance systems.
Communities for All Residents. A Guide for Local Govern- 41. Berke EM, Koepsell TD, Moudon AV, Hoskins RE, Health Educ Behav. 2002;29(5):620–639.
ments. Washington, DC: International City/County Man- Larson EB. Association of the built environment with 59. Thorpe KE, Zwarenstein M, Oxman AD, et al. A
agement Association; 2005. physical activity and obesity in older persons. Am J Public pragmatic-explanatory continuum indicator summary
Health. 2007;97(3):486–492. (PRECIS): a tool to help trial designers. CMAJ. 2009;
24. Bernard P, Charafeddine R, Frohlich KL, Daniel M,
Kestens Y, Potvin L. Health inequalities and place: 42. Gladstone D, Preau J. Gentrification in tourist cities: 180(10):E47–E57.
a theoretical conception of neighbourhood. Soc Sci Med. evidence from New Orleans before and after Hurricane 60. Kramer L, Schwartz P, Cheadle A, et al. Promoting
2007;65(9):1839–1852. Katrina. Hous Policy Debate. 2008;19:137–175. policy and environmental change using Photovoice in the
25. Carter SP, Carter SL, Dannenberg AL. Zoning out 43. Brownson RC, Hoehner CM, Day K, Forsyth A, Sallis Kaiser Permanente Community Health Initiative. Health
crime and improving community health in Sarasota, JF. Measuring the built environment for physical activity: Promot Pract. 2010;11(3):332–339.
Florida: ‘‘Crime Prevention Through Environmental De- state of the science. Am J Prev Med. 2009;36(suppl 4): 61. Glasgow RE, Fisher EB, Haire-Joshu D, Goldstein
sign.’’ Am J Public Health. 2003;93(9):1442–1445. S99–S123. MG. National Institutes of Health science agenda: a public
26. Leyden KM. Social capital and the built environ- 44. How to Turn a Place Around: A Handbook for health perspective. Am J Public Health. 2007;97(11):
ment: the importance of walkable neighborhoods. Am Creating Successful Public Spaces. New York, NY: Project 1936–1938.
J Public Health. 2003;93(9):1546–1551. for Public Spaces; 2000.
2084 | Framing Health Matters | Peer Reviewed | King et al. American Journal of Public Health | November 2010, Vol 100, No. 11You can also read
