Training Advisory Council Official Recommendation - Instill Continuous Quality Improvement in the PPB Training Division and the Bureau as a Whole
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Training Advisory
Council Official
Recommendation
Instill Continuous Quality
Improvement in the PPB
Training Division and the
Bureau as a Whole
pg. 0Table of Contents
Team (in alphabetical order) ......................................................................... 2
Acronyms Used in This Recommendation .................................................. 2
Executive Summary of Recommendations .................................................. 3
Recommendations in Detail .......................................................................... 4
Recommendation 1: Instill CQI as a core basis for the TD and eventually the
PPB as a whole. ................................................................................................... 4
Recommendation 2: Increase investment in data analysis and research. ..... 8
Recommendation 3: Begin collecting anonymous reports to assess the
effectiveness of ABLE. ...................................................................................... 11
Recommendation 4: Participate in planning for the appropriate use,
accessibility, and storage of recordings of PPB/citizen interactions taken by
members’ BWCs, stops data, and other reporting methods. ........................ 12
Recommendation 5: Enroll in the National Police Foundation’s Near Miss
Reporting program. ........................................................................................... 15
Recommendation 6: Invite community review and feedback for all new
training and appropriate access to archival records of previous training. .. 16
Recommendation 7: Institute contact surveys to assess procedural justice,
quality of service. .............................................................................................. 18
References and Sources ............................................................................. 19
Appendix: Possible Anonymous ABLE Moment Reporting Form ........... 26
pg. 1INSTILL CONTINUOUS QUALITY
IMPROVEMENT IN THE PPB TRAINING
DIVISION AND THE BUREAU AS A
WHOLE
Team (in alphabetical order)
• Nathan Castle
• Albyn Jones
• Jim Kahan
• Phil Levinson
• Mark Schorr
• Sylvia Zingeser
The CQI team wishes to thank those TD and TAC members who provided
constructive criticisms of earlier drafts of this recommendation.
Acronyms Used in This Recommendation
• ABLE: Active Bystandership for Law Enforcement
• AI: Artificial Intelligence
• BIPOC: Black, indigenous, and people of color
• BOEC: (Multnomah County) Bureau of Emergency Communications
• BWC: Body-Worn Camera
• COCL: Compliance Officer and Community Liaison (of the Department of
Justice/City of Portland Settlement Agreement; contracted to Rosenbaum
& Associates LLP, Chicago, Ill.)
• CQI: Continuous Quality Improvement
• LEO: Law Enforcement Officer
• NPF: National Police Foundation
• NTSB (United States) National Transportation Safety Board
• PPB: Portland Police Bureau
• QR: Quick Response code – a 2D barcode that can be used to share web
links
• SOP: Standard Operating Procedure
• TAC: Training Advisory Council of the PPB
• TD: Training Division of the PPB
pg. 2Executive Summary of Recommendations
The TAC makes the overarching recommendation that:
1. The Dean of Training should work with PPB leadership, TD staff, the TAC,
and other PPB Operational Councils and Community and Culturally
Specific Councils to instill CQI as a core basis of PPB’s training, with the
eventual goal of making CQI a core basis of the PPB as a whole.
To support this overarching recommendation, the TAC further recommends in
the areas of data collection, data analysis, and community engagement that:
2. The PPB should invest in comprehensive data collection and analysis
capabilities to exceed minimum operational and compliance needs and
advance the state of the art for public safety research and performance
analytics.
3. The TD should assess the effectiveness of the ABLE course by providing
all units with a simple form for anonymously recording “ABLE moments”1.
4. The TD should be included in planning for the appropriate use, analysis,
accessibility, and storage of recordings from BWCs, stops data, and other
sources to ensure that they are useful as tools to improve training.
5. The PPB should enroll in the NPF anonymous reporting system for “near
misses” when officers have been faced with the possibility (that fortunately
did not happen) of injury or death.
6. Invite community review and feedback for all new training and appropriate
access to archival records of previous training.
7. The PPB should institute an ongoing customer service feedback program,
including contact surveys that proactively seek feedback from service
users and a passive feedback survey that can be included on officer
business cards as a link and QR code.
1
An “ABLE moment” is any situation when an officer intervenes and successfully prevents
another officer from engaging in actions that would be illegal or out of policy; the early
intervention resulted in the other officer’s actions not requiring reporting.
pg. 3Recommendations in Detail
Recommendation 1: Instill CQI as a core basis for the TD and
eventually the PPB as a whole.
The TAC believes that this CQI orientation should be based upon achieving
excellence in performance, including the three pillars of the recently delivered
ABLE course (Georgetown Law School Innovative Policing Program, 2020)
completed by all sworn officers:
1. Prevent misconduct
2. Reduce mistakes
3. Promote member health and wellness
The TD should start by explicitly implementing CQI within its own processes. It
should then develop training for the Advanced Academy explaining what CQI is,
what its purpose is, and how it will positively affect officers’ jobs.
Background
The invocation of CQI as a core basis for PPB training reflects the mission of the
TAC to provide “ongoing advice to the Chief of Police and the bureau’s Training
Division in order to continuously improve training standards, practices, and
outcomes through the examination of training philosophy, content, delivery,
tactics, policy, equipment, and facilities” (Training Advisory Council Bylaws,
2020).
CQI began as a concept for production of goods. Its origin and growth are
attributed to the work of W. Edwards Deming, a statistician who developed his
ideas in post-World War II Japan and was considered responsible for the global
surge in popularity of Japanese products such as automobiles, cameras, and
electronic equipment (Deming, 1982; Mann, 1985; Wikipedia. no date).
The basis of the concept is that concern for quality should be part of the entire
production cycle from conceptualization through design, manufacture, marketing,
and customer experience. Moreover, at each stage, the concern should be part
of the task of all participants in the effort, as opposed to top-down direction. Such
an approach should lead to understanding where and how to improve each
aspect of the production process. While the origins of CQI are in production, it
soon came to be understood that the concept also applied to delivery of services.
Public safety is a service provided by law enforcement, fire, and other
professionals, and CQI applies to public safety in a straightforward manner. More
specifically, adopting a CQI orientation can enable the PPB to become more of a
learning organization, open to information from the communities it serves, its own
membership, and scientifically based evidence. This information can be used to
refine the PPB’s mission, policies, procedures, directives, leadership practices,
and training in order to fulfill its vision of being committed to protecting in an
pg. 4equitable manner all of the communities it serves in a manner that minimizes
harms.
The TD is a necessary, but not a sufficient unit within the PPB to establish a CQI
orientation. Early incorporation of CQI at the TD will facilitate the incorporation of
CQI expertise within all of the PPB. This is because training is the principal
method of improving quality once evidence has identified the specific areas that
need to improve.
For the TD itself, quality improvement encompasses not only what to train, but
how to train, as well as the appropriateness of the training criteria themselves. All
three pieces rely on an expanded use of high-quality real-time feedback to the
TD.
The very recent transfer of the PPB Policy Group, formerly within the Office of
the Inspector General, to the TD is a welcome step in that direction. As the TD
builds experience using and demonstrating a CQI orientation, members will
naturally gain exposure to CQI concepts. The TD can demonstrate commitment
to CQI by highlighting CQI in after-training surveys and encouraging trainers to
describe how CQI is improving their own work.
Challenges
While CQI is relatively easy to state, embedding it within an organizational
culture can be difficult. That said, organizations tasked with such diverse services
as health services delivery and transportation safety have established CQI
cultures. Fitting these successes into the context of law enforcement will require
an effort, but we believe it is well within the capacity of both the TD and the PPB.
Here, we identify several challenges that must be considered.
Creating Trust
The biggest challenge to instilling a CQI orientation as a basic part of PPB
operations and training is that it is a cultural change, and—as such—requires
trust. Officers must trust that participation in the CQI process will not be used to
their detriment. City Hall must trust that CQI processes are being used to
improve processes and training, and that individuals engaged in deliberate
misconduct (rather than process defects or mistakes) can still be held
accountable. As learning and positive outcomes from the CQI program
accumulate, they can be used as evidence that PPB is trustworthy, which in turn
should lead City Hall and people mistrustful of the police to improve their view of
the PPB. In this way, both sides can move to a restorative resolution.
As documented by the recent OIR Group report (Gennaco et al., 2022), the
Portland Independent Police Review (Independent Police Review, 2022), and the
Portland City Auditor (2022), the community and City Hall are able to cite
historical and some more recent instances of officer actions that led to avoidable
harms. Some of those outcomes were caused by failures in training or policy;
many of those incidents are unlikely to recur due to improvements the bureau
pg. 5has made. Taking the next steps towards implementing a holistic CQI program
will require a degree of trust where bureau members will acknowledge that there
were harms in which their behavior had a role and the communities and City Hall
will acknowledge that the bureau is taking steps in revising policy and training to
prevent recurrences of such harms.
The OIR Group report also documents the belief of PPB officers that they feel
attacked when they are doing their jobs. Thus, some officers (not all of them, but
a substantial proportion) expect hostile reactions from the community that will
make their job harder and less satisfying, while many in City Hall and various
communities (again, not all but also a substantial proportion) believe that the
police remain out of control.
We view a transparent CQI program as a way—perhaps the only way—to get out
of the circle of distrust among those charged with maintaining public safety and
those who demand genuine and equitable community policing. We recognize that
this is not an easy change and cannot happen overnight. Especially in the
beginning of such a change, each side has to take some risks in order to
generate reciprocal trust by the other side.
Minimizing Blame
Another challenge, related to lack of trust, is that many people believe that
misdeeds must always be punished—that is, not to punish is tantamount to
denying that a deed was harmful. This has to be replaced by a different
approach, focusing on learning how failures occurred and implementing
improvements to avoid future failures. Blame and the fear of blame lead to a lack
of perceived safety at work and create the conditions for mistakes to be papered
over or ignored, preventing learning and improvement. The past is immutable,
and the assignment of blame and subsequent punishment cannot undo harm.
Change must be approached from a position of improvement going forward
rather than assigning blame for the past. CQI thus considered resembles
restorative justice (Sered, 2019). Both acknowledge without looking for blame or
revenge that people have been subjected to possible or very real harm by the
actions of other people. And both attempt to go beyond the harm to restore trust
by finding ways to change beliefs and acknowledge previous harms, which can
lead to forgiveness. CQI processes accomplish this by using evidence to find
ways of improving police service.
Getting member buy-in
A third challenge is one of communicating just what CQI is and can accomplish.
As compared to a QA approach, CQI emphasizes the need for continuous
improvement. Implicit in a focus on continuous improvement is a recognition that
no person, organization, or process is perfect; there is always room for
improvement. This is highly empowering but depends on active buy-in and
participation of bureau members.
pg. 6For example, CQI relies on feedback/evidence, and that in turn requires officers
to be transparent and open if they see a situation where an alternative approach
could have led to a better outcome, even if there was no harm done. Learning
from such “near misses” requires that members proactively and voluntarily
identify such situations and share their experiences. If officers fear that such self-
reporting can lead to blame, sanctions, or negative publicity, they are unlikely to
do so. One way around this dilemma is to have anonymous self-reporting, as in
our Recommendations 3 and 5. But for that to work, the officers must trust that
things reported anonymously will remain anonymous. We believe, based upon
evidence cited in Recommendations 3 and 5, that when reports are aggregated
over units, anonymity can be actually and believably sustained. In a small unit,
everybody knows everybody, so anonymity within the unit may be close to
impossible. But if frequencies of types of incidents from aggregated units rather
than individual stories are used in reporting the opportunities for improvement,
global anonymity is preserved and local anonymity is not thereby lessened.
Furthermore, people who do not yet trust officers must understand that
anonymous reporting of ways of doing better is the very opposite of sweeping
misdeeds and mistakes under the rug. Instead, it is bringing more information
into the open so that the incidence of both near misses and actual harms can be
reduced. This communication is an essential component of CQI; we believe that
the Dean of the Training Division would be the right speaker for such
communications within and outside of the PPB.
Opportunities
CQI is not a substitute for accountability, but instead establishes a culture of
learning and improvement alongside accountability. In such a culture of learning
and improvement, information is collected solely to improve the bureau as a
whole rather than identify deficient people or units. This is an opportunity to shift
the tone and address the culture of fear/siege mentality that was identified in the
OIR group report (Gennaco, et al., 2022).
In other words, CQI is blame-free. This is illustrated, for example, in
investigations by the NTSB, which are aimed at preventing the next accident, not
figuring out who is to blame for the accident being studied. For this reason, NTSB
investigations are not admissible as evidence in any determination of culpability
for harm (Kahan, et al., 1997). In this important way, any analyses and
conclusions brought about as a result of CQI research must be entirely
independent of investigations within (e.g., the Internal Affairs Department) or
outside (e.g., the Independent Police Review Board or its likely successor) of the
PPB.
The separation between CQI and accountability systems has two positive
consequences. First, the blame-free aspect of CQI research makes it safe for
members of the organization to come forward with useful observations. Second,
pg. 7the results of CQI research are manifested in the form of recommendations for
change in training, directives, or policies that lead to improved performance.
We provide here three concrete examples of issues that the PPB has faced in
recent years where a CQI orientation would be of benefit:
• Equity in policing: The PPB has a mandate to achieve equity in policing.
The dominant focus of critics of the PPB is a claim that bureau members
behave in inequitable manners (called “overpolicing” in the critics’ jargon).
These critics advocate punishing PPB members who engage in such
practices as a deterrent. Such a punitive approach can be lethal to efforts
to learn and improve. Instead, we believe that it is preferable to use CQI
research to engage with communities and to develop training and policies
not only to avoid “overpolicing” but also to be more responsive to the
public safety needs of those communities and thereby avoid
“underpolicing” them. This would result in more mutual trust between
these communities and the PPB (Rahr and Rice, 2015; Ede, et al., 2002).
• De-escalation: PPB policy recommends de-escalation as a strategy
whenever possible, so as to minimize violence to both members and the
public. The current orientation is to identify instances where de-escalation
should have been but was not used and punish the officers involved.
Instead, CQI research can look at the circumstances and techniques that
lead to successful de-escalation, and train officers and adjust policy to
reduce the risk of escalation (Bennell, Blaskovits, Jenkins, et al., 2021;
Graham, 2021; University of Wisconsin Law School and Johnson
Foundation at Wingspread, 2017).
• Community policing: PPB policy commits the organization to community
policing yet leaves the definitions of what that constitutes as well as how
to measure improvements and train members to ameliorate quality
deficiencies somewhat vague. CQI research can lead to more concrete
definitions of what community policing means and how best to achieve it
given limited resources. (Gennaco, et al., 2022; Prenzler, et al., 2016).
Recommendation 2: Increase investment in data analysis and
research.
The PPB should make a strategic plan to substantially increase data analysis
and research capabilities, with a goal of achieving capacity for proactive research
that improves the bureau’s efficiency and customer experience. Much of that
improvement will occur as a result of training that is keyed to ways that the
bureau can change practices in accordance with improved policies and
directives.
Implementing this recommendation will help the bureau get the most value from
its investment in CQI and is essential for Recommendations 4 and 7.
pg. 8Background
CQI is founded in learning from data to make process improvements. The
pioneer in developing causal analysis methods aimed at improving quality and
reducing harmful incidents is James Reason (Aerossurance, 2014; Reason,
2013), whose work was highly influential in transportation accident reduction and
generalized to other realms of harm reduction. Landmark efforts expanding
Reason’s approach to public safety through police accountability include the work
of Samuel Walker (2005) and Jeffrey Noble and Geoffrey Alpert (2009).
With officer resources under strain and the ongoing recruiting challenges,
the PPB should increase investment in its analytics and research functions
by purchasing software and hiring analytic specialists. An enhanced
performance analytics and research focus will enable proactive
identification of trends and opportunities to improve performance. This is an
example of an ounce of prevention being worth a pound of cure.
Challenges
Data quality
The first step in using data is collecting them in a reliable and bias-free way. This
is challenging because officers are already expected to complete many data
collection and report-writing tasks. The TD needs to be mindful about increasing
the data collection burden. Feedback systems driven by service users (as
described in Recommendation 7) and automation (Recommendation 4) present
an opportunity to increase data availability without increasing the burden on
officers.
Data analysis
Three of the biggest challenges to useful analysis are:
1. Asking the right question,
2. Understanding what data are necessary to answer the question, and
3. Communicating clearly in a manner appropriate to the audience.
It is important to resist the urge to focus on answering only those questions that
are easy to answer. As the famous statistician John Tukey was fond of saying: it
is far better to get an approximate answer to the right question than an exact
answer to the wrong question. Similarly, Albert Einstein is known for saying that,
“Not everything that can be counted counts, and not everything that counts can
be counted.” Innovative—sometimes non-quantitative—data can be very helpful.
PPB has made considerable strides in data collection and reporting, but
sometimes falls short when analyzing data, deriving conclusions, assessing
statistical validity, and communicating conclusions clearly. For example,
highlighting quarterly changes in some statistic such as weapons discharges or
tear gas deployments is easy, but what, if anything, does it allow us to learn
about training, tactics, or equity? The analysts should always be asking
pg. 9“Compared to what?” What is the appropriate comparison to facilitate
understanding of shortcomings in training or systemic biases? What is the
appropriate baseline for comparisons of rates or proportions?
Finally, communication of results, especially to non-technical audiences, is often
best done through visualization: plots, histograms, causal chains, etc. Tables of
numbers may underlie the analyses, but they often don’t lead to understanding.
Staffing and resources
PPB is facing a staffing crisis. While that primarily applies to sworn members, the
labor market in general is tight. It may be difficult to hire and onboard analysts in
sufficient numbers to manage the analytics and research programs we
recommend.
Investment in performance analytics has material costs. In the case of
performance research and data analysis, PPB should be prepared to support
analysts with workstations and access to cloud computing resources or pay for
commercial solutions.
Opportunities
The further expansion of innovative data collection and analysis capability will
greatly increase the amount of information available to PPB. Taking advantage of
this information to improve performance will require substantial investment but
has the potential to be rewarding.
Appropriate use of data to make decisions is challenging. The goal of analysis
should be to inform, and CQI processes can apply to analytic efforts as well as
performance, training, and leadership. There is an opportunity to build expertise
in producing data products that enable all bureau members to improve their work.
What distinguishes a data product from data is the supporting materials,
including documentation that describes how to interpret data, limitations, and any
other information that is needed to use it successfully.
As Portland is facing a historic increase in violent crime, there is an opportunity to
use natural experiments and carefully designed interventions to learn. This will
require a team with resources to spare to identify opportunities and give
guidance on intervention design so that useful insight can be generated. As a
contrast, in the recent case of interventions to reduce violence in Mt. Scott-Arleta,
several interventions were introduced simultaneously, including lighting
improvements, traffic calming, and increased patrols (KOIN, 2022). Because the
interventions were introduced simultaneously, it isn’t possible to attribute the
promising results to any one intervention, so the city cannot learn from the
experience to prioritize interventions in other neighborhoods.
pg. 10Recommendation 3: Begin collecting anonymous reports to
assess the effectiveness of ABLE.
PPB should create an anonymous reporting system to collect data on situations
in which peer intervention was necessary (“ABLE moments”). PPB should
analyze that data and produce a quarterly report.
Background
In late 2021, all sworn officers and some other PPB members received ABLE
training, using a script provided by the creators of the program (Georgetown Law
School Innovative Policing Program, 2020). ABLE teaches the requirement to
intervene when an officer reasonably believes that another officer is likely to
engage in behavior that is misconduct, a mistake, or endangers that other
officer’s own health and wellness. The training was viewed as highly successful
by PPB leadership, the Portland Police Association, and observers. It has been
recommended (with the encouragement of the Georgetown Law School team)
that the Oregon Department of Public Safety and Standards and Training to
introduce ABLE as part of its basic training package.
PPB members report informally that they have experienced ABLE moments
(either on the giving or receiving end) and appreciate the intervention, but there
is no formal way of assessing how often or how successful such interventions
are. We propose anonymous reporting to assess ABLE moments.
Challenges
Three challenges to the implementation of an ABLE monitoring program stand
out:
• Concerns about anonymity
• Buy-in
• Analysis
• Public communication and trust
To address concerns about anonymity and buy-in, TAC suggests a reporting
format that is short, easy to fill out, and anonymous (see Appendix for a
prototype). Regardless of how the PPB decides to implement reporting, care
should be taken to ensure that it is not implemented through PPB groups
associated with internal investigation or discipline.
Another challenge is turning the anonymous reports into useful feedback for the
TD. This is addressed in Recommendation 2.
The final challenge is communicating with the public. Anonymous ABLE reports
may be shared as part of public records. It is important that the public
understands that ABLE moments are successes – situations in which a member
intervened to prevent a bad outcome in a way that provided a learning
experience for everybody involved. The PPB should take care to communicate
pg. 11how it is using data derived from ABLE moments to improve service and reduce
the likelihood of bad outcomes by showing how analyzing near misses leads to
not only fewer near misses but fewer incidents through improved training for
situational awareness of risky interactions.
Opportunities
ABLE sets the expectation that officers must intervene when they see another
officer engaged in misconduct. There is an opportunity for the bureau to go a
step further, and gain insight from the occurrence of ABLE moments. Officers are
always expected to intervene when they see misconduct or a risk to officer
safety. While an “ABLE moment” is partially a success in that a member
intervened, TD should still take interest in learning from these near misses. ABLE
reporting will enable the PPB to look for trends in the occurrence of ABLE
Moments and adjust policy or training to make intervention necessary less often.
Recommendation 4: Participate in planning for the appropriate
use, accessibility, and storage of recordings of PPB/citizen
interactions taken by members’ BWCs, stops data, and other
reporting methods.
BWCs, stops data, and other innovative measures will be key sources of insight
(Corso, 2017); this information, no matter which PPB unit performs the analyses,
will provide the TD with an unprecedented window into how training is used in
practice.
Background
With the introduction of BWCs and new stops data collection efforts, there is an
opportunity to establish PPB as a leader in the use of data to drive customer
service satisfaction and public safety outcomes.
The introduction of BWC recordings and other innovative information-gathering
means will greatly increase the amount of information that PPB records.
Consequently, the PPB will need expertise in data analysis to turn a firehose of
data into a stream of insights.
BWC, stops, and other information should be analyzed outside of the direct chain
of command. This should help support the position that the information obtained
is to improve officer safety, provide more effective public safety, and improve
community relations. The PPB can report on areas of needed improvement and
the progress that is being made as part of its ongoing community engagement
efforts.
Proactive use of data-driven insights can help PPB improve its training and
service offerings while making the most of limited resources. BWC and stops
data enable the PPB to obtain real time information and analyze that information
pg. 12as it applies to questions it has about officer safety and the quality of service it
provides to the community. For example:
• How often are police interactions respectful?
• How often are officers attempting and succeeding at de-escalation?
• Are members exhibiting any disparities in how they interact with
BIPOC Portlanders?
• Is new training resulting in observable effects?
Challenges
All aspects of implementation of the BWC program are proving contentious,
and involvement of the TD is not likely to be an exception. Indeed,
Portland’s late deployment (compared to other agencies) of BWCs for law
enforcement use is a result of structural factors that may yet impede PPB’s
efforts to get the full value from its investment. At minimum, the TD and
PPB will need to be mindful of:
• Bargaining implications arising from the use of BWCs
• The risk of bias in any automated classification system
• The use of Artificial Intelligence, which will be necessary to handle
the volume of data produced by BWCs
BWCs, stops data, and the policies surrounding any use of information are
subject to collective bargaining. As part of the policy development process,
the PPB should be mindful of its interest in access to video streams and
recordings. Securing such access will require officer trust and buy-in as part
of the bargaining process. The authors of this report believe that the
introduction of a CQI orientation, which focuses on blame-free quality
improvement, will be a helpful step towards achieving this trust.
Existing commercial solutions allow command staff access to BWC and
stops information through real-time streams and automated reports. In
addition, access to this information by analysts not in the chain of command
to look at trends that suggest policy or training improvement should be
encouraged. Bureau members will need to trust that this information is
being used to improve policy and training, and not as an invasive, punitive
tool. There is a role for clear policy and training on use of such information
to engender that trust. Advanced analysis capabilities have the potential to
improve public trust and the experience of being a Portland police officer,
but it is incumbent on TD and PPB to create the conditions for trust that
ensure adequate safeguards for officers are in place.
Another risk to consider arises from the use of automated systems to derive
insight from BWC footage to be used in training. Specifically, AI systems,
when developed using biased data sets, tend to reflect those biases. For
example, a computer vision algorithm developed only on images of white
Portlanders may fail to perform appropriately when exposed to BIPOC
Portlanders. While reviewing and/or developing any system to derive insight
pg. 13from BWC or other footage, care must be taken to ensure that such
systems themselves are and remain bias-free; otherwise, they are poor
tools to use for training.
Opportunities
Equity is an important concern in the PPB’s efforts to improve equality. The
California Racial and Identity Profiling Act (RIPA), enacted in 2015 (California
Legislative Information, 2015), provides a good model that could be introduced in
the Oregon Legislature or incorporated into PPB reporting requirements by
Portland municipal law or PPB directive. Other ways of obtaining and analyzing
police behavior regarding equity may be found at the Center for Policing Equity
(2021). The important point here is that obtaining such evidence in a valid
manner is not something that requires invention—there are acceptable
approaches that are easily accessible.
With the pending introduction of BWC, PPB is poised to drastically increase the
amount of information it collects. While it isn’t feasible for human eyes to analyze
every moment of video, it is now possible for machines to do so. There is an
opportunity for PPB to become a leader in the automated analysis of video (not
just a single officer’s BWC, but all of the relevant recordings) to generate
insights. This could involve using commercial software or developing a solution
in-house using machine learning to classify video and generate features. While
this can and should be used to identify policy violations, it can also be used to
identify procedural justice issues and evidence of systemic bias in police service
to remedy them through improved policies and training.
For example, use of machine learning/artificial intelligence could enable a
reasonably valid automatic identification of subject race and the tone of voice and
language used by officers. This could be a powerful signal for identifying training
or coaching needs. Similarly, as officers are training in procedural justice
concepts, analyzed BWC footage and stops data could be used to evaluate the
effectiveness of training.
An analysis of BWC footage by Stanford University researchers (Voight, Camp,
Prabhakaran, et al., 2017) found that officers in Oakland, California consistently
used less respectful language when speaking with Black people. The differences
weren’t blatantly obvious. For these differences to surface required analysis of
thousands of interactions.
Unfortunately, police departments are not often the subjects of university
research. Consequently, fewer than 1% of police body camera video files are
ever reviewed. To use BWC and other information for more than forensic
analysis and officer protection from false claims, an effective analytics program
should be implemented that will enable the PPB to formulate specific questions
to drive both examination of officer behavior and community perceptions of that
behavior. The PPB should capitalize on its ongoing relationships with Portland
State University and other academic institutions to jointly conduct research to
pg. 14evaluate newly available information and evidence; existing educational research
human subject protection protocols will help reduce any inappropriate use of that
information (National Science Foundation, no date).
Recommendation 5: Enroll in the National Police Foundation’s
Near Miss Reporting program.
PPB should subscribe to the National Police Foundation anonymous reporting
system for near misses. This is a first step in what we believe should be a longer-
term establishment of anonymous, non-punitive self-reporting.
Background
Member incidents, including both near misses and minor errors, can help the
bureau understand precursors to potentially serious incidents. This will enable
the bureau to proactively formulate improvements in training and policy to
prevent critical incidents.
Anonymous self-reports have been well-established in transportation safety
(CHIRP Maritime, 2021) and hospital care (Agency for Healthcare Research and
Quality, 2019); it is time that similar systems be used for public safety.
A well developed and managed self-reporting system for near misses (close
calls) exists through the NPF. The LEO Near Miss Reporting System (National
Police Foundation, no date) is modeled after successful near miss reporting
systems used in several high-risk industries like the Aviation Safety Reporting
System. The U. S. Department of Justice, Office of Community Oriented Policing
Services funded the development of and supports the LEO Near Miss Reporting
System, the use of which is free to participating law enforcement organizations.
The system is currently used by law enforcement agencies across the country.
LEO Near Miss is only used for promoting peer learning and enhancing officer
safety. The lessons learned are shared by officers and can be independently
read or incorporated into department training.
The system operates independent of an agency’s chain of command or
disciplinary process and reports are totally anonymous. The process for
submitting a report uses a well-organized form which uses multiple choice and
drop-down questions and usually takes 10 to 15 minutes to complete. After a
two-stage review by current and former law enforcement officers all identifying
information is removed and the report is made available for review.
NPF partners with participating law enforcement agencies to assist them in
developing internal policies and procedures to facilitate reporting and use of near
miss data in training, policy, and equipment related decisions.
pg. 15Challenges
• Trust in the system
• Time and effort required to submit an optional, non-mandatory report
Opportunities
CQI requires proactive not reactive behavior. A critical component of CQI is
officer safety, ensuring that every time an officer responds to a call, they have the
training and information necessary to reduce the possibility of being seriously
injured or killed. The training and information available often rely on information
obtained from “close calls.” “Close calls” information can be gathered via self-
reporting or information obtained from BWCs. The information gathered must be
carefully analyzed to identify patterns of behavior that are problematic. The data
created from this analysis of field experience will generate training that will
change how officers respond to situations, thereby decreasing their risk and
potentially having a positive impact on the Portland Police Bureau’s public image.
A significant issue in gathering and using the information obtained is officer
anonymity, except in mandatory reporting situations. While the PPB does not
have the capacity to review near misses by itself, the LEO Near Miss program
provides that capacity at no cost to the bureau, including how the PPB compares
to the overall picture in all of the agencies the program covers.
Recommendation 6: Invite community review and feedback for
all new training and appropriate access to archival records of
previous training.
In order to have technical, cultural, and community perspectives as new or
revised training curricula are developed, the TAC and representatives of
Community and Culturally Specific Advisory Councils to the PPB should
observe/participate in dry runs/early sessions of such courses. Additionally, the
TD should ensure that information about training offerings is organized and
cataloged in such a way that the general public can easily come to understand
the ‘big picture’ of PPB training. This will improve transparency and enable
interested members of the public to provide more specific, constructive feedback.
Background
It is increasingly accepted that community engagement with public safety
agencies is necessary in contemporary American society. The PPB has made
major efforts in this regard by reaching out to community-created Community and
Culturally Specific Advisory Councils and expanding its use of operational
councils. Such engagement is consistent with a CQI approach, and we applaud
this outreach. Our recommendation here is more in the nature of an expansion of
current practice than a call for change.
pg. 16While the TD regularly makes video recordings of its own training activities and
must approve any training activity that it does not originate in but employs PPB
training facilities, not all outside trainings are as accessible as the internal ones.
This has led to highly publicized and equally highly misunderstood problematic
incidents (Dickinson, 2022). Improved review and record keeping would have
helped the city identify the extent of the problem, understand the root cause, and
make changes to prevent future incidents.
Challenges
The primary challenges to increasing community involvement in training
observation are:
Availability of community members
Observation of training is time-consuming, and many community members are so
busy with the activities of daily life that attendance at training events can be
difficult. The community members PPB draws on to view and give feedback on
training are most often volunteers. Training dry runs and observations tend to
happen during business hours, which can limit participation. TD should be
prepared to accept that there will not always be volunteers to observe every
training.
TD time
The time spent preparing, delivering, and evaluating training is substantial, and
having community members present will add to that burden.
Understandability of materials
It would take considerable effort to present the Annual Training Plan, catalog of
previous training plans, and documentation of video recordings of earlier training
in a format that is readily accessible to the general public (some of whom are not
native English speakers). A related challenge is the considerable effort required
to maintain any website that might be used for these purposes.
Public access vs. need to know
Written and other documentation of training events may contain information that
contains personal or strategic information that must be kept confidential.
Explaining this in an atmosphere where there is pressure for “total transparency”
can be difficult. The existence of material appropriately limited in distribution may
be publicly acknowledged, but that the information itself is limited in access must
be explained and justified.
Opportunities
This recommendation calls for substantial community participation in TD course
development as a way to understand and incorporate diverse perspectives on
training courses as the courses develop. These perspectives can lead to ways to
pg. 17improve the training in ways that make the PPB more responsive to community
expectations. In turn, the community participants get a better understanding of
how officers think about critical situations and how they are expected to behave
in such situations. This can promote more trust in the PPB, through transparency
and a better understanding by both the PPB and participating communities of
others’ orientations and expectations.
After the racial justice protests during the of summer 2020, there has been much
talk of the need for better training, especially regarding de-escalation, for police.
Unfortunately, too few Portlanders recognize that PPB has excellent training that
far exceeds state standards, and that due to successful policy interventions,
many high-profile incidents that occurred elsewhere are extremely unlikely to
occur in Portland. Sharing a catalog of training content in an easy-to-use format
could improve engagement and extract untapped public relations value from
existing investments in training.
Recommendation 7: Institute contact surveys to assess
procedural justice, quality of service.
It is common practice in the commercial sector to use customer feedback
surveys to gather information and improve service quality. Portland should
institute an ongoing evaluation program to measure the public’s perception of
police service and proactively identify concerns and high-priority areas for
improvement. Improvement in any system is difficult without feedback, and
policing is an area where ongoing public feedback is critical to success.
Background
The relationship between the police and the community is critical to policing
effectiveness. To maintain that relationship, the PPB needs to ensure that police
service is meeting customer expectations and needs. Simply expressing genuine
interest in customer feedback can go some way to improve community relations.
TD training already emphasizes procedural justice, but there are currently few
options for assessing the implementation of procedural justice in police work.
PPB should make it easy for community members to give feedback on their
interactions with police, whether those community members are subjects,
bystanders, or victims.
TAC is not the first community group to recommend surveys to measure
community perception of police work. Surveys are a time-tested tool for gathering
feedback. Over time, survey results will help the TD identify trends and tailor
trainings to address unmet needs that are undetectable through use of force
reporting.
pg. 18Challenges
The key challenges to implementing contact surveys are the cost/effort to
implement the survey/analysis program and securing officer buy-in. It is critical
that any surveys be carefully constructed to avoid as much as possible both
positivity and negativity biases in response frequencies (Brogle, 2018).
Large scale survey programs can be difficult to design and implement well. This
challenge would ideally be addressed as part of Recommendation 2.
Alternatively, PPB could seek partnerships with local academic institutions, at
least some of which are already engaged in public safety research.
The other challenge is securing PPB member buy-in. A feedback program is
necessarily going to introduce the possibility of negative feedback. TD can
mitigate this through CQI training that emphasizes a blame-free, continuous
improvement culture.
While a feedback survey link on a business card is straightforward for PPB to
implement, 911 contact surveys may require close collaboration with BOEC.
Opportunities
Customer feedback is key to prioritizing resources, evaluating the impact of
training interventions, and improving quality. There is an opportunity to learn from
comparable cities and tailor successful efforts such as the ongoing
feedback/evaluation program in Seattle to Portland’s needs.
References and Sources
PPB Personnel Interviewed
Capt. David Abrahamson, (then) captain, Training Division
Lauren Brown, Manager, Strategic Services Division
Deputy Chief Michael Frome
Liesbeth Gerritsen, Ph.D., Training Division
Capt. Chris Gjovik, Training Division
Officer Natasha Haunsperger, Office of Community Engagement
Lt. Brian Hughes, Chief’s Office
Lt. Jason Jones, Training Division
Aubrey Perry, Strategic Services Division
Asst. Chief Jami Resch
Sgt. Aaron Schmautz, President, PPA
Lt. Greg Stewart, (then) lieutenant, Training Division
Sgt. Todd Tackett, Training Division
pg. 19Other People Interviewed
Loren Atherley, Director of Performance Analytics & Research,
Seattle, WA Police Department
Tara Candela, Portland, OR
Brett Cowell, Senior Project Associate, National Police Foundation
Jacob Denny, Economic Justice Policy Director, SPUR, San
Francisco, CA
Beth Epps, Cascadia Behavioral Healthcare
Jan Friedman, Disability Rights Oregon
Tatiana Garcia, Forensic Science and Technical Division, Los Angeles
Police Department, CA
Robert King, National Training Director, Con10gency Consulting, LLC
Cynthia Lum, Ph.D., University Professor, Department of Criminology,
Law, and Society; Director, Center for Evidence-Based Crime
Policy, George Mason University, Fairfax, VA
Mingus Mapps, Portland City Commissioner
Luann P. Pannell, Ph.D., Director, Police Training and Education, Los
Angeles Police Department, CA.
Jonathan Parham, Assessment Training Liability & Accountability
System (ATLAS) and former Chief of Police, Linden NJ.
Brett A. Parson, Training Instructor, Active Bystandership for Law
Enforcement (ABLE), Georgetown School of Law
Stuart Roberts, (retired) Chief of Police, Pendleton, OR; Law
Enforcement Risk Management Consultant, City/County Insurance
Services of Oregon
Julie Ruhlin, OIR Group, Playa del Rey, CA
Sarah Stewart, Learning and Performance Coordinator, Oregon
Department for Public Safety Standards and Training.
Documents consulted
Aerossurance (2014), “James Reason’s 12 principles of error
management.” Available at
http://aerossurance.com/helicopters/james-reasons-12-principles-
error-management/ (accessed 11 February 2022)
Agency for Healthcare Research and Quarterly Patient Safety Network
(2019). “Reporting Public Safety Events,” September 7, 2019.
Available at https://psnet.ahrq.gov/primer/reporting-patient-safety-
events (accessed 2 February 2022)
Assessment Training Liability & Accountability Systems (2021),
https://atlasperformancemanagement.com/ (accessed 08 March
2022)
pg. 20Bennell, C., B. Blaskovits, B. Jenkins, et al. (2021). “Promising
practice for de-escalation and use-of-force training in the police
setting: a narrative review.” Policing: An International Journal, Vol.
44, No. 3, pp. 377-404. Also available
at https://www.emerald.com/insight/content/doi/10.1108/PIJPSM-
06-2020-0092/full/html
Brogle, Rob (2018), How to Avoid the Evils Within Customer
Satisfaction Surveys. https://www.isixsigma.com/methodology/voc-
customer-focus/how-to-avoid-the-evils-within-customer-
satisfaction-surveys/ (accessed 6 May 2022)
California Legislative Information (2015), “AB-953 Law enforcement:
racial profiling”, enacted 1 July 2016.
https://leginfo.legislature.ca.gov/faces/billNavClient.xhtml?bill_id=2
01520160AB953 (accessed 11 February 2022)
Center for Policing Equity (2021), “Data science for justice: How do
you measure justice?” https://www.policingequity.org/ (accessed
11 February 2022)
CHIRP Maritime (2021), An independent and confidential reporting
system for the maritime industry. Maritime Feedback, Issue 65,
November 21. Available at https://www.chirpmaritime.org/wp-
content/uploads/2021/11/MFB-65-November-2021-72dpi-RGB-for-
online.pdf (accessed 2 February 2022)
Corso, J.J. (2017), “BOCA – Body-Worn Camera Analytics in Public
Safety.” Poster presented at the National Institute of Standards and
Technology, U.S. Department of Commerce Public Safety
Analytics Conference. Also available at
https://www.nist.gov/system/files/documents/2017/10/04/psiap_uni
versity_of_michigan_boca_2017.pdf (accessed 26 April 2022).
Deming, W. Edwards (1982), Quality, Productivity, and Competitive
Position. Massachusetts Institute of Technology Center for
Advanced Engineering, Cambridge, Mass.
Dickinson, T. (2022, January 26). Portland Police Proud Boy Meme
Cover-Up Sparks Angry DOJ Letter. Rolling Stone.
www.rollingstone.com/politics/politics-news/portland-police-proud-
boys-training-1290583/ (accessed 24 April 2022)
Ede, Andrew, Ross Homel, Tim Prenzler (2002). “Reducing
complaints against police and preventing misconduct: A diagnostic
study using hotspot analysis. Journal of Criminology, 2002, Vol. 35,
pg. 21No. 1. Abstract available at
https://journals.sagepub.com/doi/abs/10.1375/acri.35.1.27
(accessed 1 February 2022)
Gennaco, Michael, Stephen Connolly, Teresa Magula, Julie Ruhlin
(2022), An Independent Review of the Portland Police Bureau:
Agency Culture, Community Perception, and Public Safety in a
Time of Change. OIR Group, Playa Del Rey, Calif., January 2022.
Available at https://www.documentcloud.org/documents/21192483-
portland-oir-group-portland-culture-project-final-report (accessed
29 January 2022)
Georgetown Law School Innovative Policing Program (2020), “Active
Bystandership for Law Enforcement (ABLE) Project. Available at
https://www.law.georgetown.edu/cics/able/ (accessed 10
September 2021)i
Graham, Gordon (2021), “Deciding what needs to be decided: A 10-
step decision-making process for public safety.” Risky Ramblings:
Risk Management for Public Safety, April 28, 2021. Available at
https://www.police1.com/patrol-issues/articles/deciding-what-
needs-to-be-decided-a-10-step-decision-making-process-for-
public-safety-4NJEo7v2XlyY10PK/ (accessed 1 February 2022)
Independent Police Review (2022), Lessons Learned: City’s response
to protests exposed vulnerabilities in Portland’s police
accountability system. Portland OR, Portland City Auditor,
Independent Police Review, April 2022. (available at
https://www.portland.gov/ipr/news/2022/4/12/lessons-learned-citys-
response-protests-exposed-vulnerabilities-portlands-police
(accessed 14 April 2022)
Kahan, James P., Lisa van Dorp, John A. Stoop, et al. (1997). “The
evolution of transportation safety boards,” in H.M. Soekha (ed.),
Aviation Safety, VSP International Science Publishers, Utrecht, the
Netherlands, pp. 639-647.
KOIN (2022), “Promising results: Violence down from traffic barrels.”
Television news segment shown March 1, 2022.
https://www.koin.com/news/civic-affairs/jo-ann-hardesty-carmen-
rubio-traffic-barrels-mt-scott-arleta-press-conference-03012022/
(accessed 15 March 2022)
Lum, Cynthia, Christopher Koper, C. Gill, et al. (2016). An Evidence-
Assessment of the Recommendations of the President’s Task
Force on 21st Century Policing—Implementation and Research
pg. 22Priorities. Center for Evidence-Based Crime Policy, George Mason
University, Fairfax, VA and International Association of Chiefs of
Police, Alexandria, VA.
Lum, Cynthia, Christopher S. Koper, Megan Stoltz, et al. (2020),
“Constrained gatekeepers of the criminal justice footprint: A
systematic social observation study of 9-1-1 calltakers and
dispatchers.” Justice Quarterly,2020, Vol. 37, No. 7, pp. 1176-
1198. Available at
https://www.tandfonline.com/doi/full/10.1080/07418825.2020.1834
604 (accessed 1 February 2022)
Mann, Nancy R. (1985), The Keys to Excellence: The Story of the
Deming Philosophy. Prestwick Books, Ayleshire, Scotland.
National Institute of Statistical Sciences (2021), “Ingram Olkin Forum
Examines Analyses of Data Related to Police Use of Force,”
Available at https://www.niss.org/news/ingram-olkin-forum-
examines-analyses-data-related-police-use-force (accessed 2
February 2022)
National Police Foundation (no date), Law Enforcement Officer Near
Miss Reporting System. Available at
https://www.policefoundation.org/projects/law-enforcement-officer-
near-miss-reporting-system/ (accessed 2 February 2022)
National Science Foundation (no date)j, Human Subjects.
https://www.nsf.gov/bfa/dias/policy/human.jsp (accessed 5 May
2022)
Noble, Jeffrey J. and Geoffrey P. Alpert (2009), Managing
Accountability Systems for Police Conduct: Internal Affairs and
Police Oversight. Waveland Press, Long Grove, Ill.
Oregon Legislature House Bill HB2929 (2021) “Relating to police
officer misconduct; creating new provisions; enacted effective
January 1, 2022. Available at
https://olis.oregonlegislature.gov/liz/2021R1/Downloads/MeasureD
ocument/HB2929/Enrolled (accessed 2 February 2022)
Oregon Legislature House Bill HB4205 (2020), “Relating to duties of
police officers regarding prohibited behavior; and declaring an
emergency; enacted effective July 1, 2020. Available at
https://olis.oregonlegislature.gov/liz/2020S1/Downloads/MeasureD
ocument/HB4205/Enrolled (accessed 2 February 2022)
pg. 23Portland City Auditor (2022), Police Intelligence-Gathering and
Surveillance: Better management needed to protect civil rights.
Portland, OR: Portland City Auditor Audit Services, April 2022.
Available at
https://www.portland.gov/sites/default/files/2022/police-intelligence-
gathering.pdf (accessed 14 April 2022)
Prenzler, Tim, Tyler Cawthray, Louise E. Porter, et al. (2016).
“Reducing public complaints and use of force: the Portland Police
Bureau Experience,” Journal of Criminological Research, Policy,
and Practice, 2016, Vol. 2, No. 4, pp. 260-273. Available at
https://doi.org/10.1108/HCRPP-12-2015-0054 (accessed 15 July
2021)
Rahr, Sue and Stephen K. Rice, (2015) “From Warriors to Guardians:
Recommitting American police culture to democratic ideals.” New
Perspectives in Policing, Harvard Kennedy School Program in
Criminal Justice Policy and Management, April 2015. Available at
https://nij.ojp.gov/library/publications/warriors-guardians-
recommitting-american-police-culture-democratic-ideals (accessed
1 February 2022)
Reason, James (2013), A Life in Error: From Little Slips to Big
Disasters. CRC Press, London.
Seattle Police Department (2021) Community Feedback.
https://www.seattle.gov/police/information-and-data/community-
feedback (accessed 23 April 2022)
Sered. Danielle (2019). Until We Reckon: Violence, Mass
Incarceration, and a Road to Repair. The New Press, New York.
Sparrow, Malcolm (2015), “Measuring performance in a modern police
organization.” New Perspectives in Policing, Harvard Kennedy
School Program in Criminal Justice Policy and Management,
March 2015. Available at
https://nij.ojp.gov/library/publications/measuring-performance-
modern-police-organization (accessed 1 February 2022)
Training Advisory Council Bylaws (2020) Available at
https://www.portlandoregon.gov/police/article/763118 (accessed 6
February 2022)
University of Wisconsin Law School and Johnson Foundation at
Wingspread (2017), Conference proceedings: Best practices for
review and prevention of deadly incidents in high-hazard
pg. 24You can also read
