The Best of Both Worlds: Strengthening Medical Education Research and Evaluation Using Quality Improvement
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
COMMENTARY
The Best of Both Worlds: Strengthening Medical
Education Research and Evaluation Using
Quality Improvement
Sahar N. Rooholamini, MD, MPH,a Sherilyn Smith, MD,a,b Jimmy Beck, MDa
In 2016, pediatric hospital medicine was recognized as a subspecialty by the American Board of Medical
Specialties, with emphasis on the roles of hospitalists as educators and experts in systems-based
improvement.1–3 Nationally, pediatric hospitalists have diverse nonclinical professional responsibilities, with
medical education and quality improvement (QI) being the most common.4,5 An underrecognized area of
scholarship within pediatric hospital medicine can be described as educational quality improvement (EQI): a
rigorous approach to the implementation and evaluation of educational interventions using QI frameworks and
methodologies.6 Here we describe opportunities and benefits of the intentional incorporation and integration of
QI methodology into medical education research, highlighting the article by Bauer et al7 in the June issue of
Hospital Pediatrics as well as relevant examples from the literature revealing EQI best practices to guide the
rigorous development and evaluation of educational projects of potential interest to pediatric hospitalists.
A literature search of Ovid Medline using the keywords “plan do study act (PDSA)” and “medical education”
over the past decade yielded roughly 60 publications, most of which were published in 2018 or after. The
clinical contexts and problems for which interventions occurred varied (including medical, surgical, pediatric,
and adult specialties, with target groups of faculty, students, residents, and fellows). Projects were focused on
an array of goals, including improving procedural competence,8 handoffs,9 patient flow,10 preparation for
internship,11 and reporting of patient safety incidents.12 However, after careful review of each project’s
methods, we found that QI methods to achieve predetermined educational objectives were applied indevelopment,15 emphasizes careful framework, small sample sizes, the overnight rotation, and ED efficacy.
description of the problem(s) being convenience sampling, and concurrent Their balancing measure was assessed by
addressed before embarking on the interventions. These limitations may asking the daytime hospitalist who
intervention, structured approaches to introduce bias, obscure disparities in assumed care of the patient to determine
understanding the processes through outcomes among learners, and, thus, limit the appropriateness of care, triage
which changes occur, and continuous generalizability. Given that medical placement, and safety concerns with yes
reassessment while the intervention is education RCTs are relatively rare or no responses to monitor safety.
underway to identify further areas for compared with clinical research RCTs, Impressively, the proportion of joint calls
improvement. adapting QI methods provides a way to between a resident and ED physician
A common methodologic issue that strengthen educational studies without increased from 7% to 88% from project
contributes to lack of rigor within medical introducing additional constraints. start to completion and was sustained at
education research and impedes In the June issue of Hospital Pediatrics, >60% over a 21-month period. In addition,
successful publication is the pretest- Bauer et al7 conducted a study titled residents reported significant increases in
posttest study design. Unfortunately, “Letting Residents Lead: Implementing adequacy of triage training and self-
multiple meta-analyses16 have revealed Resident Admission Triage Call Curriculum confidence in 3 triage skills, and there
that the majority of medical education and Practice.” The authors sought to were no complications or safety concerns
studies uses a pretest-posttest study increase resident involvement in overnight for patients admitted by residents.
design. A comparison between 2 cross- admission calls from the emergency What best practices for EQI exist to guide
sections in time often provides the department (ED) to determine appropriate educators who would like to incorporate
benefits of ease and statistical power to patient placement. Before this intervention, QI frameworks and methods in their
detect a change; however, medical hospitalist attending physicians were the curricular work? Four key actions in EQI
education interventions are rarely static, ones who were involved in these study design, based on our literature
and this approach risks drawing admission triage decisions. What was review, are highlighted in Table 1 with
premature conclusions about the impact particularly impactful about this medical demonstrative examples. These 4 key
of an intervention without the benefit of education research project was that the actions are by no means exhaustive,
seeing trends over time.17 Tracking authors evaluated the impact of their meant to replace collaboration and
outcomes over time is fundamental to QI educational curriculum, which they called consultation with medical education and QI
and preferred to pre and post their admission triage curriculum, using QI experts, or a review of published
assessments to allow more accurate methodology. The primary outcome resources for scholarship, such as the
conclusions to be drawn about impact and measure was the frequency of resident Standards for Quality Improvement
sustainability. Rigorous evaluation of participation in joint and independent Reporting Excellence in Education,18 which
medical education interventions may also triage calls. Secondary outcomes included highlight important items authors should
be constrained by lack of a guiding self-perceived confidence, satisfaction with consider when applying QI methods to
TABLE 1 Key Elements of High-Quality EQI Projects With Published Examples
Element Published Example
1. Use a QI framework to clearly define what changes would constitute Handley et al19: educational objectives: increase identification of residents’
improvement: state clear educational objectives and a priori SMART aims, learning goals and goal achievement on a NICU or infant ICU rotation and
including process and balancing measures. increase rates of in-person feedback from fellows and/or attending
physicians; SMART aims: increase NICU or infant ICU rotating resident
goal identification to 65%, goal achievement to 85%, and in-person
feedback from a fellow and/or attending to 90% over 12 mo and sustain
these improvements for the following academic year
2. Develop and engage an inclusive project team with key stakeholders Dunbar et al12: faculty, chief residents, senior residents, and the hospital’s
(often a group larger than the targeted learners) to strengthen the interdisciplinary safety team all contributed to and participated in this
intervention, assist with iterative tests of change, and help sustain project to increase faculty and resident reporting of patient safety events
improvements (eg, PDSA cycles). through deliberate inclusion of patient safety into the teaching team’s
workflow.
3. Use QI tools (eg, key driver diagrams, Pareto charts, and process maps) Patel et al20: key driver diagram, included in article, outlining all of the
to organize and visually represent the gap(s) being addressed, following: project’s global aim (“modernize teaching on family-centered
intervention processes, and metrics. rounds”), SMART aim, key drivers and outcomes, and interventions with
levels of reliability for each
4. Design data collection over time. Avoid pre- and postintervention Byrd et al21: to track the number of times that a mobile device was used on
comparisons. Display data over time using run charts, SPC charts, and/or rounds by inpatient teams; weekly observations were performed and
time series analysis. analyzed over time by using an SPC chart
PDSA, plan do study act; SMART, specific, measurable, attainable, relevant, time-bound; SPC, statistical process control.
2 ROOHOLAMINI et al
Downloaded from www.aappublications.org/news by guest on November 1, 2021medical education studies. We recommend a qualitative exploration. Acad Pediatr. 13. Colli A, Pagliaro L, Duca P. The ethical
intentionally incorporating these key 2014;14(4):361–368 problem of randomization. Intern Emerg
actions and Standards for Quality 6. Rooholamini S, Starr S, Hayward K, Med. 2014;9(7):799–804
Improvement Reporting Excellence in Colson E, Smith S. Using quality 14. Langley GJ, Moen RD, Nolan KM, Nolan
Education guidelines into the planning improvement concepts to improve TW, Norman CL, Provost LP. The
stages of an EQI project so that educational curricula. In: Morgenstern B, Improvement Guide: A Practical
interventions and measurable aims can be Horak H, Konopasek L, et al, eds. Alliance Approach to Enhancing Organizational
clearly defined up front and more useful for Clinical Education Guidebook for Performance. 2nd ed. San Francisco, CA:
results can be captured over time, all of Clerkship Directors. 5th ed. North Jossey-Bass Publishers; 2009
which are more likely to lead to successful Syracuse, NY: Gegensatz Press; 2019: 15. Thomas PA, Kern DE, Hughes MT, Chen
scholarship and publication.
473–484 BY, eds. Curriculum Development for
This commentary highlights how existing Medical Education: A Six-Step Approach.
7. Bauer SC, McFadden V, Madhani K,
frameworks within QI can be integrated 3rd ed. Baltimore, MD: Johns Hopkins
Kaeppler C, Porada K, Weisgerber MC.
into medical education research to bring University Press; 2016
Letting residents lead: implementing
greater validity and strength to data 16. Cook DA. If you teach them, they will
resident admission triage call
collection and analysis. Medical education learn: why medical education needs
curriculum and practice. Hosp Pediatr.
and QI share an openness to iterative comparative effectiveness research. Adv
2021;11(6):579–586
improvement and learning by doing. By Health Sci Educ Theory Pract.
more intentionally merging approaches 8. Chen JX, Kozin E, Bohnen J, et al.
2012;17(3):305–310
from these 2 fields, the dynamic Assessments of otolaryngology resident
operative experiences using mobile 17. Brady PW, Tchou MJ, Ambroggio L,
relationship between testing interventions
technology: a pilot study. Otolaryngol Schondelmeyer AC, Shaughnessy EE.
and learning from them can be more
Head Neck Surg. 2019;161(6):939–945 Quality improvement feature series
rigorously described and disseminated.
article 2: displaying and analyzing quality
9. Quinones Cardona V, LaBadie A, improvement data. J Pediatric Infect Dis
REFERENCES Cooperberg DB, Zubrow A, Touch SM. Soc. 2018;7(2):100–103
1. Ottolini MC. Pediatric hospitalists and Improving the neonatal team
medical education. Pediatr Ann. 18. Ogrinc G, Armstrong GE, Dolansky MA,
handoff process in a level IV NICU:
2014;43(7):e151–e156 Singh MK, Davies L. SQUIRE-EDU (Standards
reducing interruptions and handoff
for QUality Improvement Reporting
2. Heydarian C, Maniscalco J. Pediatric duration. BMJ Open Qual.
Excellence in Education): publication
hospitalists in medical education: 2021;10(1):e001014
guidelines for educational improvement.
current roles and future directions. Curr 10. Kouo T, Kleinman K, Fujii-Rios H, et al. A Acad Med. 2019;94(10):1461–1470
Probl Pediatr Adolesc Health Care. resident-led QI initiative to improve
19. Handley SC, Pouppirt N, Zucker E,
2012;42(5):120–126 pediatric emergency department
Coughlin KA. Improving the resident
3. Barrett DJ, McGuinness GA, Cunha CA, boarding times. Pediatrics. 2020; educational experience in a level IV
et al. Pediatric hospital medicine: a 145(6):e20191477 neonatal/infant intensive care unit.
proposed new subspecialty. Pediatrics. 11. Monday LM, Gaynier A, Berschback M, Pediatr Qual Saf. 2020;5(6):e352
2017;139(3):e20161823 et al. Outcomes of an online virtual 20. Patel A, Unaka N, Holland D, et al.
4. Freed GL, Dunham KM; Research boot camp to prepare fourth-year Increasing physical exam teaching on
Advisory Committee of the American medical students for a successful family-centered rounds utilizing a web-
Board of Pediatrics. Pediatric transition to internship. Cureus. based tool. Pediatr Qual Saf.
hospitalists: training, current practice, 2020;12(6):e8558 2017;2(4):e032
and career goals. J Hosp Med. 12. Dunbar AE III, Cupit M, Vath RJ, et al. 21. Byrd AS, McMahon PM, Vath RJ, Bolton
2009;4(3):179–186 An improvement approach to integrate M, Roy M. Integration of mobile devices
5. Leyenaar JK, Capra LA, O’Brien ER, Leslie teaching teams in the reporting of to facilitate patient care and teaching
LK, Mackie TI. Determinants of career safety events. Pediatrics. during family-centered rounds. Hosp
satisfaction among pediatric hospitalists: 2017;139(2):e20153807 Pediatr. 2018;8(1):44–48
HOSPITAL PEDIATRICS Volume 11, Issue 11, Month 2021 3
Downloaded from www.aappublications.org/news by guest on November 1, 2021The Best of Both Worlds: Strengthening Medical Education Research and
Evaluation Using Quality Improvement
Sahar N. Rooholamini, Sherilyn Smith and Jimmy Beck
Hospital Pediatrics originally published online June 21, 2021; originally published
online June 21, 2021;
Updated Information & including high resolution figures, can be found at:
Services http://hosppeds.aappublications.org/content/early/2021/09/30/hpeds.
2021-006122
Supplementary Material Supplementary material can be found at:
References This article cites 18 articles, 6 of which you can access for free at:
http://hosppeds.aappublications.org/content/early/2021/09/30/hpeds.
2021-006122#BIBL
Subspecialty Collections This article, along with others on similar topics, appears in the
following collection(s):
Medical Education
http://www.hosppeds.aappublications.org/cgi/collection/medical_edu
cation_sub
Quality Improvement
http://www.hosppeds.aappublications.org/cgi/collection/quality_impr
ovement_sub
Teaching/Curriculum Development
http://www.hosppeds.aappublications.org/cgi/collection/teaching_cur
riculum_dev_sub
Permissions & Licensing Information about reproducing this article in parts (figures, tables) or
in its entirety can be found online at:
http://www.hosppeds.aappublications.org/site/misc/Permissions.xhtm
l
Reprints Information about ordering reprints can be found online:
http://www.hosppeds.aappublications.org/site/misc/reprints.xhtml
Downloaded from www.aappublications.org/news by guest on November 1, 2021The Best of Both Worlds: Strengthening Medical Education Research and
Evaluation Using Quality Improvement
Sahar N. Rooholamini, Sherilyn Smith and Jimmy Beck
Hospital Pediatrics originally published online June 21, 2021; originally published
online June 21, 2021;
The online version of this article, along with updated information and services, is
located on the World Wide Web at:
http://hosppeds.aappublications.org/content/early/2021/09/30/hpeds.2021-006122
Hospital Pediatrics is an official journal of the American Academy of Pediatrics. Hospital
Pediatrics is owned, published, and trademarked by the American Academy of Pediatrics, 345
Park Avenue, Itasca, Illinois, 60143. Copyright © 2021 by the American Academy of Pediatrics.
All rights reserved. Print ISSN: 1073-0397.
Downloaded from www.aappublications.org/news by guest on November 1, 2021You can also read
