National Pediatric Experience With Virtual Interviews: Lessons Learned and Future Recommendations
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
National Pediatric Experience With
Virtual Interviews: Lessons Learned
and Future Recommendations
John G. Frohna, MD, MPH,a Linda A. Waggoner-Fountain, MD, MACM,b Jill Edwards, MBA,c Jill J. Fussell, MD,d
Beth Wueste, MAEd,e Joseph Gigante, MD,f Robert J. Vinci, MD,g Nicholas M. Heitkamp, MD, MSc,h
Mekala K. Neelakantan, MD,i Laura E. Degnon, CAE,j Rebecca L. Blankenburg, MD, MPH,k Pediatrics Recruitment Study Team*
The coronavirus disease 2019 pandemic significantly impacted undergraduate abstract
and graduate medical education and created challenges that prevented a a
Departments of Pediatrics and Medicine, School of Medicine
traditional approach to residency and fellowship recruitment and interviews. and Public Health, University of Wisconsin, Madison,
Early in the pandemic, the pediatric education community came together to Wisconsin; bDepartment of Pediatrics, School of Medicine,
University of Virginia, Charlottesville, Virginia; cChildren’s
support applicants and training programs and to foster an equitable recruitment Mercy, Kansas City, Missouri; dUniversity of Arkansas for
process. We describe many of our community’s innovations, including the use of Medical Sciences, Little Rock, Arkansas; eDepartment of
Pediatrics, University of Texas Health, San Antonio, Texas;
virtual cafes to educate programs and highlight best practices for virtual f
Vanderbilt University Medical Center, Nashville, Tennessee;
recruitment and the use of regional webinars to highlight residency programs g
Boston Medical Center, Boston, Massachusetts; hDepartment
of Pediatrics, Eastern Virginia Medical School, Norfolk,
and provide information to applicants. Surveys of applicants and programs Virginia; iDepartment of Pediatrics, University of Michigan
suggest that the virtual interview process worked well overall, with applicants Medical School, Ann Arbor, Michigan; jAssociation of
and programs saving both time and money and programs maintaining a high Pediatric Program Directors, McLean, Virginia; and
k
Department of Pediatrics, School of Medicine, Stanford
rate of filling their positions. On the basis of this experience, we highlight the University, Stanford, California
strengths and weaknesses of 3 potential models for future interview seasons. *A complete list of group members appears in the
We close with a series of questions that need further investigation to create an Acknowledgments.
effective and equitable recruitment process for the future. Drs Frohna, Waggoner-Fountain, and Blankenburg
conceptualized the manuscript, drafted the initial
manuscript, and reviewed and revised the manuscript; Drs
Gigante, Heitkamp, Neelakantan, Vinci and Ms Edwards, Ms
Fussell, Ms Degnon, and Ms Wueste critically reviewed the
manuscript for important intellectual content, and reviewed
The coronavirus disease 2019 To prepare for the pandemic’s and revised the manuscript; and all authors approved the
final manuscript as submitted and agree to be accountable
(COVID-19) pandemic has impact on the 2020–2021 interview for all aspects of the work.
significantly impacted undergraduate season, the Association of Pediatric DOI: https://doi.org/10.1542/peds.2021-052904
and graduate medical education and Program Directors (APPD)
Accepted for publication Jul 26, 2021
created challenges that prevented a collaborated with other national
Address correspondence to John G. Frohna, MD, MPH,
traditional approach to residency pediatric education organizations to Departments of Pediatrics and Medicine, University of
Wisconsin, School of Medicine and Public Health, 2870
and fellowship interviews. In convene 2 action teams, the University Ave, Suite 212, Madison, WI 53705. E-mail: frohna@
response to health concerns, travel Residency Recruitment Action Team wisc.edu
limitations, and pandemic disparities, and the Fellowship Recruitment PEDIATRICS (ISSN Numbers: Print, 0031-4005; Online,
1098-4275).
residency and fellowships quickly Action Team. These teams included
Copyright © 2021 by the American Academy of Pediatrics
shifted to a virtual interview process representatives from each of these
FINANCIAL DISCLOSURE: The authors have indicated
for the 2020–2021 academic year.1 key stakeholder groups: Council on they have no financial relationships relevant to this
Despite these challenges, Pediatrics Medical Student Education in article to disclose.
successfully filled 2860 of 2901 Pediatrics (COMSEP), Council of FUNDING: No external funding.
positions (98.6%) in the 2021 match, Pediatric Subspecialties (CoPS), POTENTIAL CONFLICT OF INTEREST: The authors have
indicated they have no potential conflicts of interest
which increased slightly from the Association of Medical School
98.2% fill rate in 2020.2 For the Pediatric Department Chairs To cite: Frohna JG, Waggoner-Fountain LA, Edwards
pediatric fellowship matches that (AMSPDC), and 3 learner groups: J, et al. National Pediatric Experience With Virtual
occurred in Fall 2020, the match American Academy of Pediatrics Interviews: Lessons Learned and Future
Recommendations. Pediatrics.
rates were similar or increased for Section on Pediatric Trainees, 2021;148(4):e2021052904
16 of 17 pediatric specialties.3 NextGenPediatricians (a resident-
Downloaded from www.aappublications.org/news by guest on November 4, 2021
PEDIATRICS Volume 148, number 4, October 2021:e2021052904 SPECIAL ARTICLEand/or fellow-led mentoring PEDIATRIC EDUCATION COMMUNITY’S of COVID-19. Our initial
program for fourth-year medical APPROACH TO THE 2020–2021 communication with programs and
students who are underrepresented INTERVIEW SEASON applicants referred applicants to the
in medicine because of race and/or The COVID-19 pandemic had several Association of American Medical
ethnicity and applying to pediatrics adverse impacts on medical students Colleges (AAMC) “Apply Smart” Web
and/or combined-pediatrics and residents pursuing pediatric site,9 which provides evidence-based
programs), and FuturePedsRes residency or fellowship positions, guidance for applying to an optimal
(a student-run grassroots respectively. First, many applicants number of residency programs. We
organization to help students had an altered exposure to both also provided reminders of this
navigate the recruitment process in general and subspecialty pediatric guidance during the webinars.
pediatrics). The Residency Action disciplines. Medical students
Team included medical students experienced cancellation or APPD and COMSEP partnered with a
and residents, and the Fellowship significant disruption in their new medical student–led group,
Action Team included fellow clerkships, including subinternships, FuturePedsRes, and hosted a series
representatives. which affected both their clinical of webinars designed to improve
experiences and ability to obtain student awareness of programs
The action teams developed a letters of recommendation. across the country and help mitigate
series of recommendations for the Applicants were also impacted by applicant anxiety. In total, we hosted
2020–2021 interview season with the cancellation of away rotations. nine 90-minute webinars (2 general
the goal of optimizing the Programs had less data than in webinars regarding the application
recruitment process for both previous years to assess applicants and interview process, 5 regional
learners and programs. Four core because research experiences and webinars, 1 webinar for osteopathic
principles guided the volunteer activities were students, and 1 webinar for IMGs).
recommendations: (1) helping interrupted, and licensure The regional webinars consisted of
learners find programs that match examinations, including both US (1) a brief overview of the region,
their career goals while providing Medical Licensing Examination and (2) small groups composed of 5 to
an atmosphere conducive to their Comprehensive Osteopathic Medical 8 programs that made brief
learning styles; (2) providing Licensing Examination, were presentations on their programs,
residency and/or fellowship postponed. The COVID-19 pandemic (3) live question-and-answer
programs with a standardized disproportionately affected Black, sessions with programs, and
approach to recruitment; (3) Indigenous, and people of color (4) opportunities for students to
creating a fair and equitable applicants; those from lower meet exclusively with residents. The
application process for both socioeconomic status backgrounds; didactic content of these webinars
learners and programs; and (4) and international medical graduates lasted 60 minutes, and the time with
minimizing the disruptions created (IMGs).4–8 In response to these the students and residents was 30
by the COVID-19 pandemic and concerns, APPD, COMSEP, CoPS, and minutes. Programs developed a 1-
ensuring reasonable health AMSPDC proposed several slide infographic using a uniform
precautions during the interview recommendations and developed template to standardize the
season. several initiatives designed to assist information available to the applicant
applicants and programs. attendees. We recorded the small-
The goals of this article are to (1) group presentations and placed them
describe the pediatric education Helping Learners Navigate the on YouTube (https://www.youtube.
community’s approach to the Virtual Interview Process com/c/futurepedsres) for access by
2020–2021 interview season, Our action teams took several steps interested applicants. These webi-
findings from our preliminary to help applicants navigate the nars were well received, with more
analysis, and lessons learned and virtual interview season. Through than half of all pediatric programs
(2) using the lessons learned, the wide dissemination of our participating (N = 138 of 211) and
discuss the pros and cons of 3 recommendations and our webinars, a consistently large number of
interviewing options, including in- we reassured residency and applicants in attendance for each
person, virtual, and hybrid models, fellowship applicants that programs session (range 90–936, mean 431).
and identify questions that require recognized that there might be gaps FuturePedsRes also encouraged
further study before making long- in their clinical, research, advocacy, pediatric applicants to “take the
term changes to the interview and other extracurricular pledge” through the social media
process. experiences because of the impacts #ApplySmart Campaign to limit
Downloaded from www.aappublications.org/news by guest on November 4, 2021
2 FROHNA et altheir applications to 15 programs, Creating a Fair and Equitable they would be offered an interview,
as recommended by the AAMC Application Process for Both waitlisted, or not offered an interview
(barring other mitigating circum- Learners and Programs by December 15th. We recommended
stances), with the goal of decreas- Despite the challenges of similar adjustments for the fellowship
ing overall numbers of applications interviewing in a new format and interview season. Again, we strongly
so programs could complete holis- during a pandemic, APPD and discouraged second-look visits, even
tic applicant reviews. For fellow- COMSEP also issued guidance for internal or local applicants to
ship applicants, APPD and CoPS designed to optimize equity for all maintain a safe and equitable
hosted a virtual cafe to provide parties. First, although holistic experience for all applicants.
information about the fellowship review of residency and fellowship
applications has been discussed for We also made recommendations to
season and answer applicant
several years, our heightened focus limit postinterview communication.
questions.
on equity helped us to strongly Beyond adhering to existing
Providing Residency and/or encourage programs to move National Resident Matching Program
Fellowship Programs With a toward a holistic review of rules, we encouraged programs and
Standardized Approach to applications. The APPD described applicants to only communicate in
Recruitment and highlighted models for the event of specific questions or to
performing holistic reviews during link applicants with mentors and/or
APPD, COMSEP, and CoPS provided
webinars for program leaders. We research colleagues. Recognizing
guidance and support to programs
reminded programs that students’ that programs were under
in several different ways. First, we
access to educational, research, and significant financial constraints
encouraged programs to develop
extracurricular opportunities are because of COVID-19 and that some
strategies to showcase their
inequitable at baseline, and many programs had reduced
programs virtually, by updating
students experienced nontraditional administrative support, we asked
their Web sites and developing
clerkships during the pandemic, that programs not provide
digital brochures, videos, or other
including virtual-learning experiences applicants with gift cards for food
resources to highlight important
and online educational programming. and that they not provide any
aspects of their programs. APPD, program-related gifts to applicants.
In addition, we asked programs to
COMSEP, CoPS, and AMSPDC
reexamine their traditional criteria for
developed a series of virtual cafes to Finally, we structured the regional
offering interviews and to modify or
provide an opportunity for webinars for residency applicants
waive requirements for pediatric-
programs to share best practices in described previously to provide
specific letters of recommendation, a
virtual recruitment.10 A group of equitable opportunities for
pediatric subinternship rotation, and
>20 program coordinators formed programs. We invited all programs
completion of US Medical Licensing
the APPD Recruitment Resource to participate; elected regional APPD
Examination Step 2CK or
Toolkit Development Team and leaders gave an overview of the
Comprehensive Osteopathic Medical
compiled an extensive collection of opportunities in the region; and
Licensing Examination Level 2-CE by
tools to support virtual recruitment the time of initial review. Although we programs presented information
and the professional development of developed these recommendations in about their program using a
program learners and leaders.11 response to COVID-19 restrictions, templated infographic to reduce
applicants also benefitted from variability. To remain equitable to
Second, we provided programs with the programs as well as applicant
minimizing the use of test scores and
guidance about conducting virtual participants across the series, we
other elements of an applicant’s
interviews. Given the national randomly assigned students to small
portfolio that are known to introduce
recommendations for virtual biases into the interview process. groups for the residency program
interviews, we discouraged programs presentations, with access to
from scheduling any in-person Second, for residency interviews, we recordings of each group afterward.
interviews, even for internal or local recommended that the AAMC
applicants. We recommended that postpone the Electronic Residency PRELIMINARY FINDINGS AND LESSONS
programs offer the daytime Application Service release date by 3 LEARNED DURING THE 2020–2021
components of the interview day weeks to allow adequate time for INTERVIEW SEASON
(individual interviews, conferences, applicants and medical schools to We deployed 2 sets of surveys to
tours, etc) on a single day to make prepare their applications and asked assess the impact of the virtual
scheduling easier for applicants. programs to notify all applicants if interview season on stakeholders,
PEDIATRICS Volume 148, number 4, Downloaded
October 2021from www.aappublications.org/news by guest on November 4, 2021 3including program directors, seasons reported by respondents with on-site visits for tours and
program coordinators, and included greater equity for meetings with residents and/or
residency applicants. applicants and cost savings for both program leaders) significantly
programs and applicants. Challenges higher than the in-person or virtual
The first survey, developed by the included technological issues and models (hybrid 78.5 [SD = 28.4])
APPD Recruitment Resource Toolkit less opportunity for in-person versus in person (56.0 [26.1])
Development Team, solicited interactions. versus virtual (50.6 [27.2]) (P <
feedback from residency programs. .0001 for both comparisons).
This survey included questions to The second survey was sent to
assess the usefulness and efficacy of residency applicants who When asked about advantages of the
the virtual toolkit, as well as participated in the #PedsMatch21 virtual interview season, a majority
programs’ overall satisfaction with Webinar Series. This survey was of respondents reported significant
the tools provided and/or used. used to solicit feedback on the savings in interview costs and time,
Additionally, the survey compared webinars and to collect data related less fatigue because of lack of travel,
the 2019–2020 traditional interview to the virtual interview process, and less difficulty scheduling
season to the 2020–2021 virtual including perceptions on the current interviews. Several respondents
season regarding areas such as time season and their preferences for stated some variation of a “more
commitment, workload, and future interviewing models. This even playing field for those who do
technology. The survey consisted of survey was distributed by e-mail in not have as much financial ability to
both forced-choice and open-ended March 2021, with 2 reminders for travel,” as an additional advantage.
questions and was institutional nonrespondents, and closed before In terms of disadvantages to the
review board–approved and Match Day (March 19, 2021). The virtual season, many respondents
reviewed by the APPD Research and survey included both forced-choice reported the inability “to more
Scholarship Learning Community. and open-ended questions. thoroughly get to know a program,
We distributed the survey by e-mail its culture and people, its facilities,
to residency program directors from We received responses from 265 and its city.” Others reported fatigue
January 2021 to May 2021, with 2 residency applicants (33%) of the from being on virtual interviews for
additional reminders for 805 who attended the webinars. long periods of time, and some
nonrespondents. Recognizing that groups of applicants were concerned that
applicants may have varying programs were able to hide the less
We received responses from 97 of perspectives, we categorized desirable aspects of their programs.
200 residency program directors respondents into 3 groups: MD, DO, Some respondents also expressed
(49%). Residency programs and IMGs (Table 2). In the survey, concern about “interview hoarding”
reported completing a greater we found that DO and IMGs applied by other applicants.
number of interviews in 2020–2021 to more programs than MD students
(Table 1). Although programs and that both MD and DO applicants The Residency Recruitment and
reported an initial increase in interviewed with and ranked a Fellowship Recruitment action
workload in preparing for the similar number of programs. All 3 teams met regularly during the
virtual season, they also endorsed groups canceled a low number of interview season to review the
that the inaugural virtual interview interviews. Each of the groups experiences of stakeholders and
season went better than expected. In reported that they were only slightly after the residency and fellowship
addition, they felt that the virtual more likely to rank programs in matches to consider 3 options for
process saved time and anticipated cities that they had lived in or conducting interviews during the
continued time savings if the previously visited (MD 2.5 [SD = 2021–2022 season. In April 2021,
interview season remained virtual in 1.4], DO 2.5 [1.5], and IMGs 2.4 the Coalition for Physician
future years. Although a hybrid [1.4], in which 1 = strongly agree Accountability issued preliminary
model for interviews was not and 3 = neutral), which was recommendations from their
offered in this survey, residency unexpected given that previous Undergraduate Medical Education to
directors (58%) had a preference studies identified geography as an Graduate Medical Education Review
for virtual interviews only if an important predictor regarding how Committee for the undergraduate
applicant had extenuating applicants rank programs.12,13 Using medical education to graduate
circumstances, and only 2% a slider scale from 0 (no preference) medical education transition.14 The
expressed a preference for in-person to 100 (best situation imaginable), recommendations Coalition for
interviews for all applicants. all applicants rated a hybrid model Physician Accountability made about
Advantages to the virtual interview (defined as virtual faculty interviews the interview process were
Downloaded from www.aappublications.org/news by guest on November 4, 2021
4 FROHNA et alTABLE 1 Residency PD Interview Season Experience and Recommendations for Future Interview Models
Residency PDs
(N = 97 of 200)
Interview Season Comparison 2019–2020 Season, % 2020–2021 Season, %
No. applicants interviewed
26–50 1 —
51–100 6 5
101–150 17 7
151–200 21 11
201–250 19 18
251–300 14 16
>300 22 43
Estimated total time commitment per week (combined hours for all team
members: PDs, APDs, coordinators, chief residents, faculty), h
50–100 31 30
101–200 31 25
>200 38 45
The transition to virtual interviews affected the workload for my program (5 = 3.92 3.92
significantly increased, 1 = significantly decreased)
The process of virtual interviewing was (% better than expected) 97 97
The virtual interview season saved time
Yes 46 46
Unsure 14 14
No 40 40
Virtual interviewing will save time in future years
Yes 61 61
Unsure 16 16
No 23 23
On the basis of the experience in the 2020–2021 interview season, in future
seasons, our program would offer interviews in the following format:
All in person 2 2
All virtual 27 27
Virtual interviews for applicants with extenuating circumstances only 58 58
Virtual or in-person interviews, depending on applicant preference 13 13
APD, assistant program director; PD, program director; —, not applicable.
consistent with the consensus of our Benefits of this model include better been estimated that the median cost
action teams. ability of programs to assess applicant of a traditional residency interview
communication skills; more organic season is $4000, with a range
The 3 options for interviews that and informal conversation between from $1000 to >$11000.15 The
the action teams discussed included applicants and residents and/or median cost per program for
conducting interviews all in person, faculty and/or coordinators at residency recruitment for internal
all virtually, and a hybrid approach programs; and the opportunity for medicine programs was $148 000 in
(consisting of virtual interviews and applicants to evaluate the facilities and 200916; using different
a separate, often abbreviated in- resources of the programs. This option methodology, authors of a survey of
person component) (Table 3). Below also allows applicants to see the city in family medicine programs estimated
we discuss the pros and cons of which they might live and work. In a cost in excess of $25 000 per
each model: addition, UIM applicants have program in 2019.17 In addition to
highlighted that in-person interviews the financial costs, applicants
Interviews All in Person usually spend several weeks
provide an opportunity to consider the
This model is the traditional model for institution’s culture and inclusion. traveling to and interviewing at
residency and fellowship interviews, in programs, which results in
which interviews occur in person and There are several costs associated decreased time on clinical rotations.
applicants are able to tour the clinical with this option. Although the In addition to financial costs, there
facilities, meet with program residents financial cost to applicants varies on are broader environmental impacts
and faculty, and potentially explore the the basis of specialty choice and of interview travel. Recently, in a
city in which they would be living. range of geography desired, it has study from the University of
PEDIATRICS Volume 148, number 4, Downloaded
October 2021from www.aappublications.org/news by guest on November 4, 2021 5TABLE 2 Applicant Interview Season Experience and Recommendations for Future Interview Models
MD Students (n = 161) DO Students (n = 48) IMGs (n = 56)
Applications and interviews
Programs applied to 27.7 (10.4) 47.1 (18.1) 117.3 (116.0)
Interviews canceled 2.6 (3.2) 2.3 (4.1) 0.7 (1.5)
Programs interviewed at 14.7 (4.4) 13.6 (5.3) 8.6 (7.1)
Programs ranked 14.4 (4.0) 13.1 (4.2) 8.2 (6.9)
I would have canceled more interviews if the interviews were in persona
Yes 43 30 20
No 21 28 54
Maybe 36 41 26
Rating of future interview models (0–100 slider scale)b
In person 56.4 (26.2) 48.9 (24.9) 53.5 (28.2)
Virtual 50.2 (27.5) 55.6 (22.9) 55.7 (31.0)
Hybrid 77.6 (29.4) 81.2 (25.4) 75.6 (32.3)
Values are shown as mean (SD).
a
International applicants were less inclined than MD or DO students to cancel interviews if they were held in person (P < .05).
b
All groups preferred the hybrid model significantly more than either the in-person or virtual model (P < .0001 for both comparisons).
Michigan, researchers estimated a are further considerations for the applicants because it may foster an
carbon footprint from travel of 3.07 2021–2022 interview season. increased focus on the application
metric tons of carbon dioxide per materials, with attempts to select
student, who attended an average of
Interviews All Virtual candidates on the basis of a broad
14.4 interviews per person.18 The advantages of this model, based range of characteristics that are
on the 2020–2021 interview season, important to programs. Virtual
Although vaccination rates are were the significantly reduced financial interviews may also mitigate some of
rising, especially among health care costs for applicants and programs and the inequity inherently present in the
personnel, the medical risk of less time away from the education current interview process. For the
travel, the differing ability of health program, an observation that was 2020–2021 interview season, having
systems to host large numbers of particularly notable for fellowship all programs and applicants interview
applicants, and the variable applicants. An all-virtual model for using this model equalized the playing
restrictions on travel by home interviews may also support the shift field for all involved. The significantly
institutions and around the country toward a more holistic review of lower financial costs of virtual
TABLE 3 Pros and Cons of 3 Interview Models
All in Person All Virtual Hybrid
Pros It is the standard Better for applicants in terms of equity, Maximizes the benefit of the virtual
cost, and time interviews and allows applicants to see
Applicants get to visit the program Encourages transition to holistic review of the program in person and, potentially,
candidates for the program to meet the applicants
Programs meet the applicants Programs have already invested in
resources for virtual interviews
Cons Significant costs for applicants in terms of Limited opportunity to experience the feel May not be equitable if the applicants feel
time and travel for the program, community, city, etc they must visit all programs to
demonstrate their interest
Equity concerns Could contribute to application excess Increases the work for programs and
(although not seen in pediatrics in 2020) applicants
Environmental impact with substantial May negatively impact small programs
carbon footprint that may benefit by having applicants
see the program
Bottom line Allows applicants to see all programs and Emphasizes equity and safety Provides some measure of equity and
programs to meet all applicants allows programs flexibility for their
own circumstances
May be a health and/or travel risk until Aligns with Coalition for Physician Increased work for programs and
pandemic resolves Accountability recommendation for increased costs for applicants
virtual interviews in 2021–2022
More expensive for both applicants and May disadvantage some programs and
programs applicants
Downloaded from www.aappublications.org/news by guest on November 4, 2021
6 FROHNA et alinterviews also resulted in a more model might maximize the benefits nature of such signals are subjective
equitable opportunity for applicants of each option while providing measures that can add bias to the
who could not afford to interview at flexibility for applicants and interview process. An option to limit
as many places in person. The virtual- programs. The hybrid model has this bias would be to separate the
only interview process may have been explored by O’Malley et al in-person visit from the ranking
encouraged applicants to apply to before the COVID-19 pandemic.21 In process, but the logistic challenges
more geographically diverse programs. their model, the University of of scheduling these events would
The virtual model was also used Arizona Internal Medicine Residency still exist. One consideration is to
successfully for medical school program developed a “SPLIT” partner with the National Resident
interviews, with students citing many interview process in 2017 to Matching Program to move the
of the same advantages as residents.19 address costs, faculty and resident program rank date a month earlier
burden, and duplication of program than applicants, thereby allowing
In contrast, a virtual-only interview director efforts. Their outcomes applicants the option to complete an
process makes it challenging for have included increasing the in-person visit without worry that it
programs and applicants to really get a number of interviews by 50%, would influence the program’s rank
feel for each other, an impact that is improved flexibility, and decreased list. Another option is to have a
hard to define and difficult to measure. total visit events by 70%, and centralized graduate medical
Additionally, the lower cost of allowed 100% of applicants to be education office run the in-person
interviewing could lead to applicants interviewed by a program director visit component to minimize any
interviewing at more programs, adding and associate program director.21 bias from program and/or applicant
to the administrative work of programs interaction. Finally, in-person visits
to review more applications. This was There are 2 major obstacles to the increase the cost to both applicants
counteracted for residency applications widespread use of this model, and programs, challenging the goals
in 2020 with the extensive particularly for the main residency of equity that our community strives
FuturePedsRes, APPD, and COMSEP match: scheduling logistics and to uphold.
campaign to only apply to 15 programs potential for inequity. Planning a
for the residency match. Electronic hybrid schedule is complex because RECOMMENDATIONS AND AREAS FOR
Residency Application Service reports this requires programs to allow for FURTHER STUDY
that overall, pediatric applicants applied a series of virtual interview days in After gathering data from
to a slightly decreased median number addition to a set of in-person stakeholders, including applicants,
of programs in 2020–2021 compared sessions. Applicants would have to program leadership, and chairs, the
with 2019–2020 (30 vs 31), whereas manage a more complicated recruitment action teams have
other large specialties saw increases in scheduling process to accommodate recommended proceeding with
the mean number of applications per all of these events. Secondly, virtual-only interviews in 2021–2022,
person (family medicine 11.46, regarding equity, even if programs with the plan to reevaluate for future
internal medicine 17.55, and general stated that an in-person visit was years thereafter.22 There is still
surgery 15.74). Given that IMG and DO not required for an applicant to be significant uncertainty about the
applicants generally apply to more ranked, applicants would likely feel impacts of the COVID-19 pandemic on
programs, and more IMG applicants an obligation to travel because of health systems and cities across the
were in the pools for general surgery the inherent competitiveness of the country. Given that programs and
and internal medicine, we also looked application process, and some applicants felt that virtual residency
at the change in the mean numbers of applicants may be better positioned recruitment was overall effective this
applications for MD applicants from than others to avail themselves of past year, the action teams believe
2019–2020 to 2020–2021 seasons for this option. There is also the that the benefit of reduced financial
these specialties and found that only potential for programs’ bias toward costs and increased applicant equity
pediatrics had fewer applications per those applicants who attend in- in a virtual system outweigh the
applicant: pediatrics (-2.06), family person events. The COVID-19 downsides of not having program
medicine (11.86), internal medicine pandemic has shed light on these leaders and applicants meet in person.
(11.89), and general surgery inequities in the art of recruitment, If the health risks continue to decline,
(13.63).20 and although we must balance the it would certainly be possible for
potential impact of signals related to applicants to visit the cities in which
Hybrid Model postinterview communication, they might potentially match; we still
Given the challenges of both the in- interview interactions, and so on, recommend that programs not host
person and virtual models, a hybrid we must also be mindful that the applicants in their health systems
PEDIATRICS Volume 148, number 4, Downloaded
October 2021from www.aappublications.org/news by guest on November 4, 2021 7(because of burdens on the system) or program has to conduct to reliably fill Douglas Carlson, MD, Southern Illinois
have program personnel meet with their program? Can some of the University; Jennifer G. Duncan, MD,
the applicants (because of the risk for signaling methodologies (in which Washington University School of Medi-
inequity among applicants and applicants are given a limited number cine; Molly Rose Elkins-Ryan, Rush Uni-
programs during such a visit). of signals that they can send to versity; Rebekah Fenton, MD, Ann &
certain programs to indicate a high Robert H. Lurie Children’s Hospital of
An important finding from this work level of interest in the program), as Chicago; Hayley Gans, MD, Stanford
was the value of advocacy from the used in otolaryngology residency School of Medicine; Nami Jhaveri MD,
student-led, grassroots group programs,24 be a solution? Can MPH, Kaiser Permanente Northern Cali-
FuturePedsRes. Navigating the myriad programs enter their match list before fornia; Carrie M. Johnson, MBA, Stanford
challenges related to moving to a applicants, allowing time for School of Medicine; Abigail Keogh, MD,
virtual interviewing season was applicants to experience in-person Brown University; Jennifer C. Kessel-
supported by the innovative visits to a select number of programs heim, MD, Med, Dana-Farber/Boston
FuturePedsRes organization and their before entering their own match lists? Children's Cancer and Blood Disorders
use of social media to develop a more Center, Harvard Medical School; Rachel
effective communication strategy for Finally, do residency and fellowship E. Korus MD, Boston Children’s Hospital
pediatric applicants. They were also programs need to function in the and Harvard Medical School; Alexandra
able to support our efforts to same way? What special Mientus, MD, University of Louisville;
recommend adoption of the Apply accommodations might be made for Amanda D. Osta, MD, Loyola University
Smart Campaign, and their advocacy fellowship applicants, for whom a Medical Center; Kris Rooney, MD,
efforts may play a role in assisting our niche program or the connection Lehigh Valley Reilly Children’s Hospital;
pediatric organizations in many with specific mentors might be more Sandra M. Sanguino, MD, MPH, North-
domains of the medical education important than for applicants of a western University Feinberg School of
process in the future. residency program? Is a modified Medicine; Danielle Shin, MD, PhD, Stan-
process needed for smaller ford School of Medicine; Adrienne
There are many questions to be Smallwood, MD, Baylor School of Medi-
programs or those in rural areas to
answered to make recommendations cine; Meghan Stawitcke, BA, Stanford
attract applicants? How do
for the future, beyond the current School of Medicine; Alicia Williams, MD,
applicants who are couples
pandemic.23 First, what is the impact Ann & Robert H. Lurie Children’s Hospi-
matching in different specialties
of not having applicants and tal of Chicago; Xavier Williams, MD, Uni-
manage the differences in interview
programs meet in person? Will more versity of North Carolina
processes? And can modifications
residents and fellows change
reduce the impact of the interview
programs or is there enough
season on the environment?
information in virtual interviews to
ABBREVIATIONS
ensure a good match? Can programs
The COVID-19 pandemic has created AAMC: Association of American
improve their own virtual
an opportunity to reexamine the entire Medical Colleges
interviewing skills to glean the
recruitment process for residency and AMSPDC: Association of Medical
information necessary to mitigate
fellowship training. It has also School Pediatric
possible mismatches? Can programs
highlighted inequities in the traditional Department Chairs
create virtual events to share their
system and forces educators to refine, APPD: Association of Pediatric
program culture in a way that allows
or even redefine, the process so it is Program Directors
applicants to get a feel for their future
effective and equitable for both for COMSEP: Council on Medical
program? What is the impact of
applicants and programs. Student Education in
virtual-only interviews on the
geographic distribution of trainees? Pediatrics
PEDIATRICS RECRUITMENT STUDY CoPS: Council of Pediatric
TEAM Subspecialties
Second, are there ways to improve
equity during the interview process? Melissa Bales, C-TAGME, Indiana Uni- COVID-19: coronavirus disease
What are the best practices for versity School of Medicine; Katherine M. 2019
decreasing bias? Can the interview Bartoletta, MD, MS, University of Wash- DO: doctor of osteopathy
season evolve to allow for reduced ington, Seattle Children’s Hospital; IMG: international medical
financial expense for applicants and Debra Boyer, MD, MHPE, Boston Child- graduate
programs? Can we find a way to ren’s Hospital; April O. Buchanan, MD, MD: medical doctor
benefit applicants while reducing the University of South Carolina School of
number of interviews that each Medicine Greenville/Prisma Health;
Downloaded from www.aappublications.org/news by guest on November 4, 2021
8 FROHNA et alREFERENCES Available at: https://students-residents. 17. Nilsen K, Callaway P, Phillips JP, Walling A.
1. Coalition for Physician Accountability’s aamc.org/apply-smart-residency. How much do family medicine residency
Work Group on Medical Students in the Accessed May 11, 2021 programs spend on resident recruit-
Class of 2021 Moving Across Institutions 10. Blankenburg R, Poitevien P, Gonzalez Del ment? A CERA study. Fam Med.
for Post Graduate Training. Final report Rey J, Degnon L; Virtual Cafe Study Team. 2019;51(5):405–412
and recommendations for medical edu- Virtual cafes: an innovative way for rap- 18. Donahue LM, Morgan HK, Peterson WJ,
cation institutions of LCME-accredited, idly disseminating educational best prac- Williams JA. The carbon footprint of resi-
U.S. osteopathic, and non-U.S. medical tices and building community during dency interview travel. J Grad Med Educ.
school applicants. Available at: https:// COVID-19. Acad Pediatr. 2021;13(1):89–94
www.aamc.org/system/files/2020-05/ 2020;20(6):756–757
covid19_Final_Recommendations_ 19. Garden E, Loebel E, Sanky C, Chudow J,
05112020.pdf. Accessed June 30, 2021 11. Association of Pediatric Program Direc- Fallar R, Parkas V. Key insights from the
tors. Recruitment action team resource development and implementation of a
2. National Resident Matching Program. toolbox. Available at: https://apps.appd. novel virtual interview process for medi-
Advance data tables. 2021 main resi-
org/shareWarehouse/index.cfm?page= cal school admissions during the COVID-
dency Match. Available at: https://www.
recruitmentToolbox, Accessed July 22, 19 pandemic [published online ahead of
nrmp.org/main-residency-match-data/.
2021 print May 4, 20201]. Acad Med.
Accessed May 11, 2021
12. Lee J, Alferi M, Patel T, Lee L. Choosing doi:https://doi.org/10.1097/
3. National Resident Matching Program. ACM.0000000000004149
family medicine residency programs:
NRMP Results and Data Specialties
Matching Service, 2021 Appointment what factors infuence residents’ deci- 20. Association of American Medical Col-
Year. Washington, DC: National Resident sions? Can Fam Physician. leges. Electronic Residency Application
Matching Program; 2021 2011;57(3):e113–e121 Service Dataset. Accessed May 25, 2021
4. Park J, Rhim HC. Consequences of coro- 13. Love JN, Howell JM, Hegarty CB, et al. 21. O’Malley CW, Bergin C, Shinar B. Virtual
navirus disease 2019 on international Factors that influence medical student Residency Interviewing Strategies. SPLIT
medical graduates and students applying selection of an emergency medicine resi- Recruitment: a novel approach to resi-
to residencies in the United States. dency program: implications for training dency interviews during COVID-19 and
Korean J Med Educ. 2020;32(2):91–95 programs. Acad Emerg Med. beyond. 2020; https://youtu.be/
2012;19(4):455–460 yDMaYY9_r5g. Accessed May 11, 2021
5. O’Connell RL, Kemp MT, Alam HB. The
potential impact of COVID-19 on the medi- 14. Coalition for Physician Accountability. Ini- 22. Association of Pediatric Program Direc-
cal school application. J Med Educ Curric tial summary report and preliminary rec- tors, Council of Medical Student Educa-
Dev. 2020;7:2382120520940666 ommendations of the Undergraduate tion in Pediatrics, American Medical
6. Nwora C, Allred DB, Verduzco-Gutierrez M. Medical Education to Graduate Medical School Pediatric Department Chairs,
Mitigating bias in virtual interviews for Education Review Committee (UGRC). FuturePedsRes, NextGenPediatricians. Let-
applicants who are underrepresented in Available at: https://physicianacc ter to our pediatrics community about
medicine. J Natl Med Assoc. ountability.org/wp-content/uploads/2021/ the residency recruitment process. Avail-
2021;113(1):74–76 04/UGRC-Initial-Summary-Report- able at: https://higherlogicdownload.s3.
and-Preliminary-Recommendations- amazonaws.com/APPD/
7. Flores LE, Muir R, Padilla KE, Salinas M,
1.pdf. Accessed May 11, 2021 b07d8237-43e8-4219-842d-b292e9b24d4e/
Silver EM, Lorello GR. Three (more) tips
to reduce socioeconomic bias in virtual 15. Association of American Medical Col- UploadedImages/2021_05_19_-_
anesthesiology interviews. Can J Anaesth. leges. The cost of interviewing for Pediatrics_Residency_Recruitment_
2021;68(8):1296–1297 medical residency. Available at: Recommendations_Final.pdf. Accessed
https://students-residents.aamc.org/ July 13, 2021
8. Marbin J, Hutchinson YV, Schaeffer S.
Avoiding the virtual pitfall: identifying and financial-aid-resources/cost- 23. Hammoud MM, Standiford TC, Carmody
mitigating biases in graduate medical interviewing-residency. Accessed May JB. The 2020-2021 residency application
education videoconference interviews 11, 2021 cycle: lessons learned and lingering
[published online ahead of print January 16. Brummond A, Sefcik S, Halvorsen AJ, problems. JAMA. 2021;325(22):2249–2250
12, 2021]. Acad Med. doi:10.1097/ et al. Resident recruitment costs: a 24. Chang CWD, Pletcher SD, Thorne MC, Mal-
ACM.0000000000003914 national survey of internal medicine pro- ekzadeh S. Preference signaling for the
9. Association of American Medical Col- gram directors. Am J Med. otolaryngology interview market. Laryn-
leges. Apply Smart for residency. 2013;126(7):646–653 goscope. 2021;131(3):E744–E745
PEDIATRICS Volume 148, number 4, Downloaded
October 2021from www.aappublications.org/news by guest on November 4, 2021 9National Pediatric Experience With Virtual Interviews: Lessons Learned and
Future Recommendations
John G. Frohna, Linda A. Waggoner-Fountain, Jill Edwards, Jill J. Fussell, Beth
Wueste, Joseph Gigante, Robert J. Vinci, Nicholas M. Heitkamp, Mekala K.
Neelakantan, Laura E. Degnon and Rebecca L. Blankenburg
Pediatrics 2021;148;
DOI: 10.1542/peds.2021-052904 originally published online July 30, 2021;
Updated Information & including high resolution figures, can be found at:
Services http://pediatrics.aappublications.org/content/148/4/e2021052904
References This article cites 12 articles, 1 of which you can access for free at:
http://pediatrics.aappublications.org/content/148/4/e2021052904#BI
BL
Subspecialty Collections This article, along with others on similar topics, appears in the
following collection(s):
Medical Education
http://www.aappublications.org/cgi/collection/medical_education_su
b
Permissions & Licensing Information about reproducing this article in parts (figures, tables) or
in its entirety can be found online at:
http://www.aappublications.org/site/misc/Permissions.xhtml
Reprints Information about ordering reprints can be found online:
http://www.aappublications.org/site/misc/reprints.xhtml
Downloaded from www.aappublications.org/news by guest on November 4, 2021National Pediatric Experience With Virtual Interviews: Lessons Learned and
Future Recommendations
John G. Frohna, Linda A. Waggoner-Fountain, Jill Edwards, Jill J. Fussell, Beth
Wueste, Joseph Gigante, Robert J. Vinci, Nicholas M. Heitkamp, Mekala K.
Neelakantan, Laura E. Degnon and Rebecca L. Blankenburg
Pediatrics 2021;148;
DOI: 10.1542/peds.2021-052904 originally published online July 30, 2021;
The online version of this article, along with updated information and services, is
located on the World Wide Web at:
http://pediatrics.aappublications.org/content/148/4/e2021052904
Pediatrics is the official journal of the American Academy of Pediatrics. A monthly publication, it
has been published continuously since 1948. Pediatrics is owned, published, and trademarked by
the American Academy of Pediatrics, 345 Park Avenue, Itasca, Illinois, 60143. Copyright © 2021
by the American Academy of Pediatrics. All rights reserved. Print ISSN: 1073-0397.
Downloaded from www.aappublications.org/news by guest on November 4, 2021You can also read
