CTSA Program Webinar April 28, 2021 - clic-ctsa.org
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
CTSA Program Webinar
April 28, 2021
The University of Rochester Center for Leading Innovation and Collaboration (CLIC) is the coordinating center for
the Clinical and Translational Science Awards (CTSA) Program, funded by the National Center for Advancing
Translational Sciences (NCATS) at the National Institutes of Health (NIH), Grant U24TR002260.
TIME TOPIC PRESENTER(S)
Michael G Kurilla, MD, PhD
2:00 pm ET Welcome Director, Division of Clinical Innovation
Michael G Kurilla, MD, PhD
Director, Division of Clinical Innovation
Joni L. Rutter, PhD
2:00 pm - 2:15 pm NCATS & CTSA Program Updates Acting Director, NCATS
Clare K. Schmitt, PhD
Acting Deputy Director, NCATS
Boyd Knosp, MS
2:15 pm – 2:25 pm University of Iowa
Enterprise Data Warehouse for Research - Phase 2 Thomas R. Campion Jr., PhD
Weill Medical College of Cornell University
Inclusion of Older Adults as a Model for Special Karen Bandeen Roche, PhD
2:25 pm - 2:35 pm Johns Hopkins University
Populations
Heidi Hanson, PhD, MS
University of Utah
2:35 pm – 2:45 pm Life Course Visual Toolkit Development Shari Barkin, MD, MSHS
Vanderbilt Medical Center
Hongfang Liu, PhD
Text Analytics Toward Semantic Interoperability and Data Mayo Clinic
2:45 pm – 2:55 pm Peter Elkin, MD, MACP, FACMI, FNYAM
Sharing
State University of New York at Buffalo
Martin Zand, MD, PhD
2:55 pm – 3:00 pm CLIC Updates MPI, CLIC
3:00 pm ET Adjourn 2
clic-ctsa.org
NCATS/CTSA Program Updates
Michael Kurilla, MD-PhD
Director, Division of Clinical Innovation
NCATS
April 28, 2021
A few words from
Dr. Joni Rutter
NCATS Acting Director
and
CTSA Program
Steering Committee Co-Chair
A few words from Dr. Clare Schmitt NCATS Acting Deputy Director
Budget: FY 2021
➢ Consolidated Appropriations Act, 2021
➢ P.L. 116-260, 12/27/2020
➢ Contains all 12 bills funding the federal gov’t, plus coronavirus
supplemental funding Dollars in millions
FY 2019 FY 2020 FY 2021
Final Final Enacted
NCATS Approp. 806.4 832.9 855.4
CTSA 557.8 578.1 586.8
CAN 44.5 49.1 "up to 60.0"
Budget: FY 2022
➢ President’s FY 2022 Discretionary Request – released 4/9/21
➢ Overview: https://www.whitehouse.gov/omb/FY-2022-Discretionary-Request/
➢ NIH – Request for $51B ($9B increase over FY21)
➢ No details on NIH Institute/Center funding requests
➢ Detailed request coming in next few months
➢Congressional Hearings
➢ House Appropriations Subcommittee Hearing for HHS – 4/15/21
➢ HHS Sec. Xavier Becerra was the witness
➢ No hearings for NIH scheduled yet
CTSA Program Info & Reminders NIH Policies & Updates • NOT-TR-20-036: Notice of Change to Key Dates for PAR-18-940 (CTSA U54). August 15, 2021 receipt date has been moved to July 15, 2021. • NOT-OD-21-073: Upcoming Changes to the Biographical Sketch and Other Support Format Page for Due Dates on or after May 25, 2021 • Non-Compliant Publications in RPPRs: Publications not compliant with the NIH Public Access Policy will cause a delay in review and processing of the applicable Notice of Grant Award. • NOT-TR-21-020: Key Date Change PAR-19-337 Competitive Revision Awards for the CTSA Program. Early expiration: the final September 27, 2021 due date is eliminated (there are no remaining receipt dates). POC: Pablo Cure, MD, MPH • NCATS Advisory Council (June 10-11, 2021) Several CTSA concepts will be presented: https://ncats.nih.gov/advisory/council/meetings Funding Opportunity Announcements • PAR-21-203 and NOT-TR-21-025 Clinical and Translational Science Award (CTSA) Consortium-Wide Centers: Resources for Rapid Demonstration and Dissemination (C3-R2D2) Due Date June 21, 2021 • Notice of Intent to Publish Research Opportunity Announcements for the NIH Post-Acute Sequelae of SARS-CoV-2 Infection Initiative: PASC Data Repositories and Mobile Health Platform
RADx Underserved Populations (RADx-UP)
• RFA-OD-21-009 – Emergency Award: RADx-UP - Social, Ethical, and
Behavioral Implications (SEBI) Research on Disparities in COVID-19 Testing among
Underserved and Vulnerable Populations (U01 Clinical Trials Optional)
• RFA-OD-21-008 – Emergency Awards: Community-engaged COVID-19
Testing Interventions among Underserved and Vulnerable Populations- RADx-UP
Phase II (U01, Clinical Trial Optional)
• NOT-OD-21-103 – Notice of Special Interest (NOSI): Emergency Competitive
Revisions for NIH Grants to Add or Expand Community-engaged COVID-19 Testing
Interventions among Underserved and Vulnerable Populations – RADx-UP Phase II
(Emergency Supplement- Clinical Trial Optional)
• NOT-OD-21-101- Notice of Special Interest (NOSI): Administrative Supplements for
Rapid Acceleration of Diagnostics-Underserved Populations (RADx-UP) Phase I
Projects to Address Vaccine Hesitancy and Uptake
• NOT-OD-21-097 – Notice of Intent to Publish a Research Opportunity Announcement
for RADx-UP Return to School Diagnostic Testing Approaches (OT2 Clinical Trial
Optional)
References:
• https://www.nih.gov/research-training/medical-research-initiatives/radx
• https://grants.nih.gov/grants/guide/COVID-Related.cfm
ACTIV-6
Re-Purposed Drugs Mild to Moderate COVID-19
X
Prevent Hospitalization
Clinical Coordinating Center Data Coordinating Center Site SupportUpcoming: •QA/QC Meeting: •Date: May 7, 2021 •Registration Link •CTSA Program Annual Meeting •Date: December 1-3, 2021
Working Group Reports
Enterprise Data Warehouse for Research - Phase 2
Boyd Knosp, MS (University of Iowa) and Thomas R. Campion Jr., PhD (Weill Medical College of Cornell University)
Inclusion of Older Adults as a Model for Special Populations
Karen Bandeen Roche, PhD (Johns Hopkins University)
Life Course Visual Toolkit Development
Heidi Hanson, PhD, MS (University of Utah) and Shari Barkin, MD, MSHS (Vanderbilt Medical Center)
Text Analytics Toward Semantic Interoperability and Data Sharing
Hongfang Liu, PhD (Mayo Clinic) and Peter Elkin, MD, MACP, FACMI, FNYAM (State University of New York at Buffalo)
12
clic-ctsa.orgEnterprise Data Warehouse for Research (EDW4R) Working Group CTSA Program Webinar April 28, 2021 Boyd Knosp, MS Thomas R. Campion, Jr., Ph.D. Institute for Clinical and Translational Science
Goals and outcomes
• Goals
• Increase understanding of enterprise data warehouse for research
(EDW4R) activities at CTSA hubs
• Identify best practices to support clinical and translational researchers
with electronic patient data
• Outcomes
• Novel studies describing EDW4R current state and next steps
• References for EDW4R activities
• Engagement across the CTSA community
Institute for Clinical and Translational ScienceEnterprise Data Warehouse for Research (EDW4R)
Data Sources Transform Repositories Applications Uses
Patient
Data
Study
Self- feasibility
EHR service
Databases
Normalize Pop.
health
Researchers
External Data marts
data Basic
informatics
Filter
Registries Query Quality
improvement
tools
Internal
data
Research
networksEDW4R Working Group Timeline
• Spring 2018: initial discussion at iEC (iDTF) meeting
• CY 2019: Phase 1
• CY 2020: Phase 2
• Spring 2021: planning for Phase 3
Institute for Clinical and Translational ScienceEDW4R Phase 1: Study of CTSA Hubs
• Focus areas
• Architecture
• Access and engagement
• Service management
• Maturity
• Qualitative methods
• 20 semi-structured interviews with CTSA informatics leaders
• Directed content analysis
Institute for Clinical and Translational ScienceEDW4R Phase 1: Academic Output
Understanding enterprise data warehouses to support clinical and
translational research
Thomas R Campion, Jr, Catherine K Craven, David A Dorr, Boyd M Knosp
Journal of the American Medical Informatics Association,
ocaa089, https://doi.org/10.1093/jamia/ocaa089
Published:
17 July 2020
Institute for Clinical and Translational ScienceEDW4R Phase 2: Follow-up Study
• Focus areas (identified in Phase 1 for future research)
• Enterprise IT relationship
• Data governance
• Workforce
• Cloud computing
• Qualitative methods
• Maturity index pilot: https://redcap.link/nfu8wswl
Institute for Clinical and Translational ScienceEDW4R Phase 2: Community Discussions • April: COVID-19 support from EDW4R • May: Cloud with NIH STRIDES project leaders • June: Data governance part 1 • July: Data governance part 2 (with Adam Wilcox of CD2H) • August: Work force & JAMIA publication • September: Relationship with enterprise IT • October: Maturity model • November: UT Health honest broker approach with Elmer Bernstam • December: Stanford cloud computing approach with Michael Hallas • February 2021: Phase 2 analysis discussion & maturity model pilot kick off Institute for Clinical and Translational Science
EDW4R Phase 2: Deliverables
• Conference abstracts in 2020 (cancelled)
• AAMC GIR
• AMIA Summits
• Participation in CD2H maturity model project
• AMIA Informatics Summit 2021
• Panel on EDW4R operations and trends
• Poster on EDW4R maturity
• Translational Science 2021
• Panel on maturity models
• AMIA Annual Symposium 2021
• Podium abstract on EDW4R phase 2 (under review)
Institute for Clinical and Translational ScienceEDW4R: JAMIA Special Focus Issue
Call for Papers for Special Focus Issue
Best Practices in Research Patient Data Repositories
Guest Editor-in-Chief:
Leslie A. Lenert, Medical University of South Carolina Submissions due
Focus Issue Guest Editors:
Shawn N. Murphy, Mass General Brigham
June 15, 2021!
Michael J. Becich, University of Pittsburgh School of Medicine
Thomas R Campion, Weill Cornell Medical College
Boyd M. Knosp, University of Iowa
Genevieve Melton-Meaux, University of Minnesota
Shyam Visweswaran, University of Pittsburgh School of MedicineBest Practices in Research Patient Data
Repositories (RPDRs) –Topics of Interest
• Gathering leadership support and funding for • Novel Academic Industry Partnerships for
the RPDRs RPDRs
• Showing a return on investment for the RPDRs • Data governance (all flavors)
• Novel architectures (including cloud • Workforce for the RPDRs
environments) for RPDRs
• The role of enterprise IT and/or research
• Design of “ideal” next-generation RPDRs faculty in RPDR operations; intersection of
RPDR with academic informatics; intersection
• RDW assessment tools such as maturity of RPDR with quality of care
models and data quality processes
• Organization, data structures and ontologies,
• Lessons learned from implementation of tools to access, for RPDRs
RPDRs
• The role of RPDRs in learning health systems
• Relationship of RPDRs to Enterprise Data
Warehouses and Clinical Data Warehouses • Security and compliance practices for RPDRs
• Innovative methods for supporting RPDRs • RPDR literacy for researchers
• Sustainability models for supporting RPDRsEDW4R Phase 3: Planning
• Step back consulting
• Best practice products
• Questions PIs should ask about enterprise IT
• Seven things to know about EDW4R and cloud computing
• Maturity index
• Case studies
• Natural language processing and EDW4R
• Sustainability approaches
Institute for Clinical and Translational ScienceQuestions and comments
• EDW4R Phase 3 interest: CLIC coordinators/website
• Boyd Knosp: boyd-knosp@uiowa.edu
• Tom Campion: thc2015@med.cornell.edu
Institute for Clinical and Translational ScienceInclusion of Older Adults as a Model
for Special Populations
Karen Bandeen-Roche
Frank Hurley and Catharine Dorrier Professor and Chair
Department of Biostatistics
On Behalf of the Workgroup
CTSA Program Meeting
April 28, 2021Goals and Outcomes
• Promote the integration of special populations in
across the human lifespan
• Train and cultivate the translational science workforce
• Innovate processes to increase the quality and
efficiency of translational research
• Implementation of a process or best practice
• Knowledge dissemination relevant to CTSA goals
• Increased collaboration
© 2014, Johns Hopkins University. All rights reserved.Workgroup Members
• Karen B-R • Marco Pahor, UFL
• Cynthia Boyd, JHU • Todd Manini, UFL
• Elizabeth Eckstrom, OHSU • Steve Anton, UFL
• Steve Kritchevsky, WFU • Daniel Mullins, UMD
• Susan Stark, WUSL • Wendy Kohrt, Colorado
• Jay Magaziner, UMD • Kady Nearing, Colorado
• Elena Volpi, UTMB • Jerry Gurwitz, Meyers Inst.
• Mark Supiano, Utah
Enterprise Committee Affiliation: LIFESPAN
© 2014, Johns Hopkins University. All rights reserved.Why is the work needed?
Policy demands it
“Applications and proposals involving human subjects research must
address plans for including individuals across the lifespan in
the PHS Human Subjects and Clinical Trial Information Form. Any age-
related exclusions must include a rationale and justification based on a
scientific or ethical basis.” (NIH Policy on Inclusion across the lifespan effective
January 25, 2019)
“For most cancers, clinical trials should include a representative
population of older adults. Older adults, including those with frailty,
should be enrolled in all phases of clinical trials, when they can be safely
and ethically enrolled.” (FDA Draft Guidance or Industry Sponsored Trials in
Oncology, March 2020).
© 2014, Johns Hopkins University. All rights reserved.Barriers to Representation of Older Adults in Cardiovascular Disease
Trials Before and After the Inclusion Across the Lifespan Policy
Nanna MG. JAMA Intern Med. 2020 3
© 2014, Johns Hopkins University. All rights reserved.
0Inclusion of Older Adults . . .
• Meets the Needs of a Changing Nation
• Satisfies Our Desire for Justice
• Is Consistent with Our Scientific Values
• Has Practical Benefits
• Is Consistent with the Flow of Policy
clic-ctsa.orgWorkgroup Goals and Deliverables
1. Conference on Older Adult Inclusion
> Identify knowledge / practice gaps
© 2014, Johns Hopkins University. All rights reserved.Research Centers Coordinating Network /
Workgroup Workshop: February 22-23, 2021
© 2014, Johns Hopkins University. All rights reserved.Research Centers Coordinating Network /
Workgroup Workshop: February 22-23, 2021
• 15 speakers spanning 14 institutions
• Sessions
• Current Situation: In subspecialties
• Barriers
• Making the case
• Next steps: Implementation!
• Follow ups: Implementation roundtable, grants
Website: https://www.afar.org/events/rccn-event-
inclusion-of-older-adults-in-clinical-research
© 2014, Johns Hopkins University. All rights reserved.Workgroup Goals and Deliverables
1. Conference on Older Adult Inclusion
➢ Identify knowledge / practice gaps
2. Toolkit: Modular Presentation Materials
➢ Workshops; Advocacy; Knowledge
➢ Reference repository
© 2014, Johns Hopkins University. All rights reserved.Modules
Why include older adults (You should!)
Practical strategies
Consenting/Inclusion considering cognitive impairment
Multimorbidity
Polypharmacy
Design and data analysis strategies
Focusing protocols on geriatric outcomes
Community engagement
Cultural considerations / diverse inclusion
Institutional socialization
© 2014, Johns Hopkins University. All rights reserved.Workgroup Goals and Deliverables
1. Conference on Older Adult Inclusion
➢ Identify knowledge / practice gaps
2. Toolkit: Modular Presentation Materials
➢ Workshops; Advocacy; Knowledge
➢ Reference repository
3. Pilot Workshops
➢ ACTS; SGIM; Subspecialty; AGS
© 2014, Johns Hopkins University. All rights reserved.You Should be Recruiting
Older Adults
Stephen B. Kritchevsky, PhD
Wake Forest School of Medicine &
Sticht Center for Healthy Aging and Alzheimer’s Prevention
clic-ctsa.orgWorkgroup Goals and Deliverables
1. Conference on Older Adult Inclusion
➢ Identify knowledge / practice gaps
2. Toolkit: Modular Presentation Materials
➢ Workshops; Advocacy; Knowledge
➢ Reference repository
3. Pilot Workshops
➢ ACTS; SGIM; Subspecialty; AGS
4. Inclusion Tracking Data Template
5. Lessons for other special populations
© 2014, Johns Hopkins University. All rights reserved.How you can get involved
• Join us!
• Karen Bandeen-Roche - kbandee1@jhu.edu
• Advocate for older adult inclusion
• Modular materials to be available soon
• CTSA appeal: RFAs, further institutional and
implementation resources
• Next steps
• Finalize, polish slide set
• Adjunct materials
• 1-2 more workshops
• Tracking data template, lessons beyond older age
© 2014, Johns Hopkins University. All rights reserved.Life Course Research Visual Toolkit: Why and What Heidi Hanson, PhD, MS Assistant Professor of Surgery University of Utah Shari Barkin, MD, MSHS Professor of Pediatrics Vanderbilt University Medical Center
Developmental Origins of Health and Disease: A Lifecourse approach to the prevention of non- communicable disease. Baird J et al. Healthcare, 2017.
Traditional approaches to science • Based on a Newtonian paradigm • Linear cause and effect
Past Approaches: Simplify
Problem: It ignores the
interdependence of the
factors. We are
understanding health and
disease in a vacuum.COMPLEXITY SCIENCE 4/28/2021 45
46
Hanson, HA., Leiser, C., Bandoli, G., Pollock, B., Karagas, M., Armstrong, D., . . . Barkin, S. (2021). Charting the life course: Emerging opportunities to advance scientific approaches using life course research. Journal of Clinical and Translational Science, 5(1), E9. doi:10.1017/cts.2020.492
Introducing……
• 6 full webinars with more than 15 national and international experts
• 32 number of shorter educational skills building videos that include
content and, importantly, methods for how to implement this
research
48•An Introduction to Life Course Research and Complexity Science •The Promise of the Exposome: Scaling it Up
•All of Us Research Program •Understanding Health Across the Lifecourse
•Journey into Complexity Science: The Promise of the Exposome •An Overview of Longitudinal Trajectory Methods
•The Importance of Measuring Social and Chemical Stressors
•Using Electronic health Records for Medical Research • New Methods for Identifying Complex Patterns of Disease in
•Discovery of a Drug-Drug Interaction Families and Linking them to their Etiological Roots
•A How-to Guide for Studying the Elusive Exposome in Complex • Assessing Multiple Exposures Across Time
Disease with Large Data • Machine Learning to Classify Subgroups of Disease and
•Dissecting P=G+E with Big Data Predict Outcomes
•Applying Data Science Techniques to Identify Phenotypic • Developing and Validating Ways to Model High-Dimensional
Variation Data
•Understanding Potential Biases Inherent in Data
•Linkage Across Diverse Data Resources
•Project Viva: Challenges and Lessons Learned
•Utilizing Data Science Resources to Prepare and Package
•Research Opportunities with a Single Cohort and Beyond
Integrated Datasets
•How to Build a Longitudinal Cohort
•Using Trajectory Methods to Identify Sensitive Periods During
•Using Microsimulations to conduct Virtual Experiments Pregnancy
•How to Implement a Microsimulation Model: Application • Using Trajectory Methods to Capture Timing, Dose, and
Duration of Exposure
•Evaluating Interventions with a Complex Systems Approach •Using Trajectory Methods to Identify Sensitive Periods During
•Moving Towards a Complex Systems Approach to Health Pregnancy
Intervention Research • Using Trajectory Methods to Capture Timing, Dose, and
Duration of Exposure4/28/2021 50
Dissemination Strategy
Target audience: Learners at all stages interested in
understanding the importance of lifecourse research,
the potential applications, and emerging methods to
conduct this type of research
Current dissemination strategy:
• CLIC website and blog
• Highlight on CTSI’s webpages
• Distribute to academic societies reaching across the
lifespan
• Send to the leaders of medical schools and schools
of public health
51Working group members
Heidi A. Hanson,Shari Barkin, MD, Frederick Kaskel, Joemy Ramsay, Nate O’Neil, MC
PhD, MS MSHS MD, PhD PhD, MS University of Utah
University of UtahVanderbilt Medical Albert Einstein University of Utah
Center School of Medicine
Margaret R. Karagas Dartmouth College
Jonathan N. Tobin Albert Einstein College of Medicine
Carrie Dykes University of Rochester
Maureen Monaghan Children's National
Peter Szilagyi University of California, Los Angeles
Christopher Seplaki University of Rochester
Elizabeth Eckstrom Oregon Health and Science University
Mark Schleiss University of Minnesota Twin Cities
Gretchen Bandoli University of California San Diego
Bradley Pollock University Of California Davis
Anne Hoen Dartmouth College4/28/2021 53
WG: Text analytics towards semantic
interoperability and data sharing for
the research use of EHR
Progress Report to iEC Lead Team
Peter Elkin, University at Buffalo
Hongfang Liu, Mayo Clinic
Justin Guinney, Sage BioNetworks
clic-ctsa.orgText Analytics WG Goals
• Goal 1: Containerize open source text analytics with
instructions for their integration into local applications
• Goal 2: Contribute to the community-wide effort on data sharing
through collaboration with CD2H and TIN
• Goal 3: Create a community share task to advance text analytics
for concept encoding and de-identification
• Goal 4: Load de-identification and text analytics applications to
the NCATS cloud
clic-ctsa.orgText Analytics WG Deliverables Year 1
• NLP Cloud Sandbox Pilot de-identification of narratives (led by
Bradley W. Taylor & Russ Waitman)
• OHNLP Artifact Discovery and Preparation Toolkit, NLP-ADAPT (led
by Serguei Pakhomov) (github.com/nlpie/nlp-adapt-kube) – see Best
Practice Presentation at Monday, September 16, 2019.
• Ongoing Text Analytics WG and CD2H collaboration for N3C NLP
Effort
• An End-to-End Sample Implementation of NLP for Multi-site Cohort
Studies
• Catalog best practice and review articles
clic-ctsa.orgText Analytics WG Deliverables Year 2
• Text Analytics WG and CD2H collaboration for N3C NLP Effort and
NLP Sandbox – in production
• A text analytics infrastructure and
• A community-wide federated NLP evaluation solution
• Codification of Clinical Problems in SNOMED CT, Lab test result
names in LOINC and Medications in RxNorm
• Catalog best practice and review articles – in progress
clic-ctsa.orgU01TR002062
Open Health
Natural Language
Processing
Clinical and Translational Science Collaboratory
Institute (University of Minnesota)
Center for Clinical and Translational
Science (Mayo Clinic)
Irving Institute for Clinical and
Translational Research (Columbia
University)
Partnership
Center for Clinical and Translational
Team
Sciences (UT Health)
University at Buffalo
Aim 1: Obtain NLP NLP
Aim 3: Develop privacy-
artifacts across sites Innovations preserving computational
for improving NLP phenotyping enhanced
algorithm development with NLP
Aim 2: Generate a Aim 4: Partner with
synthetic text corpus diverse communities for
for exploratory analysis Privacy-
translation science
Computational
of clinical narratives Preserving
Phenotyping excellence leveraging
Computing
EHR
clic-ctsa.orgNational COVID Cohort Collaborative
https://github.com/OHNLP/N3C-NLP-Documentation/wiki
clic-ctsa.orgView N3C NLP as a Real-world AI Project
• Behave as it • Can provide
should be evidences of
the decision
Health Explainability
Security Reproducibility
• Can tolerate
attacks, i.e., • All predictions
efforts to must be
change or reproducible
manipulate its
behavior
Reproducibility Explainability Implementability Interoperability Transparency
clic-ctsa.orgHistory of Technology Stack of MVP
N3C NLP
● History
○ 2009 - BioTagger GM [1] and 2011 - Concept extraction [2]
■ Dictionary Lookup and Machine Learning (Weak Supervision)
○ 2013 - Refactor to UIMA and develop OHNLP common type systems for facilitating
interoperability
○ 2013 - MedTaggerIE [3], clinical researchers want explainable NLP (Human-AI Trust)
○ 2016 - Big Data NLP Infrastructure [5]
○ 2019 - MedTagger Suite to Enterprise Implementation - NLPaaS [6]
1. Torii M, Hu Z, Wu CH, Liu H. BioTagger-GM: a gene/protein name recognition system. J Am Med Inform Assoc. 2009;16(2):247-255. doi:10.1197/jamia.M2844
2. Torii M, Wagholikar K, Liu H. Using machine learning for concept extraction on clinical documents from multiple data sources. J Am Med Inform Assoc
. Sep-Oct 2011;18(5):580-7. doi: 10.1136/amiajnl-2011-000155. Epub 2011 Jun 27.
1. Wu S, Kaggal C, Dligach D, et al A common type system for clinical natural language processing. J Biomed Semantics . 2013 Jan 3;4(1):1. doi: 10.1186/2041-1480-
4-1.
2. Liu H, Bielinski SJ, Sohn S, et al. An information extraction framework for cohort identification using electronic health records. AMIA Jt Summits Transl Sci Proc.
2013;2013:149-153. Published 2013 Mar 18.
3. Kaggal V, Komandur R, Mehrabi S, et al.Toward a Learning Health-care System - Knowledge Delivery at the Point of Care Empowered by Big Data and NLP
Biomed Inform Insights . 2016 Jun 23;8(Suppl 1):13-22. doi: 10.4137/BII.S37977. eCollection 2016.
4. Wen A, Fu S, Moon S, et al. Desiderata for delivering NLP to accelerate healthcare AI advancement and a Mayo Clinic NLP-as-a-service implementation. NPJ Digit
Med. 2019;2:130. Published 2019 Dec 17. doi:10.1038/s41746-019-0208-8
clic-ctsa.orgNLP Algorithm Development Process
Not surprising to everyone, majority of the effort in clinical concept extraction application
is “Task formulation” and “Annotation guideline development”
Sunyang Fu et al. Development of Clinical Concept Extraction Applications: A Methodology
Review, Journal of Biomedical Informatics (2020): 103526.
clic-ctsa.org 62The portability of the statistical NLP system (i.e.,
Generalizability Challenges machine learning-based system) highly depends on the
distributional statistics.
The institutional, EHR, and documentation variations
across different institutions make the underlying
distribution dramatically varying.
Training a statistical NLP system in a new environment
is very expensive.
clic-ctsa.orgNLP Algorithm Implementation Environment
To innovate, domain expertise must be collected
and preserved
To facilitate, toolsets should engage and
empower domain experts
To accelerate, NLP platforms should be
responsive and scalable
clic-ctsa.orgN3C Text Analytics Backbone Infrastructure
clic-ctsa.orgN3C NLP Open Development Framework
Due to PHI concerns, NO large clinical document collections.
If we cannot share the text data with the public, can we share NLP
knowledge (lexicon, pattern)?
clic-ctsa.orgOngoing:
Incorporate
Information
Retrieval into the
infrastructure
https://medinform.jmir.org/2020/10/e17376
clic-ctsa.orgN3C NLP Community Participatory
Progress Updates
○ Mayo Clinic (Hongfang Liu)
○ Tufts Medical Center (Andrew Williams, Robert Miller)
○ University of Kentucky (Ramakanth Kavuluru, Daniel Harris)
○ University of Kansas Medical Center (Mei Liu)
○ University of Minnesota (Rui Zhang)
○ Wake Forest University (Umit Topaloglu)
○ University of Washington (Meliha Yetisgen)
○ University of Alabama at Birmingham (James Cimino, John Osborne)
○ John Hopkins University (Chris Chute, Masoud Rouhizadeh)
○ Columbia University (Karthik Natarajan)
○ University of Buffalo (Peter Elkin)
○ Stony Brook (Joel Saltz, Janos Hajagos)
○ UT Health (Hua Xu, Kirk Roberts)
○ University of North Carolina (Emily Pfaff)
○ University of New Mexico (Larissa Myaskovsky)
○ ….
clic-ctsa.orgCD2H NLPSandbox.io: Overcome data access barriers
in biomedical tool benchmarking
● Critical patient information derived from academic research, health care and clinical
trials are off limit for traditional data-to-model challenges. Existing barriers include:
○ Access to big and sensitive data
○ Lack of effective frameworks for assessing performance & generalizability
clic-ctsa.orgNLPSandbox.io: Benchmarking NLP tools on private
data
clic-ctsa.orgPHI Annotation & De-identification
first series of NLP Sandbox tasks
● The first series of tools that can be benchmarked on NLPSandbox.io target PHI Annotation & De-identification
in clinical notes.
○ Date annotation
○ Person name annotation
○ Physical address annotation
● The community can contributes by
submitting new NLP Sandbox tasks
NLP Sandbox tool specifications
Annotation Knowledge graph generation nlpsandbox/nlpsandbox-schemas (GitHub)
clic-ctsa.orgDevelopment of NLP Sandbox tools
● Robust and reliable
● Reproducible
● Re-usable
● Portable
● Cloud-friendly
nlpsandbox/date-annotator-
example
nlpsandbox/nlpsandbox-schemas
nlpsandbox/date-annotator-example-
java
clic-ctsa.org“The tool deployed in production is a tool benchmarked”
● A small change made to a tool when adapting it to a production environment can
dramatically reduce its performance. The NLP Sandbox promotes the development of tools
that are production-ready, interchangeable, and re-usable.
The performance of the
NLP Sandbox PHI
Deidentifier improves as
more performant building
blocks are submitted to
NLPSandbox.io.
phi-deidentifier.nlpsandbox.io
clic-ctsa.orgNLPSandbox.io
Join us on the NLP Sandbox Discord Server
nlpsandbox.io/discord
nlpsandbox.io
GitHub: https://github.com/nlpsandbox
Data hosting sites:
● Sage Bionetworks (2014 i2b2 de-identification challenge data)
● Medical College of Wisconsin (private data - onboarding)
● Mayo Clinic (private data - onboarding)
clic-ctsa.orgRecord Process
Collaborative Pipeline Web Server
Enterprise
Enterprise
Transcribed Record Java Bean Container
Processed Record Record to Processor
to Storage Map Text To and Return
Map Text To
Terminology and
Terminology
Store
EMR / OMOP
EMR Terminology
Terminology
Repository Server
Server
Intelligent Query
Intelligent Query
to Database
Database
Handle Query
Handle Queryand
and
Explode Matches
Explode Matches
Query to Processor
and Return
Query
Record Retrieval
Process
clic-ctsa.orgclic-ctsa.org
NLP
Component:: Compositional
Expression Generator
clic-ctsa.orgUIMA
NLP & CE
Example
clic-ctsa.orgclic-ctsa.org
"DOC_ID SEC_ID PROP_ID CONCEPT_CODE INEX_TYPE PTS" "2 52 1 23685000 7 1" "2 52 1 255302009 7 1" "2 52 1 79619009 7 1" "2 52 2 53059001 7 1" "2 52 2 237679004 7 1" "2 52 2 11092001 7 1" "2 52 3 49436004 7 1" "2 51 13 43364001 7 1" "2 51 14 258707000 7 1" "2 51 14 255260001 7 1" "2 51 14 102522009 7 1" "2 51 15 226630009 7 1" "2 51 15 51440002 7 1" "2 51 15 182334003 7 1" "2 51 15 43364001 7 1" "2 56 43 229799001 7 1" "2 56 43 clic-ctsa.org 258684004 7 1"
Observational Data are formatted for OMOP (OHDSI) and i2b2,and
PCORNet
clic-ctsa.orgclic-ctsa.org
Ensemble
Machine
Learning
Method
clic-ctsa.orgOngoing Activities
Towards network-wide text analytics infrastructure deployment
➢Pilot testing with four N3C sites, each adopts a different common data
model
➢For N3C data ingestion and harmonization, collaboration with the Standard
Operation Procedure(SOP) will be posted on the N3C Phenotype Data
Acquisition wiki: https://github.com/National-COVID-Cohort-
Collaborative/Phenotype_Data_Acquisition/wiki/NLP-Submission-Process
Bringing model to data for federated benchmarking and seamless NLP
deployment
➢Example COVID 19 rulesets will be posted on NLPSandbox
• NLPSandbox is currently running a deid sharetask
• https://nlpsandbox.io
clic-ctsa.orgOngoing: Catalog review and best practice
articles
• iEC Text Analytics WG monthly meeting: the best practice session
• Kirk Roberts from UT Health – Dec 21, 2020
• Meliha Yetisgen from University of Washington – Jan 18, 2021
• Stephane Meystre from MUSC – March 15, 2021
• Working with CD2H on a scope review related to the use of NLP for clinical
and translational research with standard operation procedures (SOPs)
clic-ctsa.orgThank you clic-ctsa.org
CLIC Updates
87
clic-ctsa.orgCLIC Career Development Community
• An interactive space available to the CTSA
Program and greater translational science
community where individuals can collaborate,
network, and share ideas with peers from
across the consortium.
• Who Can Join? Anyone within the translational
science community interested in the career
development of trainees, scholars, and CRPs.
• Questions? education@clic-ctsa.org
Join Now!
https://clic-ctsa.org/career-development-community 88
clic-ctsa.orgExploring the Inclusion of Community
Hospitals in Clinical Research
Thursday, May 27, 2021
11:00 AM – 3:00 PM ET
Registration
Now Open!
89
clic-ctsa.orgCTSA Program Working Group Application
Submission - Cycle 6
• What is required in the application?
‒ Individuals/Groups need to propose and deliver well-defined projects or deliverables that fill identified translational
gaps and/or further the CTSA Program objectives in high priority areas within clinical and translational science.
• Who can submit an application? Application Deadline:
‒ Any group or individual within the CTSA Program Tuesday, June 1, 2021
(no later than 11:59 pm EST)
• Who will receive CLIC support?
‒ Any Working Group approved by the CTSA
Program Steering Committee
• How does someone apply?
‒ Log onto the CLIC website under the Groups;
Working Group and then Working Group
Applications.
clic-ctsa.orgTransition to CM-PRISM Software Platform
• Scorecard licenses expire April 30, 2021
• Hubs wishing to continue using Scorecard
for non-Common Metrics data – contact Clear Impact
• Migration of Careers and Informatics data
• Pilot funding and IRB not supported on new platform
• “How to Use CM-PRISM” webinars
• Thursday, May 6th at 2:00 ET
• Monday, May 17th at 3:00 ET
• Visit the CMI webpage for details
• 2020 CMI data due August 31, 2021 – submitted in CM-PRISM
91
clic-ctsa.orgInsights to Inspire 2021 – The Basics
Informatics: The Journey to Interoperability
• Series of webcasts
• 12 webcasts
• 10-20 minutes in length
• Based on themes identified in 2019 TTC Plans
• Subject Matter Experts present on topics
• Dissemination
• Posted on CLIC website
• Posted on YouTube and Vimeo
• Launch Date – June 2021
• Stay tune for additional details 92
clic-ctsa.orgReminders
• The next scheduled CTSA Program Webinar:
▪ Date: May 26, 2021
▪ Time: 2:00-3:00pm Eastern Time.
• Group coordinator:
▪ ctsa_program_webinar@clic-ctsa.org
• CTSA Program Webinar meeting archives/information:
▪ https://clic-ctsa.org/groups/ctsa-program-group
93
clic-ctsa.orgCLIC/NCATS Communication Channels
Sharing Content: CLIC Website Twitter
❖ News (Consortium News & Mike’s Blog): ❖ NCATS: twitter.com/ncats_nih_gov
clic-ctsa.org/news ❖ CLIC: twitter.com/CLIC_CTSA
❖ Events: clic-ctsa.org/events ❖ Hashtag: #CTSAProgram
❖ Education & Career Development Gateway:
https://clic-ctsa.org/education-careers
▪ Education Clearinghouse: clic-ctsa.org/education CLIC Contact Us
▪ Opportunities Board: https://clic- ❖ Have a question and not sure where to direct it?
ctsa.org/opportunities-board clic-ctsa.org/contact
▪ Diamond: https://clic-ctsa.org/diamond
❖ Funding opportunities (RFAs): Newsletters
▪ Synergy Papers:
❖ CTSA Ansible: Subscribe Here
https://clic-ctsa.org/collaboration/clic-synergy-papers
❖ CLIC News Roundup: Subscribe Here
▪ Un-Meeting
https://clic-ctsa.org/collaboration/clic-un-meetings ❖ NCATS e-Newsletter: ncats.nih.gov/enews
94
clic-ctsa.orgCTSA Program Initiative Channels
ACT Network Recruitment Innovation Center (RIC)
❖ Website: http://www.actnetwork.us/National ❖ Website:
❖ Subscribe to newsletter: https://trialinnovationnetwork.org/recruitment-
https://bit.ly/2HQGsM5 innovation-center
❖ Subscribe to newsletter: https://bit.ly/2OpEDHc
IREx SMART IRB
❖ Website: https://www.irbexchange.org
❖ Website: https://smartirb.org
❖ Subscribe to newsletter: https://bit.ly/2TtQG7b
❖ Subscribe to newsletter: https://bit.ly/2JFbiK3
National Center for Data to Health Trial Innovation Network (TIN)
(CD2H) ❖ Website: https://trialinnovationnetwork.org
❖ Website: https://ctsa.ncats.nih.gov/cd2h/ ❖ Subscribe to newsletter: https://bit.ly/2TXHQDZ
❖ To Join: N3C & CD2H Login/Registration
95
clic-ctsa.orgYou can also read
