FOCUS GROUPS: A VALUABLE TOOL FOR YOUR ASSESSMENT ARSENAL
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
FOCUS GROUPS: A VALUABLE TOOL
FOR YOUR ASSESSMENT ARSENAL
Sandra S. Bowles, RN, EdD, CNE
Assistant Dean for Assessment & Special Projects
University of Charleston, School of Pharmacy
Association for Assessment of Learning in Higher Education
June 4, 2012
OUTCOMES THIS PRESENTATION IS DESIGNED TO INCREASE PARTICIPANT KNOWLEDGE OF THE: 1. VALUE OF FOCUS GROUPS FOR COURSE AND PROGRAM ASSESSMENT. 2. USE OF FOCUS GROUPS IN ASSESSMENT. 3. GUIDING PRINCIPLES IN DEVELOPING FOCUS GROUPS.
• FOCUS GROUPS CAME INTO USE IN THE 1940s AS A QUALITATIVE RESEARCH TECHNIQUE • PROVIDE OPPORTUNITY TO LOOK AT: – PERCEPTIONS – EXPERIENCES – VALUES
USED TO: GATHER DATA TEST ASSUMPTIONS VALIDATE ATTAINMENT OF OUTCOMES IDENTIFY PROGRAM STRENGTHS OR NEED FOR IMPROVEMENT GENERATE DATA FOR MARKETING
FEEDBACK TYPICALLY DEEPER, RICHER AND MORE INSIGHTFUL THAN TYPICAL RESULTS FROM TRADITIONAL SURVEYS
USEFUL IN TRIANGULATION TO SUPPORT OR AUGMENT OTHER ASSESSMENT METHODS.
LIMITATIONS REQUIRE TIME COMMITMENT SMALL SAMPLES MAY NOT BE SUITABLE FOR GENERALIZATIONS TO LARGER POPULATION REQUIRE THOUGHTFUL DEVELOPMENT OF QUESTIONS GROUP LEADER MAY INFLUENCE GROUP INTERACTION DATA SOMEWHAT CUMBERSOME TO ANALYZE
NURSING EDUCATION
EARLY ASSOCIATE DEGREE PROGRAMS USED
PRE- AND POST-CONFERENCE MODEL:
PRE-CONFERENCE “SET THE STAGE”
ASSIGNMENT
POST-CONFERENCEASSIGNMENT
PRE - POST -
CONFERENCE CONFERENCE
PROCESSING
(LEARNING)
(ASSSESSMENT)“TAKING A TRIP” PRE-CONFERENCE (PLANNING) ASSIGNMENT (THE TRIP!) POST-CONFERENCE (SHARING AND PROBLEM- SOLVING)
APPLICATIONS IN A PHARMACY PROGRAM
UCSOP COMPREHENSIVE PLAN • PROGRAM • STUDENT LEARNING/ACADEMIC ACHIEVEMENT • PERSONNEL
MISSION
STRATEGIC
PHARM UC
PLANNING
RESIDENCY
ACCREDITATION
PROGRAM
PROGRAM
EMPLOYERS FACULTY
GRADUATES PRECEPTORS
STUDENT
PRACTICE SITES
SATISFACTIONADMISSION
DATA
NORTH
AMERICAN
DEFINING
PHARMACIST
ISSUES
LICENSURE
(NAPLEX)
APPE STUDENT CRITICAL
LEARNING/ACADEMIC
EVALUATIONS ACHIEVEMENT THINKING
IPPE CLASSROOM
EVALUATIONS ASSESSMENTS
PHARMACY
CURRICULUM
OUTCOMES
ASSESSMENTCREDENTIALS
INDIVIDUAL
DEVELOPMENT &
FACULTY/STAFF
EDUCATIONAL
PROFESSIONALISM
ASSESSMENT
(IDEA)
PRECEPTOR CLASSROOM
EVALUATION EVALUATIONS (BY
(BY STUDENTS) STUDENTS)
PERSONNEL
STAFF PEER
EVALUATION EVALUATIONS
FACULTY
FACULTY ANNUAL
SATISFACTION &
REPORT
FACILITY SURVEYSALAD SEMINAR
SALAD SEMINAR (Student Assessment of Learning and Decision-making) • IPPE activities P2 students engaged in? • How do students perceive their experience? • Opportunity to do/observe patient teaching? • Opportunity for critical thinking? • Opportunity for problem solving? • Interactions with health care team? • How could experience improved?
SALAD SEMINAR QUESTIONS What is a typical assignment at your site? How effective were you interactions with other members of the health care team? What was one important thing you learned from this experience? How could this rotation be more effective?
CHALLENGES IDENTIFIED • Uneasy talking with physicians & nurses • Sometimes no one available or “had no time” to talk with student • Sometimes felt “out of place” and “in the way” • Didn’t always understand what they were seeing
HOW HAS FEEDBACK BEEN USED? • BETTER STUDENT ORIENTATION TO IPPE • ROTATION SITE SELECTION • PRECEPTOR TRAINING • IMPROVED FEEDBACK TO PRECEPTORS • ADDITIONAL STAFF TO VISIT SITES
MARKETING
• WHAT ARE THE BEST QUALITIES OF THE UCSOP? • WHAT IS OUR NICHE, WHAT IS SPECIAL HERE AT THE UCSOP? • HOW CAN WE MORE EFFECTIVELY MARKET THE SCHOOL?
STUDENT RESPONSES: • SERVICE TO THE COMMUNITY/ADVOCACY • DIVERSITY OF FACULTY – THEIR KNOWLEDGE AND EXPERTISE • TECHNOLOGY – LEARNING & PREPARATION FOR PRACTICE AS GRADUATES • CERTIFICATIONS - MEDICATION THERAPY MANAGEMENT (MTM) & IMMUNIZATION
“The University of Charleston School of Pharmacy provides a challenging curriculum, preparing students for a successful pharmacy practice “. “The curriculum in the UCSOP provides students with patient interactions “early and often” as they explore the many opportunities offered by the pharmacy profession “. “The University of Charleston School of Pharmacy is a demonstration lab for excellence in pharmacy education, community involvement and patient advocacy “.
GRADUATING STUDENTS (P4)
TRIANGULATION
• STUDENTS COMPLETE AACP GRADUATING
STUDENT SURVEY
• ALL STUDENTS (70) IN FIRST GRADUATING CLASS
(2010) INVITED TO PARTICIPATE IN FOCUS
GROUPS
FINDINGS COMPAREDPROCESS • DURING MARCH & APRIL RANDOMLY CONFIGURED GROUPS OF 6 TO 10 STUDENTS MET AT OFF-CAMPUS SITE • DINNER/PARKING PROVIDED • DEAN UPDATED STUDENTS • ALUMNI OFFICE PROVIDED INFORMATION • DISCUSSION FACILITATED BY FACULTY MEMBER • RECORDER
QUESTIONS 1. REFLECTING OVER YOUR 4 YEARS IN THE UCSOP, WHAT HAS BEEN A POSITIVE EXPERIENCE FOR YOU? 2. WHAT DID YOU LIKE ABOUT THE PROGRAM/ CURRICULUM? 3. WHAT DIDN’T YOU LIKE ABOUT THE PROGRAM/ CURRICULUM?
4. HOW HAVE THE EXPERIENTIAL ROTATIONS BENEFITTED YOUR DEVELOPMENT? 5. IF YOU COULD CHANGE ONE THING ABOUT THE UCSOP, WHAT WOULD YOU CHANGE AND WHY? 6. IS THERE ANYTHING ELSE YOU WOULD LIKE TO SHARE THAT MIGHT BE HELPFUL IN STRENGTHENING THE UCSOP?
GRADUATING STUDENT FOCUS GROUP
STRENGTH SUPPORTING COMMENTS
Faculty and staff know who you are as an individual
Faculty follow-up to make sure students understand lectures/material
FACULTY AND STAFF
I feel welcomed at UCSOP, comfortable, the environment facilitates learning
Received personal attention from faculty and staff, people are caring, I felt appreciated
Faculty and staff are interested in students as individuals; their life circumstances
New faculty not set in their ways-open to new ideas
Faculty willing to help; even after hours
The curriculum is innovative; up to date
Offerings many other schools do not have, i.e. MTM, Immunization Training
CURRICULUM
Emphasis on advocacy and patient counseling
Received good education; provided knowledge to be a good pharmacist
The school prepared me for my career
Curriculum builds on previous learning
NAPLEX preparation was very helpful
I have grown in all ways but particularly professionalism
PROFESSIONALISM Even though students complained about it; agree it was very beneficial
Support for student participation in organizations
Community outreach is stressed
Seeing patients is a rewarding experienceCONCERNS IDENTIFIED BY GRADUATING STUDENTS
CONCERNS SUGGESTIONS FROM P4 STUDENTS WHAT WE HAVE DONE/PLAN TO DO
Need to use mannequins more effectively/often Efforts continue to provide faculty with additional training and updates
in use of simulated mannequins to increase use in the curriculum
Provide more opportunity for students interested Faculty have increased numbers of students involved in research and
in research to work with faculty research projects participation in Brown Bag events to share their involvement
Paid for clickers, did not use/either use or Students no longer required to purchase Personal Response Stations/
PEDAGOGY/TECHNOLOGY eliminate provided when faculty want to use this technology in the classroom
Link articles to patient cases P3 faculty engaged in increasing use of active learning in the classroom
More faculty/student interaction in classroom with follow-up research to determine effectiveness
Faculty attendance at teaching/learning conferences to increase teaching
expertise
On-line Ethics – need to assure Internet access On-line sessions scheduled earlier/Students receive schedule with
syllabus to be better able to plan for finding Internet access in remote
areas
Place drug literature later in program when more Beginning in 2012-2013 PHAR 511 Drug Lit will remain as is in P1 year;
relevant however, an additional 2 hour course will be created for P3 year.
Consider placing Journal Clubs earlier so not so Will no longer require P4 Journal Clubs in December and April. Surveys
rushed the week prior to graduation determined all P4 students have opportunity to present multiple times
CURRICULUM during rotations.
Law content limited – need more exposure to
Federal Law Beginning 2011-2012 pharmacy faculty member will teach law content.
Ethics overemphasized Working toward integrating ethics concepts throughout curriculum with
P4 capstone focused on dealing with “real-life” ethical issues
Need more business management in curriculum/ Preliminary discussions with graduate school of Business underway.
consider offering an elective in management2011 & 2012 • Sampled classes via random groups • Provided opportunity for those who wanted to participate and not captured in random groups • Student curiosity piqued – asked to participate
EMPLOYERS
EMPLOYER FOCUS GROUP • All regional employers invited for an evening focus group on campus. • Provided dinner. • University President & Dean SOP present • Group discussion
QUESTIONS FOR EMPLOYERS:
• Are University of Charleston School of Pharmacy (UCSOP) graduates prepared for
entry into practice, including being able to:
– demonstrate professional behavior.
– communicate effectively with patients, caregivers, and other members of the inter-
professional health care team.
– provide patient education.
– develop and use patient-specific pharmacy care plans.
– develop disease management programs.
– manage the system of medication use.
– apply state and federal laws and regulations to the practice of pharmacy.
• Can you share some indication(s) that UCSOP graduates understand the health
care system/environment?
• Can you provide evidence or anecdotal that indicate that UCSOP graduates are
prepared for the management aspects of pharmacy?• What do UCSOP graduates lack in the areas of:
– pharmacy content/didactic education?
– clinical training (skill set)?
• As you think about where your company or agency is going in the future,
what do we need to be adding to the content or skill set of our
graduates?
• What else would you like to tell us about our graduates? Our Program?
• What would you identify as the major strength(s) of UCSOP graduates,
i.e., what attributes would lead you to choose UCSOP graduates over
graduates from other schools of pharmacy?EMPLOYER IDENTIFIED STRENGTHS OF THE UC SCHOOL OF PHARMACY
STRENGTH SUPPORTING COMMENTS
Communication is a strength of UCSOP graduates was specifically mentioned x5
Communicate effectively with co-workers and patients
COMMUNICATION Good interaction with interviewers and preceptors
Able to effectively speak with physicians
Relate well with faculty and staff
Able to work well with diverse populations
UCSOP Graduates are very knowledgeable--specifically mentioned x2
Know where to obtain information
KNOWLEDGE BASE Demonstrate excellent scores on NAPLEX and MPJE
Community education
Emphasis on wellness
Good understanding of hospitals
Prepared to provide MTM services--specifically mentioned x3
SPECIFIC SKILLS Immunization training
Excellent with technology x2
Able to provide community education
FUTURISTIC Forward thinking/planning mentioned x3
Prepared for changing profession of pharmacy
PROFESSIONALISM Professionalism identified x2
Professional dress
Good work ethic was mentioned x2
PERSONAL CHARACTERISTICS Flexible x2
Adaptable
CommitmentEMPLOYER RECOMMENDATIONS/SUGGESTIONS
AREAS IDENTIFIED SUGGESTIONS FROM EMPLOYERS WHAT WE HAVE DONE/PLAN TO DO
Faculty will be invited to identify opportunities in the curriculum to
Assist students to develop more skill in: provide students with opportunity to practice related HR skills.
Managing employees/workforce Continue to use case studies/scenario in PHARM 807 to provide
Supervisory skills opportunity for students to work through ethical situations related to
HR TRAINING Dealing with co-worker issues personnel issues.
Customer service skills Invite practicing pharmacists to participate in teaching IPR
Conducting crucial conversations Faculty workshop on Crucial Conversations Fall 2011
Conflict resolution
Increase student ability to understand: Faculty will need to evaluate current content and determine need to
MANAGEMENT/BUSINESS CONCEPTS Reports, i.e. P & L statements explore ways to incorporate or increase opportunity for delivering this
Pharmacy Benefits Management (PBM) content in the curriculum.
Insurance – particularly the “major players”
Encourage: Explore active learning opportunities to better prepare students to be
More face-to-face time with patients less reliant on computers/websites and increase their ability to provide
PATIENT INTERACTIONS Less reliance on technology information to patients from their own knowledge base,
Learning to gain patient trust
Drawing on knowledge base rather than
relying on computer (lose patient trust )
Increase: Curriculum mapping to determine opportunity to increase or reinforce
Emphasis on pharmacology this content.
PHARMACY CONTENT Knowledge of base chemical/pharmacology
of popular medications
Ability to draw from own knowledge instead of
relying so heavily on the computer for
information
Unrealistic to require residency/currently not Findings of this exercise have been reported to students and faculty.
OTHER important to employers
Assist students to learn to deal with change Encourage APPE rotations outside the area.
Prepare graduates to work out of the areaFOLLOW-UP ON EMPLOYER GROUP • Curriculum modifications – Increasing HR concepts in Management Course – Increase focus on interviewing skills • Alumni CE presentations with Commencement weekend – Conducting Interviews for Potential Employees – Management Issues • Planning webinar series for fall
GUIDING PRINCIPLES
CLEARLY IDENTIFY YOUR PURPOSE KNOW WHAT YOU WANT TO ACCOMPLISH KNOW HOW YOU WILL USE THE FINDINGS
DETERMINE QUESTIONS (5 TO 7) • OPEN-ENDED • AVOID SENSITIVE OR BIASED QUESTIONS • ASK BOTH POSITIVE & NEGATIVE ASPECTS • DEVELOP A SCRIPT • PLAN ON 1-2 HOUR TIME FRAME
TRAIN FACILITATORS • MODERATOR MUST BE IMPARTIAL • MODERATOR SETS TONE & IDENTIFIES OR HELPS GROUP SET GROUND RULES • NOTE TAKING SUPPLEMENTED BY AUDIO OR VIDEO TAPING
GROUP SELECTION GROUPS OF 6 TO 10 IDEAL MULTIPLE GROUPS TO “PALPATE” LARGER GROUP REPLICATION OF GROUPS INCREASES RELIABILITY INFORM GROUP OF PURPOSE & HOW GENERATED INFORMATION WILL BE USED
SITE SELECTION • SHOULD BE NEUTRAL • ROOM SHOULD ALLOW FOR COMFORT • PARTICIPANTS FACE ONE ANOTHER
ENCOURAGING PARTICIPATION AMENITIES IMPORTANCE OF PARTICIPATING FEEDBACK
CONDUCT WITH RESPECT ALL PARTICIPANTS MUST BE HEARD DRAW OUT QUIETER MEMBERS STAY ON TRACK DO NOT ALLOW TO BECOME “GRIPE” SESSION
SUMMARIZE SUMMARIZE FOR GROUP ORGANIZE NOTES TAKE CARE TO NOT IDENTIFY INDIVIDUALS NOTE FREQUENCY OF COMMENTS
SHARE FINDINGS
WITH GROUP – PROVIDE FEEDBACK
USE DATA TO:
-VALIDATE WHAT YOU ARE DOING WELL
-IDENTIFY NEED FOR CHANGE/ACTIONGROUP DISCUSSION/EXERCISE
HOW HAVE YOU USED FOCUS GROUPS?
GROUP EXERCISE Assume you are selected to develop a focus group to evaluate this 2012 AALHE Conference. Identify: 1. How you will select a representative group(s)? 2. How you might get individuals to participate? 3. What questions to ask? 4. Where you will conduct the session? 5. Criteria for the facilitator? 6. How you will share your findings?
SUMMARY
FOCUS GROUPS: • CAN PROVIDE RELATIVELY INEXPENSIVE METHOD OF GATHERING VALUABLE INFORMATION. • MUST BE CONDUCTED WITH SAME CARE AND RIGOR AS OTHER ASSESSMENT METHODS • CAN BE USED AT ALL LEVELS OF HIGHER EDUCATION AND IN ANY PROGRAM
THE GATHERING AND USE OF
DATA TO VALIDATE STRENGTHS OR
TO IMPROVE COURSE OR
PROGRAM QUALITY A FUNCTION
AND VALUABLE COMPONENT OF
ASSESSMENT.FOCUS GROUPS CAN PROVIDE THE “VOICE” – THE REAL LIFE FEEDBACK IN ASSESSMENT! QUESTIONS?
PLEASE COMPLETE THE EVALUATION
FOR THIS SESSION AND LEAVE IN THE
BACK OF THE ROOM.CONTACT INFORMATION Sandra S. Bowles, EdD, RN, CNE Professor of Nursing & Dean Emeritus Bradford Division of Health Sciences/ Assistant Dean for Assessment University of Charleston School of Pharmacy sbowles@ucwv.edu (304) 357-4376
You can also read
