Putting the Teaching Quality Framework Initiative - no animation for full ...
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Putting the Teaching Quality Framework Initiative
Into Action:
A case-study comparison of three departments engaged
in transforming teaching evaluation
Sarah E. Andrews, Jessica Keating, Joel Corbo, Mark Gammon,
Daniel Reinholz, and Noah Finkelstein
ASCN Transforming Institutions Conference
April 4, 2019
sarah.andrews-1@colorado.edu
@sarahfoofoo79
#ASCNHigherEdTI2019
Visit us at:
https://www.colorado.edu/teaching-quality-framework/ (TQF)
http://teval.net/ (multi-institutional umbrella project TEval)
Why change current teaching evaluation practices?
Take 45 seconds to talk to your neighbor:
What is one reason to change practices?
OR
What is one thing you would change about current teaching
evaluation practices?
Why change current teaching evaluation practices?
• Scholarly approach • Multiple voices (from students,
• Improve reputation from peers, from self-reflection)
• Help faculty navigate RT&P • Add qual/de-emphasize quant
processes • Evidence-based, reliable
• Help faculty become better measures
teachers/support faculty in • De-emphasize SETs
their growth • Eliminate or reduce bias
• Improve student outcomes • Consistency/ability to compare
• Align values to evaluation growth over time
The Teaching Quality
Framework:
A model of improved
teaching assessment
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
The Teaching Quality
Framework:
A model of improved
teaching assessment
Improved Evaluation System
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
A case study example: The Juniper* Dept
• Dept readiness
• Flexible process
• Early successes
• Iterative process
DAT = Departmental Action Team (Corbo et al., 2015)
* All dept names are fictitious
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
The Teaching Quality
Framework:
A model of improved
teaching assessment
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Brainstorming Activity: Card Sorting Activity:
Ideal Teacher Literature-based components of effective teaching
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Peer Observation Protocol
Peer (Adapted
Coursefrom the UTeach
Evaluation Observation Protocol, UTOP 1)
Plan
Peer Evaluation of Courses/Teaching
You have been asked to Overview/purpose
complete a peer observation forPurpose a [DEPT] faculty member for inclusion in
The [x] Department has collectively committed to developing best practices in teaching. The process
their reappointment, comprehensive review, promotion, This or tenure case.provides
Please see theteaching
[DEPT] Peer aboutand
Guideoutlined
for writinghere will peer classroom observation letters
utilize the voices of faculty peers towards document
a) improving background
individual information
development, peerb) classroom observation
for evaluative reporting, and proposes a system and peer
Course Evaluation Plan [insert
such aslink when finalized]Promotion,
for Reappointment, evaluation
for full form whereby
instructions
and Tenure on faculty
the peer
Review. can be evaluated
It isevaluation
designed process.
with severalongoals:
their courses and[DEPT]
a) to provide teaching. CLASSROOM INTERVIEW GUIDE
greater A copy of this document should be
attached to peer
consistency in the scheduling of observations and inobservation
evaluating protocols
componentsWhen ofwriting
submitted for your
teaching that peer classroom
reappointment, promotion,
the department observation
values and letter,
as tenure please address the following items1 (if they areMERIT Self-reflective teaching statement
review.
effective practices; b) to be formative and developmental for faculty at allrelevant). To provide
ranks in improving feedback
teaching overClassroom
on thisand
time; c) Interview
guide, Process
please respond Guide
to the questions onfor
thereflecting
2nd page andon your
returnteaching in preparation for writing your Faculty Statement on
❏ Before printing thistoform, insert
foster the instructor’s
a departmental ofBackground
culturelearning goals
scholarly (e.g., Student
teaching and sharedLearning
visions.Objectives
To achieve these goals, the process Reflect on your Teaching
teaching during the past calendar year. The PEC will use your statement as part of the
Unstructured peerincluding
classroom to
observation,[X].
i.e., those that are
implements
(SLOs)) for this course directlystrategies that are
into the space backed
provided by research,
in item A1) Learning employing
Goals. a standard protocol fornot based on
classroom a set of core criteria, can result in
observations
Option A)department
Print out (AAAS,
the classroom interview form below and have evaluation process.
each student Your reflection
complete should address some or all of the following guiding questions. You
the form
inconsistency and do not always address teaching practices
and incorporating those observations within a broader process of consultation and conversation (e.g., Brinko, that are valued by
1993 a 2012).
; of the class period observed For this do not have to answer all of the questions.
❏ Use the “Evidence/Notes” boxes 1 to make notes regarding
reason, each
the question
scholarly during
literature the
on observation.
best practices in teaching recommends at thethat end
academic units spell out the best (5-10
When
teaching minutes).
preparing Collect
to write yourallFaculty
responses and summarize
Statement on Teachingthem
for reappointment, promotion,
AAAS, 2012) . Review of syllabus and other course materials:
Please check NA if a particular item did not apply topractices
the classforyou field and define core criteria to use in the observationinprocess
theirobserved. the classroom
(AAAS, 2012,observation protocol
Cornell). Feedback in the
onand space
tenure, weprovided
recommend [insert
Please link].
reflectinglimiton theresponse
your following to
components
no more thanof effective teachingpages.
2 single spaced and
teaching can be more effective in promoting ● growth
Did the andsyllabus clearlywhen
improvement describe expectations,
it focuses on specificrequirements,
issues, containsand assessments for the course?
Evaluation types incorporating them into your statement. While these components are based on foundational
❏ As soon as possible after the observation, review your notesinformation,
concrete and write aand quick
is summary/key
based on specific data (rather than general impressions) (Brinko, 1993). To that
Option B) Dedicate 15-25 minutes of classroom end, a group
There are two types of evaluation: a full peer course evaluation consisting of a pre-observation, classroom scholarshiptime
1
, thisto a more
guide is in1)focus
the
How group
early
didstages
yourstyle interview:
of development
courses andcomment
go? Please we welcome your feedback!
on achievement of course goals, level of student
takeaways in the space provided. of five instructors and faculty from [DEPT], partnering
observation(s), a post-observation discussion, and a formal report (see 3Technical in Procedures
with facilitators from
aspects/classroom the NSF-funded Teaching Quality Framework
mechanics:
below); and an abbreviated a) Please ensure that the faculty member being observed leaves the student
room before
(TQF) Initiative, have developed new standardized protocols for peer classroom observation that align with the TQF engagement, learning outcomes. You may want to address aspects of your courses and/or
❏ There is additional space at theconsisting
observation end of thisofform if you
a single run out
classroom
Initiative and ofare
space
observation
basedwithin
and
on an aestablished ●measure,
box.report (see
formal Resources.
4 in Procedures Resources
the UTeach below). selected
Observation forbeginning.
Protocol the(UTOP),
class (e.g., boardatwork,
developed the slides, handouts, etc.) were
Goals for student learningteaching that proved to be particularly effective and/or ineffective, as well as how you assessed
❏ Please retain the completed protocol for your records.
University of Texas at Austin. educationally valuable, well executed, and beneficial for the students.● What knowledge, skills, and attitudes
efficacy. You may are focusimportant
on one for student
course success
or several in your
courses.
Selection of observers b) Briefly explain the purpose of this process.
● Organized. The instructor’s activities were well organized, structured, and made good use of
discipline?
The department chair, director of undergraduate studies, and chair of the mentoringofcommittee
How the protocols will be used for evaluation teaching in will[DEPT]
meet twice annually
time. c) Form students into small groups, depending on class2)size.
Instructor (August and December) to determine Thesewhostandardized
will be observedtools,inwhich replace the
the upcoming unstructured
semester based letters
upon the previously
frequency detailedfor Reappointment, Promotion, ● Are course goals appropriately
submitted challenging
What (if any) changesfor didthe
you level of the course?
introduce in your classes and why?
and Tenure Review, contribute to the consistency ● Accuracy.and rigorInstructor’s
of our evaluation writtenof and spoken
teaching, ourcontent
comparisoninformation
of teachingwas accurate
● Whatand areappropriately
you preparing students for?
below, and prioritizing those due for Reappointment, Promotion, and Tenure Review. The committee will also appoint d) Pose your chosen questions to the class.
practices over time, and our development of a shared
thorough vision for teaching
(e.g.toinformation practice
written in [DEPT]. In addition, it is our hope
Course Name faculty to conduct evaluations andthat
to select the appropriate
the structured form will evaluation type (faculty
better enable members
more formative be observed
assessment will be on board,
and self-reflection
in hand-outs).
on teaching practices.
● What are key challenges in 2a) the What
teaching-learning
adjustments process?
did you make in response to prior course feedback (FCQs, peer
Course Number/Section consulted as part of this process). Generally, some observers will be assigned ● from
Active Learning
within the faculty (ifmember’s
applicable).e) Have
field The
of instructor
each group employed
discuss active
andlearning
come to strategies appropriate
a consensus on eachobservation,
question. FTEP observation, Qualtrics, surveys)?
study and some will be chosen from Foroutside
reappointment, promotion,
fields. However, and to
in order tenure review,
balance for the size
the
workload, instructor under
and structure
schedules, review
etc., of submits
thethe class, all
andclassroom
in line with observation
departmental expectations.
Preparation for teaching
Date/Time/Room/Bldgdepartment chair will have final sayprotocols completed
in the selection by their peers to the PUEC. The instructor will also have the
of observers. f) opportunity
Have eachtogroup share
reflect on theseout their responses to the whole class. Write these responses
● What preparation have you 2b) done to acquire
What steps havedeepyou knowledge
taken to of the subject or develop
enhance/further build your knowledge about effective
protocols when writing their teaching statements for reappointment, promotion, and tenureeveryone
where review, andcan in writing their whiteboard/chalkboard, PowerPoint slide).
see (e.g.,
# of students enrolled/# of self-reflection for annual merit review. What the instructor was doing/efforts made to engage students: topics into your courses? teaching practices, methods, and/or materials? You may wish to consider factors such as
Procedure ● on How do you know whataskpractices are evidence based and what are not? literature; attending FTEP, ASSETT, or COLTT
students who attended ● Content. The instructor choseg)examples If time and youdetails
may leadthat were appropriate
a discussion and worthwhile
these items for
and/or reading
students in scholarship
to vote on of teaching
most and learning
1. All first-time observers, or observers who are new to this process, shall meet with the department chair to review
helpingprotocol
students learn contentimportant takeaways. ● In what ways do you incorporate evidence-based
sessions; teaching practices
engaging in discussion into youretc.
with colleagues; courses?
How have these opportunities initiated
Observer Table
and become familiar with this 1. Alignment of [DEPT] Peer Classroom Observation
process. items withthe
components in effective
of this course.
classroom teaching from the
TQF rubric. ● Motivation. The instructor provided context and made clear attempts●to How pointdo
outyouthe identify
relevancestudent challenges?
of reflection about your teaching and changes in your teaching?
h) Record responses/key takeaways in
●realHowthedoclassroom
you work observation
to overcome protocol
them in the space
or address them when they arise?
2. Once you have been assigned the role of
Component aseffective
an observer, it isteaching
classroom the material,
your responsibility to initiatee.g.,
the by connecting
following process. it to other subjects,items
Corresponding giving examples,
in the Peer world applications, etc.
providedObservation
[insert link].
● Depth. The instructor delivered content and answered questions in a ●wayWhat changes have with
you made 2c) to yourmade
syllabus and/or course
were materials that
to reflect a what changes did you introduce in
a) Did the syllabus include the Required Syllabus Statements (i.e. Disability Protocols If you changes that not related 2a or 2b,
that was consistent
Accommodation; Religious 3. Holidays; Classroom
Full peer course Behavior;
evaluation Sexual
process Content,Misconduct, Yes deep knowledge of the Classroom commitment to careful planning and meaningful
your classes and why? assignments?
Goals, and Alignment subject. Interview
A3) Learning Goals Questions
Discrimination, Harassment, and/or Retaliation; and Honor
a. Pre-observation What are Code),
studentsper CU to learn from
expected No the courses taught? Are course goals appropriately ● What efforts have you made to design your courses, syllabi, and/or materials that show
challenging? Is content aligned with the curriculum ●andReasoning. The instructor
integrate other topics Select 2 orA6)
highlighted
and/or courses? 3 Prior
ofthe
the following
Knowledge
ideas behind questions
the content to andinclude in your
encouraged classroom
students to interview (ideally make this
Boulder policy? a commitment
i. Request and review the following items prior to the in person consultation:
increase their reasoning decision
ability. in consultation with the observed instructor). Youto meaningful
may 3)tailor
Whatany assignments
steps have you
question (e.g.,
taken that
specificallyto connect
evaluate
to to
thereal worldofapplications,
quality student understanding in a formative as well as
1. Syllabus, including
Preparation course learning goals (e.g., departmental Student Learning
for Teaching A2) Tools and Resources prior knowledge, and/or future course
summative fashion? applications)?
Formative assessments are low-stakes opportunities to monitor student learning
b) Did the syllabus clearly describe expectations and requirements for
Did the instructor the course,
demonstrate Yes
sufficient content, ● Informal
background, Assessment.
and pedagogical the
The
knowledge? instructor’s
instructor
Was or Learning
took
A5) Active to your interest.
advantage of opportunities to gauge student
Objectives (SLOs)) (required)
the instructor well-prepared in terms of classroom mechanics (e.g. grading, prepping activities, A7) Depth (Content Courses) ● How do you prepareand for are
all aspects
typicallyofnot
classroom
graded or mechanics (e.g.,points
only assigned grading, use on
based of completion. Summative assessments are
including departmental Student Learning Objectives (SLOs), iftech
applicable? No class handouts/exams,
understanding (e.g.,tobytheasking
2. Optional materials
materials, you
use, may consider:
etc.)? access course1)them What
A9)questions).
aspects
Accuracy of [insert
(Language Courses)faculty member’s name] teaching were most effective in helping
technology, prepping activities, lectures,
high-stakes measuresdemonstrations,
used to evaluateetc.)?
student learning and are therefore graded for
management system, prior FCQs, etc. you learn?
*If no in (a) or (b), what was missing/unclear? A4) Participation correctness/quality (e.g., quizzes, exams, projects).
ii. Set up an in Methods
person consultation
and Teaching Practices What the students were doing/student engagement: A5) Active Learning
What assignments, assessments, and activities are implemented? Are methods appropriate for and Methods and teaching practices
1. This aligned
meeting should occur before any classroom observation,
with the learning environment, the student●population
Engagement. early in
(inclusive ed,Students
the 2)
semester
course level)appeared
and
What aspects
A8) Critical
toA8)
beCorrectiveThinkingof
on taskFeedback [insert
(Content
throughout faculty
Courses) member’s name] teaching were least effective in helping
the class and engaged in learning. 4) Describe
(Language Courses)● What teaching methods do you the
use? teaching accomplishment(s) from the past year that you value the most.
2. Purpose: scheduling
departmental, course,the
andclass
studentvisit(s),
goals? understanding the goals of the course and/or the you learn? strengths/expertise
● Participation. The instructor established an environment that gave all
C1) Instructor students the opportunity to
● How do these methods contribute to your goals for students?
class session(s) you’ll be observing, and explaining/asking questions about course
Presentation and Student Interaction
materials. participate fully, including encouraging 3) How their participation
would you describe in class.
[insert●faculty
Why member’s name]
are these methods level of interest
appropriate
5) What other in contributions,
for use
teaching helping
in your discipline? For theorclassroom
challenges, concerns would you like to share?
A4) Participation
Are methods from above implemented effectively? Are students supported (e.g. student/teacher
3. During this consultation, it is also recommended that the observer and instructor discuss
interaction)?
students
A7) Engagementlearn? Explain
(Language and provideenvironment
Courses) example(s).(e.g., lecture, lab)? For the course level?
the possibility of supplementing the course Additional
evaluation considerations:
process with additional forms of ● What assignments, assessments, and learning activities are implemented?
data,Student
such as:outcomes
student interviews (FTEP CLIP
4) What could students do to help improve this class?
● service or a modified
Observations version
of what
What impact do these courses have on learners? What evidence shows the level of student
the -instructor
see did well.
A4) Participation
addendum), otherAreFTEP services, and/or(shiftASSETT’s VIP service.
● Suggestions for the instructor to improve theirFeedback
teaching.
understanding? measures of learning in student performance as 5) How has [insert faculty member’s name] worked to ensure members of the class were
a result of A8) Corrective (Language Courses)
class/instruction) aligned with goals?
engaged
● Comments on your interactions with during the semester?
the instructor. 1
1
Developed from the Rubric for Statements of Teaching Philosophy developed by Matt Kaplan, Chris O’Neal, Debbie Meizlish,
American Association
Marchfor2018
the from
Advancement of Science. 2012. Describing & Measuring Undergraduate STEM Teaching Practices.
1. The UTeach Observation Protocol (UTOP). Retrieved https://utop.uteach.utexas.edu/.
http://ccliconference.org/files/2013/11/Measuring-STEM-Teaching-Practices.pdf 6) The instructor has identified Rosario
a keyCarillo, and Diana Kardia (http://www.crlt.umich.edu/sites/default/files/resource_files/TeachingPhilosophyRubric.pdf) and
learning goal for this course as [insert goal that the
from foundational scholarship, including Scholarship Reconsidered (Boyer, 1990), Scholarship Assessed (Glassick, Huber, &
[Insert references
Brinko, K.T. 1993. The practice of giving feedback cited] What is effective? The Journal of1Higher Education 64(5): 574-593.
to improve teaching: instructor would like assessed]. How
Maeroff, much
1997), progress
and work do you
at the University feel(e.g.
of Kansas you Theare making
Evaluation towards
of Faculty Teaching Rubric), and with sponsorship from
https://www.jstor.org/stable/2959994 the National Science Foundation (DUE-1725959).
thisProtocol
1. Items were adapted from the UTeach Observation goal?(UTOP). Retrieved March 2018 from
https://utop.uteach.utexas.edu/.
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Teaching Quality Framework (TQF) - Aligning Data Sources to TQF Assessment Rubric
Sources of evidence you could collect
Professional (3) Voice Rank
Goals, Content, and Goals/content are well-articulated, high quality, up-to-date, and Self-reflection Classroom Student interviews/
Alignment appropriate observation CLIP
What are students expected Teaching statement
Course materials/content are aligned with course goal, include
to learn from the courses Review of syllabus/ FCQs
taught? Are course goals
high-quality elements
Course portfolio course materials
appropriately challenging? Range/depth of course topics is appropriate, integrate other Student letters
Is content aligned with the topics/courses Review of portfolio
curriculum?
Some examples of innovation, connection to current issues,
Review of learning
developments in field
objectives
Preparation for Knows subject on deep level, including current research, interaction Self-reflection Classroom Student interviews/
Teaching with other topics observation CLIP
Mapping data
Content/ Background Teaching statement
Knowledge; Pedagogical Teaching practices/methods or materials are evidence-based, shown Review of syllabus/ FCQs
Knowledge (i.e. teaching to enable learning Course portfolio course materials
generally and teaching Student letters
subject material);
Can identify most common student challenges; activities/innovations
sources and
to help students overcome common challenges Review of portfolio
Classroom mechanics
preparation (e.g. grading,
prepping activities, Syllabus, materials, and course material are well-planned, integrated, Faculty interviews
materials, tech use, etc) and reflect commitment to meaningful assignments
voices to the TQF
Methods and Teaching Often uses effective or innovative evidence-based* methods to Self-reflection Classroom Student interviews/
Practices improve understanding *See: NAS Indicators observation CLIP
What assignments, Teaching statement
assessments, and activities Activities provide opportunities for practice/ feedback on important
Review of syllabus/ FCQs
are implemented? Are skills and concepts course materials
Course portfolio
framework rubric
methods appropriate for Student letters
environment and aligned for Regular opportunities to practice skills
Inventory (e.g., TPI, Review of portfolio
student population Survey (e.g.,
(inclusive ed, course level) Students consistently engaged, w/ occasional high levels of TBI)
engagement COPUS SALG)
and goals (departmental,
course, student)
Implementation is high-quality/consistent Inventory (e.g., TPI, Inventory (e.g., TPI,
TBI) TBI)
Presentation and Evidence that classroom climate is respectful, cooperative, and Self-reflection Classroom Student interviews/
Student Interaction encourages motivation and engagement observation CLIP
What are the students’ views Teaching Statement
of the learning experience? Student reports of instructor accessibility and interaction skills are Review of portfolio FCQs
How has student feedback positive Course Portfolio
informed the teaching? Are Student letters
methods (#3) implemented Students perceive that they are learning important skills or knowledge
effectively? Are students
supported (e.g.
Survey (e.g.,
student/teacher Instructor gathers student feedback and articulates some lessons SALG)
interaction)? learned
For more information about the Teaching Quality Framework, please visit our website: www.colorado.edu/teaching-quality-framework
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)Departmental examples of adapting rubric mapping to their own available data sources
What do we know about a given instructor’s teaching and how do we know (i.e., where do we look for this information)?
How do we know/where do we look?
Corresponding items from:
What do we know about an instructor’s RT&P
Merit
teaching?1 (everything in merit plus…)
Component of effective classroom Your
Self-reflection Peer Observation FCQs Syllabi and course materials
teaching Score (1-5) Goals, Content, and Alignment FCQs:2 Peer Observations3
What are students expected to learn from the - S3 (intellectual challenge) - Review of syllabus
courses taught? Are course goals appropriately - CP5 (helpful materials) - A1) Resources
Goals, Content, and Alignment High weight High weight Low weight High weight
challenging? Is content aligned with the curriculum - OP1 (connections to other - A5) Content
What are students expected to learn A3) Learning Goals S3 (intellectual challenge) and integrate other topics and/or courses? classes/life) - A6) Motivation
from the courses taught? Are course A6) Prior Knowledge CP5 (helpful materials)
goals appropriately challenging? Is FCQs: Peer Observations
content aligned with the curriculum and - CP4 (invested in student - Review of syllabus
Preparation for Teaching success) - A1) Resources
integrate other topics and/or courses? Did the instructor demonstrate sufficient content, - CP5 (helpful materials) - A2) Organized
background, and pedagogical knowledge? Was the - A3) Accuracy
instructor well-prepared in terms of classroom - A4) Active Learning
Methods and Teaching Practices Equal weight Equal weight Equal weight Equal weight
mechanics (e.g. grading, prepping activities, - A5) Content
What assignments, assessments, and A4) Participation S3 (intellectual challenge) materials, tech use, etc.)? - A7) Depth
activities are implemented? Are A5) Active Learning CP7 (opportunities for - A8) Reasoning
methods appropriate for and aligned A8) Critical Thinking (C) discussion) - A9) Informal Assessment
with the learning environment, the A8) Corrective Feedback (L) CP8 (helpful feedback on work)
FCQs: Peer Observations
student population (inclusive ed, course C1) Strengths/expertise Methods and Teaching Practices - S3 (intellectual challenge) - A4) Active Learning
level) and departmental, course, and What assignments, assessments, and activities are - CP7 (opportunities for - A8) Reasoning
student goals? implemented? Are methods appropriate for and discussion) - A9) Informal Assessment
aligned with the learning environment, the student - CP8 (helpful feedback on work) - A10) Engagement
population (inclusive ed, course level) and - A11) Participation
Low weight Mid weight High weight Low weight departmental, course, and student goals? - C1) observations of what the
Presentation and Student Interaction A4) Participation S4 (how much learned) instructor did well
Are methods from above implemented A7) Engagement (L) S6 (encourages interest) FCQs: Peer Observations
effectively? Are students supported (e.g. S7 (instructor availability) Presentation and Student Interaction - S4 (how much learned) - A6) Motivation
student/teacher interaction)? S8 (respect for students) Are methods from above implemented effectively? - S6 (encourages interest) - A9) Informal Assessment
CP3 (respectful environ.) Are students supported (e.g. student/teacher - S7 (instructor availability) - A10) Engagement
interaction)? - S8 (respect for students) - A11) Participation
- CP3 (respectful environment)
High weight High weight Mid weight Mid weight
Student outcomes
A4) Participation S3 (intellectual challenge) Student Outcomes FCQs: Peer Observations
What impact do these courses have on What impact do these courses have on learners? - S3 (intellectual challenge) - A9) Informal Assessment
A8) Corrective Feedback (L) S4 (how much learned)
learners? What evidence shows the level What evidence shows the level of student - S4 (how much learned) - A11) Participation
CP4 (invested in student understanding? Are measures of learning (shift in - CP4 (invested in student
of student understanding? Are
success) student performance as a result of class/instruction) success)
measures of learning (shift in student aligned with goals? - CP8 (helpful feedback on work)
CP8 (helpful feedback on work)
performance as a result of
See also: Classroom Interview
class/instruction) aligned with goals? Mentorship and Advising FRPA Teaching Activities:
Q2 How effectively has the faculty member worked - Advisees/mentees
individually with undergraduate or graduate - Students’ supported in
students? independent work
Reflection, Development, & Teaching High weight NA NA High weight
Service/ Scholarship FRPA Service Activities:
How has the faculty member’s teaching - Attendance at teaching
changed over time? To what extent has Reflection, Development, & Teaching Service/
professional development
activities (Service Activities)
the teacher reflected on and improved Scholarship
- Participation in
their own teaching? How has the faculty member’s teaching changed
departmental/institutional
over time? To what extent has the teacher reflected
teaching-related committees
on and improved their own teaching?
Enter the average score across all components: FRPA Scholarly Works:
- Pedagogical publications
1
Seven components of effective teaching from the TQF rubric
2
For FCQs, S = standard questions, CP = core pilot questions, and OP are [DEPT]-specific optional pilot questions.
3
Peer observations are based on draft items currently under review.
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
The Teaching Quality
Framework:
A model of improved
teaching assessment
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Conclusions and Future Directions Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
Conclusions and Future Directions
• How does the dept teaching eval. process
compare to TQF framework before starting
DATs?
• What counts as movement toward TQF
framework?
• Do we see a shift in departmental culture
toward a more scholarly approach to
teaching?
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)Interested in learning more?!
Hear more about TQF, DATs, and other TEval affiliated
projects at these sessions later today:
• 1:15pm-1:45pm, Oral Presentation Session F in Woodlawn I (DATs):
• Characterizing departmental culture and assessing change with the DELTA survey
• 1:15pm-2:45pm, Thematic Symposium in Fountainview:
• National Academies' Roundtable on Systemic Change in Undergraduate STEM
Education: Directions and Opportunities
• 2:45pm-3:30pm, Poster Session in Admiral (Student Evaluations):
• Building a better SET: An "outside-in" approach to developing and implementing
improved student evaluations of teaching
• 3:30pm-5pm, Thematic Symposium in Waterfront (Rubric, etc.):
• Improving Learning by Transforming the Evaluation of Teaching: Resources,
Challenges, and Change Processes
sarah.andrews-1@colorado.edu
@sarahfoofoo79
#ASCNHigherEdTI2019
Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)Questions? We are grateful for the support of the National Science Foundation (grant DUE-1725959), Bayview Alliance, the Association of American Universities, CU Boulder College of Arts and Sciences, College of Engineering & Applied Science, and Leeds School of Business. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of any of these funders. Visit us at https://www.colorado.edu/teaching-quality-framework/ (TQF project) and http://teval.net/ (multi-institutional umbrella project TEval)
References Association of American Universities. 2014. Framework for Systemic Change in Undergraduate STEM Teaching and Learning. https://www.aau.edu/sites/default/files/STEM%20Scholarship/AAU_Framework.pdf Benton, S.L., & Young, S. June 2018. Best practices in the evaluation of teaching. IDEA Paper #69. IDEA Center, Inc., Manhattan, KS. https://files.eric.ed.gov/fulltext/ED588352.pdf Berk, R.A. 2018. Start spreading the news: Use multiple forms of evidence to evaluate teaching. The Journal of Faculty Development 32(1): 73-81. Bernstein, D. 2016. Now is a good time to recognize teaching as serious intellectual work. InSight: A Journal of Scholarly Teaching 11: 9-13. http://insightjournal.park.edu/wp-content/uploads/2016/07/2-Editorial-Bernstein.pdf Corbo, J.C., Reinholz, D.L., Dancy, M.H., & Finkelstein, N. 2015. Departmental Action Teams: Empowering faculty to make sustainable change. In 2015 Physics Education Research Conference Proceedings, pp. 91-94. Doerer, K. Jan 13 2019. Colleges are getting smarter about student evaluations. Here’s how. The Chronicle of Higher Education. https://www.chronicle.com/article/Colleges-Are-Getting-Smarter/245457 Finkelstein, N.F., and Keating, J. Aug 13 2018. Promoting scholarly evaluation of teaching: Addressing the third rail of academia. ASCB Newsletter. https://www.ascb.org/science-news/promoting-scholarly-evaluation-teaching-addressing-third-rail-academia/ Hansen, W.L. Summer 2014. Rethinking the student course evaluation: How a customized approach can improve teaching and learning. Liberal Education Vol. 100, No. 3. https://www.aacu.org/publications-research/periodicals/rethinking-student-course-evaluation Mills, M., and Hyle, H.E. 1999. Faculty evaluation: A prickly pair. Higher Education 38: 351-371. https://doi.org/10.1023/A:1003735227936 National Research Council. 2003. Evaluating and Improving Undergraduate Teaching in Science, Technology, Engineering, and Mathematics. Washington, DC: The National Academies Press. https://doi.org/10.17226/10024. Research Corporation for Science Advancement. 2015. Searching for Better Approaches: Effective Evaluation of Teaching and Learning in STEM. https://rescorp.org/gdresources/publications/effectivebook.pdf Tunks, K.W. Aug 20 2012. Transforming teaching through supplementary evaluations. Faculty Focus: Higher Ed Teaching Strategies from Magna Publications. https://www.facultyfocus.com/articles/faculty-evaluation/transforming-teaching-through-supplementary-evaluations/
You can also read
