The Teacher Performance Rate and Accuracy Scale (TPRA): Training as Evaluation
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Education and Training in Developmental Disabilities, 2005, 40(4), 411– 423
© Division on Developmental Disabilities
The Teacher Performance Rate and Accuracy Scale (TPRA):
Training as Evaluation
Denise E. Ross Jessica Singer-Dudek
Teachers College, Columbia University St. John’s University
R. Douglas Greer
Teachers College, Columbia University
Abstract: The purpose of this paper is to introduce the Teacher Performance Rate and Accuracy Scale (TPRA)
which is a method of direct teacher observation used in the teacher evaluation and training component of the
Comprehensive Application of Behavior Analysis to Schooling (CABAS®) model of schooling. The TPRA builds
on the concept of academic engaged time (a measure frequently employed during ecobehavioral assessment) by
counting the presence or absence of learn units (interlocking three-term contingencies for both students and
teachers) during instruction. Implementation procedures for the TPRA, its application for identification and
analysis of instructional problems, and its use for training and ongoing evaluation of teachers are presented
and discussed.
Evaluating teachers’ instructional effective- of instructional effectiveness such as parental
ness and providing feedback are components questionnaires, students’ evaluations, and
of teacher training that have been used to teachers’ self-reports (Bahr & Bahr, 1997;
improve both teachers’ performances and stu- Hersen & Bellack, 2002). While these mea-
dents’ learning (Andrejko, 1998; Howard & sures can provide feedback to teachers, re-
McColskey, 2001; Munby, 1999; Rauch & search suggests that indirect measures of class-
Whittaker, 1999; Smith, Harris, & Sammons, room instruction do not always reflect actual
2001). Approaches to teacher evaluation have changes in instructional effectiveness or stu-
included reviewing professional development dents’ learning, but rather the reporter’s ver-
plans (Holland & Adams, 2002), examining bal description of an intervention’s effective-
teacher work samples (Denner, Salzman, & ness (Hawkins, 1991; Poling, Methot, &
Bangert, 2001), conducting peer reviews LeSage, 1995). For example, Miller and Kelley
(Kumrow & Dahlen, 2002), and evaluating (1994) found that although homework com-
professional portfolios (Moore & Bond, pletion rates increased, parents reported dis-
2002). Research suggests that such ap-
satisfaction with their children’s rates of
proaches have been most effective when they
homework completion following an interven-
occurred regularly, were part of proactive pro-
tion to increase homework completion rates.
fessional development programs, were based
Similar incongruent relationships between
on multiple measures, and resulted in infor-
teachers’ reports and students’ performances
mation to help improve instruction (Prothe-
have also been identified in both science and
roe, 2002).
Such methods of teacher observation and music education (Maranzano, 2000; Moore,
feedback may often include indirect measures 2003).
Some researchers have suggested that the
primary measure of instructional effectiveness
should be objective measures and not infor-
Correspondence concerning this article should
be addressed to Denise E. Ross, Health and Behav-
mation obtained from subjective evaluations
ior Studies, Teachers College, Columbia University, (Hawkins, 1991; Schwartz & Baer, 1991). Di-
525 West 120th Street, Box 223, New York, NY rect observation is one method of teacher
10027. evaluation and training that may more accu-
TPRA / 411rately and objectively show actual changes in Ysseldyke and Christenson (as cited in both teachers’ behaviors and students’ learn- Christenson & Anderson, 2002) designed The ing than the previously discussed indirect Instructional Environment Scale (TIES-II) measures (Hawkins; Schwartz & Baer). Re- which is another model of ecobehavioral as- search on classroom environments has shown sessment used to collect direct observational that direct observational scales can measure data on individual students’ classroom behav- both teachers’ and students’ behaviors which iors across four categories: planning, manage- can, subsequently, be manipulated to increase ment, delivery, and monitoring/evaluation students’ learning (Greenwood, Carta, Kamps, (Spicuzza et al., 2001). Compiled information Terry, & Delquadri, 1994). For example, re- is reported in percentages of time and can be search on classroom environmental variables used to determine relationships between en- has shown direct positive correlations between vironmental factors and an individual stu- improved student learning and teachers’ ma- dent’s learning and to promote factors that nipulation of students’ rates of: 1) opportuni- increase learning (Quinn & McDougal, 1998). ties-to-respond (Greenwood, Hart, Walker, & TIES-II has been used to compare significant Risley, 1994), 2) active student responding differences between variables in classrooms of (Heward, 1994), 3) academic engaged time typical elementary school children and chil- (Fisher et al., 1980) and 4) contingent rein- dren at-risk for severe emotional disabilities forcement (Madsen, Becker, & Thomas, 1968). (Lago-Delello, 1998; Montague & Rinaldi, A common method of measuring relation- 2001) and to show the differential effects of a ships between teachers’ behaviors and stu- math curriculum on the pre and post aca- dents’ learning is ecobehavioral assessment. demic achievement of students with varying This type of assessment uses momentary time ability levels (Spicuzza et al., 2001). sampling to compare students’ behaviors in Research suggests that while ecobehavioral the context of teachers’ behaviors and other scales have been useful for identifying class- classroom variables and to determine which room variables that contribute to students’ variables promote high rates of academic en- learning, their usefulness and efficiency for gaged time (Greenwood & Delquadri, 2002; changing teachers’ classroom instruction is Logan & Malone, 1998). Greenwood, Carta, et limited for at least three reasons (Greenwood al. (1994) described a computerized ecobe- et al., 2002). First, while the primary purpose havioral software system—the Ecobehavioral of such analyses has been to measure aca- Assessment Scale (EBASS)— designed to demic engaged time (Greenwood, Carta, et record the presence or absence of a students’ al., 1994; Logan & Keefe, 1997), ecobehav- behaviors (i.e., academic and inappropriate ioral assessments have been largely restricted classroom behaviors), a teachers’ behaviors to descriptive-correlational analyses and have (i.e., teacher’s position in the classroom), and not resulted in changes to teachers’ instruc- classroom variables (i.e., physical arrange- tional behaviors that affect academic achieve- ment) in inclusive, specialized, and preschool ment (Greenwood et al., 2002; Juniper Gar- settings (Greenwood & Delquadri). EBASS dens Children’s Project, 2001). Second, codes has been used to accurately record data on used during ecobehavioral observations can observable classroom variables such as effects be complex. Greenwood and Delquadri of time of day on students’ behavior (2002) noted that, to date, EBASS has been (Muyskens & Ysseldyke, 1998), time spent on limited to observations of single students be- reading for students with learning disabilities cause of the observations’ complexity. Finally, and emotional/behavioral disorders (Vaughn, implementing ecobehavioral assessments can Levy, Coleman, & Bos, 2002), amount of aca- become costly. Greenwood et al. (2002) sug- demic engaged time for students with disabil- gested that use of computerized software for ities in inclusive settings (McDonnell, Thor- ecobehavioral assessment was expensive and son, McQuivey, & Kiefer-O’Donnell, 1997; required resources that were not routinely Wallace, Anderson, & Bartholomay, 2002), available to school personnel. and variables that promote engagement dur- The purpose of the present paper is to in- ing academic responding (Greenwood, Hor- troduce the Teacher Performance Rate and ton, & Utley., 2002). Accuracy Scale (TPRA), a relatively simple 412 / Education and Training in Developmental Disabilities-December 2005
and efficient method of direct teacher obser- term contingencies for students and interlock-
vation and evaluation that has been used to ing three-term contingencies for teachers. In
change teachers’ behaviors and students’ other words, learn units measure occurrence
learning by employing a measure that builds of antecedents, behaviors, and consequences
on the concept of academic engaged time for both teachers and students during instruc-
(Greer, 1994, 2002; Greer, Keohane, & Healy, tion. Greer (2002) termed the measure learn
2002; Ingham & Greer, 1992). Specifically, the unit since both teachers and students “learn”
TPRA directly measures three-term contin- from the interaction in a symbiotic manner.
gencies presented by teachers in order to as- Using the learn unit as a measure of teaching
sess functional interrelationships between allows for isolation of teaching and learning
teachers’ behaviors, students’ responses, and from moment-to-moment as students respond
instructional stimuli—interrelationships which to teachers’ instructions and teachers re-
research has shown to comprise academic en- spond, in turn, based on behaviors of their
gaged time (Heward, 1994). The TPRA is an students. Using the learn unit as a measure
integral part of the teacher evaluation and permits simultaneous assessment of both
training component of the Comprehensive teachers and students since learn units are
Application of Behavior Analysis to Schooling measures of teachers’ behaviors and responses
(CABAS®), which is a behavior analysis sys- to learn units are measures of students’ behav-
tems approach to education. The TPRA was iors.
designed to reflect and increase components Findings from behavior analysis, the TPRA,
of teaching shown to be effective in the liter- and the learn unit comprise the research
ature such as higher rates of students’ correct background for the TPRA. First, according to
responses, higher rates of opportunities to re- Greer (1994), at least four bodies of research
spond, lower rates of students’ incorrect re- from 1960-1990 in behavior analysis have con-
sponses, and lower rates of teachers’ instruc- tributed to the research background for the
tional errors (Greer, 2002). TPRA. These include research bodies showing
relationships between: 1) rates of contingent
teacher approvals and disapprovals and corre-
Research Background
sponding decreases and increases in undesir-
The TPRA builds on the concept of measur- able student behaviors (Heward, 1994), 2)
ing academic engaged time by counting the conditioning and generalized stimulus con-
presence or absence of learn units (Albers & trol which suggested that previously non-pre-
Greer, 1991; Emurian, 2004; Emurian, Hu, ferred academic tasks such as reading and
Wang, & Durham, 2000; Greer, 1994; Greer, writing could be reinforced to levels that
2002; Greer, McCorkle, & Williams, 1989; would maintain behavior in non-instructional
Greer et al., 2002; Ingham & Greer, 1992; settings (Greer, 1994), 3) increased correct
McDonough & Greer, 1999; Selinske, Greer, student responding and students who re-
& Lodhi, 1991). Table 1 provides an example ceived high numbers of opportunities-to-re-
of a learn unit. Learn units consist of three- spond (Greenwood, Hart, et al., 1994), and 4)
TABLE 1
Example of Learn Unit with Interlocking Three-term Contingencies for Teacher and Student
Three-term Contingencies Three-term Contingency
for Teacher Instructional Components for Student
First teacher antecedent Student attends to teacher
First teacher behavior Teachers says, “Count to 10” Student antecedent
First teacher consequence/
second teacher antecedent Student correctly counts to 10 Student behavior
Second teacher behavior Teachers says, “Good job!” Student consequence
Second teacher consequence Teacher records data and learn unit is completed
TPRA / 413applied behavior analysis and academic behav- Greer, 1997) as well as higher numbers of
iors such as those found in the research on correct responses and objectives attained by
Direct Instruction which showed that applied students (Greer et al., 1989; Ingham & Greer;
behavior analysis was useful for academic tasks Selinske et al., 1991) and as much as four to
as well as classroom management (Becker, seven times more correct responding than
1992). during baseline (Albers & Greer; Greer et al.,
Second, research on the TPRA showed re- 1989; Ingham & Greer). Finally, recent re-
lationships between TPRA observations and search suggests that learn units are useful for
students’ learning. Three studies found corre- measuring and changing instruction in both
lations between high numbers of supervisor- college classrooms (Bahadourian, 2000) and
conducted TPRA observations and increased computer-assisted instruction (Emurian,
teacher productivity, contingent conse- 2004). Subsequent research has shown that
quences during instruction, and learning of learning problems can be pinpointed and cor-
students with severe mental retardation at- rected based on the components of the learn
tending CABAS® schools (Greer, McCorkle, & unit (Greer, 2002; Keohane, 1997).
Williams, 1989; Lamm & Greer, 1991; Selin-
ske, Greer, & Lodhi, 1991). Additionally, Ing-
Implementing the TPRA
ham and Greer (1992) found generalized and
higher functional relationships between TPRA observations are generally conducted
TPRA scores and correct student responses by trained supervisors or mentor teachers, and
when compared to nonspecific feedback (i.e., can be used with varying class sizes (i.e., whole
“Nice lesson”) for teachers of students with groups or individual students) by an observer
mental retardation. Albers and Greer (1991) whose use of the TPRA has been calibrated to
showed similar results for students in remedial 90% accuracy across multiple observational
mathematics classrooms and also found that sessions with a trainer. Figure 1 and the pro-
high rates of correct academic behaviors for cedure outlined below illustrate steps to com-
teachers and students were not different for plete a TPRA observation form for a single
vocal versus written teacher instructional pre- student. Table 2 describes steps to complete
sentations. the TPRA for groups of students and for com-
Third, research on the learn unit resulted plex academic behaviors.
in response definitions for TPRA compo- First, the teacher and supervisor identify the
nents. Research suggested that antecedent following instructional components: 1) a tar-
presentations should be unambiguous (Albers get student, 2) a target instructional program,
& Greer, 1991; Ingham & Greer, 1992) and 3) operational definitions of target behaviors
that students must actively respond during re- for the student and corresponding correct
sponse opportunities (Greenwood, Hart, et and incorrect responses, 4) schedule(s) of re-
al., 1994; Heward, 1994). Additionally, stu- inforcement for the instructional session, 5)
dent learning increased more with the conse- antecedents and postcedents, 6) instructional
quence component than with the opportuni- conditions under which behavior should oc-
ty-to-respond alone (that is, the antecedent cur, and 7) necessary prerequisite skills that
and intraresponse time) (Albers & Greer; the student should have before instruction is
Greenwood, Hart, et al.; Greer, 1996). Re- implemented. The supervisor also observes
search also showed that postcedents emitted availability of related instructional materials
in the form of correction or reinforcement such as data collection forms, items serving as
operations were necessary for student learn- reinforcers, writing utensils, and curricular
ing (Albers & Greer; Ingham & Greer) and supplies (i.e., textbooks, flashcards). Finally,
that students should attend to written discrim- the supervisor and the teacher review the stu-
inative stimuli presented during correction dent’s graphs to determine 1) trends in stu-
procedures (Hogin, 1996). Research on the dent’s responses to the target program prior
number and rate of learn unit presentations to the pending observation, 2) consistency of
suggested that faster and higher rates of learn plotted data with accurate graphing protocol,
units resulted in increased correct responses and 3) appropriateness of the target program
by students (Ingham & Greer; Linhart-Kelly & and expected level of student response.
414 / Education and Training in Developmental Disabilities-December 2005Figure 1. Example of a completed TPRA form.
Second, the supervisor and, when appropri- within a set of learn units which comprise an
ate, the teacher (i.e., in a fluency program) instructional session. The timer is stopped
start a timer (i.e., a stopwatch) or record the only if an interruption occurs (i.e., a problem
analog time in minutes and seconds. Timing with another student or school-related disrup-
during TPRA observations begins with the tions) in which case the timer begins again
teacher’s presentation of the first antecedent when instruction resumes. The timer is for-
TPRA / 415TABLE 2 tion tool. When responses to textual passages
are the target behavior, the teacher records
Procedures for TPRA Observations Conducted
with Groups and Complex Academic Behaviors correctly and incorrectly read words by mark-
ing on an identical version of the passage that
1. Record learn units for classroom management is photocopied or covered with a transparent
separately by using an “A” for correct social overlay.
approvals and a “D” for correct social Using the TPRA form (see Figure 1), the
disapprovals administered by the teacher. supervisor records the accuracy of each com-
2. Use written work that incorporates vocal ponent of the learn unit based on information
responses such as worksheets or computerized collected with the teacher before the observa-
programs. tional session. First, the supervisor measures
3. Indicate the beginning and end of the teacher’s presentation of instructional an-
observational periods with vertical lines drawn
tecedents as correct or incorrect. Correct an-
on written work (i.e., worksheets).
tecedents occur if the teacher’s vocal and/or
4. Record vocal learn units (V) separately from
written learn units (W). nonvocal antecedent stimulus was unambigu-
5. Count written learn units when the student has ous, consistent with the lesson plan or script,
reviewed corrections and completed changes. and, in the case of curricular materials, the
This may not occur until the next class period. target stimuli were flawless (i.e., targeted stim-
6. Elapsed time is calculated by adding the time ulus features were salient). Correct anteced-
required to complete problems on the day of ent presentations are recorded under the “an-
the observation to the time involved to tecedent” column (A) with a checkmark.
complete the corrections. Incorrect antecedent presentations are re-
7. Homework time is included by adding the
corded under the same column with a circled
number of minutes in a day that responses are
checkmark.
reviewed and corrected by students to the actual
instructional time. Second, students’ responses are recorded in
the same manner described above for the
teacher. That is, the observer marks correct
student responses with a plus (⫹) and incor-
mally stopped when the teacher delivers the rect student responses with a minus (⫺).
final postcedent. Elapsed time is used to cal- These data are compared with the teacher’s
culate rate of instruction and TPRA scores. collection of student data upon completion of
During the instructional session, the the observation for interobserver agreement
teacher records students’ correct and incor- purposes.
rect responses to instruction based on the op- Third, the teacher’s presentation of postce-
erational definition described before the les- dents is recorded. These are measured as cor-
son. Correct student responses occur when rect and incorrect based on their contingent
the student emits a behavior consistent with relationship to the student’s responses. That
the operational definition within a specified is, teachers should perform reinforcement op-
intraresponse time (i.e., 5 seconds). Incorrect erations contingent upon correct student re-
student responses occur when the student sponses and correction procedures contin-
emits a behavior inconsistent with the opera- gent upon incorrect student responses. A
tional definition, does not emit the correct correct reinforcement operation is defined as
behavior within the specified intraresponse immediate presentation of a stimulus that
time, omits a response, or emits a correct functions as a reinforcer for the target student
response after emitting an incorrect response on the schedule of reinforcement specified
(i.e., self-correction). Incorrect responses and for the instructional session. An appropriate
response omissions are recorded as minuses correction procedure is defined by the
(⫺) and correct responses are recorded as scripted program, school, or classroom proto-
pluses (⫹). Using event recording, teachers col. In most cases, correction procedures
record responses immediately after each post- include the teacher’s presentation of the les-
cedent by using pencil and paper method (on son’s antecedent stimulus with an accompany-
a pre-existing data collection sheet), mechan- ing prompt or model for the target behavior.
ical counters, or a computerized data collec- Corrections are not reinforced and the stu-
416 / Education and Training in Developmental Disabilities-December 2005dent is required to emit the corrected re- instructional problems and problem-solve us-
sponse. Table 3 identifies various teacher post- ing a decision tree (Greer, 2002; Keohane,
cedents and the corresponding TPRA codes 1997) by asking the supervisor questions
used to record them. based on written comments. When feedback is
During or after formal observation, the su- given within an instructional session, teachers
pervisor provides feedback to the teacher in correct problem areas during subsequent
one or more of three forms. Supervisors may learn unit presentations or lessons.
provide oral feedback by reviewing correct
and incorrect learn units with the teacher
Formulas
based on the TPRA form. Written feedback is
provided by recording comments about spe- The first calculation is rate of correct and
cific learn units on the TPRA form and giving incorrect responses for the student. First, all
the form to the teacher to review. Supervisors responses (including response omissions) are
may also stop TPRA observations and correct tallied and total number of incorrect re-
teachers’ behaviors immediately—in which sponses is separated from total number of
case the TPRA observation begins again as correct responses. Next, elapsed time is con-
instruction resumes. When this happens, the verted into minutes by dividing seconds by 60.
supervisor may change an instructional proce- One minute and 20 seconds (1:20), for exam-
dure, model a correct procedure, verbally ple, would become 1.33 minutes. Then, num-
prompt the teacher to perform an instruc- ber of correct and incorrect responses is each
tional operation, or explain a procedure. divided by the converted time and a rate of
Feedback is not provided during a session in correct and incorrect responses is obtained.
which an interruption would be disruptive for The second calculation is rate of correct
a student. Teachers respond to TPRA feed- and incorrect responses for the teacher. Learn
back by demonstrating that they can identify units are correct if both antecedents and post-
cedents were presented accurately during ob-
servation. Learn units are incorrect if either
TABLE 3 antecedent or postcedent were presented in-
correctly during observation. Both correct
Codes for TPRA Observations
and incorrect learn units are summed and
divided separately by the converted time to
Code Response Definition
obtain a rate of correct and incorrect re-
√ Antecedent presented correctly
sponses for the teacher.
䡬
√ Antecedent presented incorrectly For example, during a lesson whose dura-
tion was 10 minutes, a teacher delivered 20
⫹ Correct student response
⫺ Incorrect student response instructional antecedents accurately but ig-
R Reinforcement operation nored two correct responses which means that
presented correctly the teacher presented 18 correct learn units
䡬
R Reinforcement operation omitted with 2 errors. Since the lesson’s duration was
or presented incorrectly 10 minutes, 18 would be divided by 10 for a
C Correction operation presented TPRA score of 1.8 correct learn units pre-
correctly
䡬
sented per minute and .2 incorrect learn units
C Correction operation omitted or
presented incorrectly
presented per minute. That means that stu-
D Social disapproval presented dents were given opportunities to actively re-
correctly spond and to receive reinforcement and feed-
䡬
D Social disapproval omitted or back for their responses approximately twice
presented incorrectly per minute.
A Social approval presented
correctly
䡬
A Social approval omitted or Analysis of Data
presented incorrectly Comparing TPRA scores is restricted to the
W Written antecedent presentation
same instructional programs by the same
V Vocal antecedent presentation
teacher with the same students because of
TPRA / 417variations in students’ learning histories, cur- ber of supervisor observations completed
ricular objectives, and levels of teachers’ ex- (Greer et al., 1989; Ingham & Greer, 1992),
pertise. For instance, some learn units contain supervisor expertise in solving instructional
multiple behaviors instead of a single student problems (Greer, 2002; Keohane, 1997), and
response; such is the case for programs that setting learn unit targets for teacher perfor-
involve larger teacher antecedents, larger stu- mance (Albers & Greer, 1991). When teach-
dent responses, varying schedules of rein- ers’ incorrect performances are relatively low
forcement, and different types of reinforcers. and stable, cumulative data reflecting the
When this happens, regardless of number of number of observations with and without er-
student behaviors, it is the teacher’s delivery rors are displayed.
of the consequence that defines the learn
unit. A common example is when students Use for Instructional Decisions
learning to write complex essays have fewer
When accurate and inaccurate data do not
learn units than students learning to write 25
reflect divergent trends in performance (that
spelling words because the placement of the
is, ascending and descending trends, respec-
teacher’s consequence during instruction de-
tively) but teacher performance is errorless,
termines size of the learn unit (at the end of
other components of instruction are reviewed
an essay versus at the end of each spelling
as possible sources for student learning prob-
word). Similarly, completing a page of math-
lems by using an instructional analysis deci-
ematical problems would include multiple re-
sion protocol and the learn unit context (i.e.,
sponses to a single antecedent. Likewise, cit-
motivational variables or learning history)
ing a 10-digit telephone number may begin
(Greer, 2002; Keohane, 1997). Possible sources
with a single response (i.e., the number 4)
of the problem may be that the (a) student
but, eventually, require 10 different numbers
lacks the prerequisite skills to respond to the
for a single correct response.
material being presented, (b) student lacks
TPRA scores can be displayed graphically
the topography or the response, (c) instruc-
across time or summed and divided by the
tional materials are insufficient for acquiring
number of observations to obtain a mean ac-
stimulus control, or d) schedule of reinforce-
curacy TPRA score. Improved TPRA scores
ment is too thin. While ability to change some
suggest the following: 1) shorter latent time
curricular problems does not lie in the TPRA
periods between learn units to students which
observation itself (i.e., what to teach when a
translates into greater amounts of instruction,
student lacks prerequisite skills), the observa-
2) fluent teacher presentations, and 3) in-
tional procedure allows an observer or teacher
creased contingency-shaped behaviors instead
to indicate that this is the area of the problem.
of rule-governed behaviors (i.e., teachers who
Later, the information can be used in con-
emit automatic behaviors instead of accessing
junction with a skilled teacher mentor to
procedures to instruct). For both teachers and
change a curricular problem. Changing a cur-
students, accurate rates should increase, inac-
ricular problem is usually completed by using
curate rates should decrease, and changes in
the decision protocol (Keohane, 1997) to an-
students’ performances should be analogous
alyze the learn unit content and to determine
to changes in teachers’ performances.
which of 200 research-based tactics (Greer) is
TPRA scores can be used in school-wide
likely to solve a particular instructional or
summaries of data and for teacher-supervisor
learning problem. Table 4 lists common in-
conferences conducted after observations.
structional errors, their associated TPRA com-
Specifically, when mean weekly scores for a
ponents, and potential solutions. Case studies
single teacher are integrated into a school’s
illustrating the application of TPRA data to
TPRA data, both composite and individual
instructional decisions are provided below.
TPRA data can be used to help identify learn-
ing objectives for students and teachers. Com-
Case Studies
posite data help schools analyze a number of
variables that CABAS® research has shown to Changing instructional errors. Teachers A
be functionally related to accurate student and B were co-teachers in an inclusive first
and teacher performance including the num- grade classroom where Janet, a 6-year old girl
418 / Education and Training in Developmental Disabilities-December 2005TABLE 4
Common Instructional Errors Related to TPRA Components
TPRA Component Instructional Errors Potential Solutions
Antecedent Vocal or written antecedent Review antecedents in program script
presented incorrectly or or protocol
inconsistently
Antecedent Target stimuli flawed or not salient Make target stimulus dimensions
salient
Antecedent Opportunity-to-respond inconsistent Use timer to prompt teacher to begin
or omitted and end intraresponse time (i.e., five
seconds)
Antecedent Incorrect prompt presented or Review prompt levels and fading
correct prompt presented procedures with teacher
inconsistently
Behavior Student lacks prerequisite skills Teach student prerequisite skills
Behavior Inappropriate prompt level Use different prompt level or error
correction procedure
Behavior Response topography is too complex Task analyze target behavior and
include prompts where needed
Behavior Size of learn unit too large Task analyze target behavior to change
size of learn unit
Consequence Student has no opportunity-to- Prompt teacher to include
respond consequence component opportunity-to-respond
Consequence Reinforcers not potent or satiation Vary reinforcers or perform
establishing operation
Consequence Reinforcers not delivered on Decrease or increase schedule of
appropriate schedule reinforcement
Consequence Reinforcers not delivered Prompt teacher to deliver reinforcers
immediately or contingently before he/she records students’ data
Number per minute Correct responses per minute are Increase teacher’s automatic/
too low contingency-shaped learn unit
presentations
with autism, was learning to read sight words and Janet achieved mastery criteria within the
aloud. Teachers used the learn unit—that is, next two instructional sessions.
interlocking three-term contingencies for the Increasing number of learn units emitted per
teacher with the potential operant for the stu- minute. Teacher C was a reading teacher for
dent—to teach Janet the target words. The 9th grade students with learning disabilities.
teachers alternately presented each of four She received TPRA observations on her imple-
20-learn unit instructional sessions. Janet only mentation of a new behavioral reading pro-
emitted a mean of 14.5 correct responses gram following training. During one 30-
(range, 13 to 18) and did not obtain the in- minute observation, she presented 64
structional objective of 90% mastery. TPRA complete learn units, but 104 additional ante-
observations showed that Teacher A pre- cedents to which students responded but were
sented 6.51 correct learn units and 0 incorrect not reinforced or corrected. The supervisor
learn units per minute, but Teacher B pre- showed Teacher C the TPRA scores and re-
sented 4.56 correct learn units and 1.95 incor- quested that she present a higher number of
rect learn units per minute. Teacher B’s er- contingent correction and reinforcement op-
rors were in the omission of an opportunity- erations during the next lesson. During a sec-
to-respond during correction operations. The ond 30-minute observation, Teacher C pre-
supervisor modeled the appropriate correc- sented 102 complete learn units and only 52
tion operation, observed Teacher B again, antecedents without consequences.
TPRA / 419Training student teachers. As part of the ecobehavioral assessment was intended to
masters of arts program at Teachers College, identify variables that promote academic en-
groups of student teachers wrote instructional gagement (e.g., instructional tasks or groups)
programs for children in CABAS® schools (Greenwood et al., 2002). However, while in-
which were, subsequently, evaluated using the formation derived from ecobehavioral assess-
TPRA. One group wrote an instructional pro- ment has increased students’ academic re-
gram to teach children with disabilities to emit sponding (Greenwood et al.), to date
conversations in three increasingly complex ecobehavioral assessments have mostly been
social situations. Children obtained mastery descriptive-correlational and have not identi-
criteria in the simplest social situations but not fied ways to promote engagement through
in the most complex social situation. Since altered classroom instruction (Greenwood et
multiple TPRA data showed that student al.). What appears to be lacking in the re-
teachers correctly presented all antecedents search literature is a description of a simple
and consequences in each social situation, the measure that assesses teachers’ instructional
mentoring teacher could eliminate faulty an- behaviors and that can be used to change
tecedents and consequences as the source of them (Greenwood et al.). The TPRA is a rel-
learning difficulty and examine other areas of atively simple and efficient method of teacher
the learn unit as possible sources for the chil- observation that has been used to change
dren’s low number of correct responses (i.e., teachers’ instructional behaviors during train-
prerequisite skills or the program’s generali- ing and evaluation.
zation components). When the TPRA was used weekly over one
or more academic years as part of a teacher
training and evaluation program, researchers
Conclusion
found correlational and functional relation-
The purpose of this paper was to describe the ships between its use by a trained observer and
TPRA by presenting its research background, teachers’ and students’ instructional re-
implementation procedures, formulas and sponses, including increased numbers of: 1)
data analysis, and instructional implications. instructional sessions taught, 2) learning ob-
The TPRA, an integral component of teacher jectives achieved, 3) correct student re-
training and evaluation in CABAS® schools, sponses, 4) learn units per minute, 5) oppor-
measures teacher-student interactions during tunities-to-respond, and 6) presentations of
instruction by assessing frequency of learn learn units during non-observational periods
units. Learn units are defined as interlocking (Greer et al., 1989; Ingham & Greer, 1992;
three-term contingencies (antecedents, be- Lamm & Greer, 1991; Selinske et al., 1991).
haviors, and consequences) for teachers and Based on these studies, a possible explanation
students. During timed instructional sessions for the procedure’s effectiveness and why it
conducted with various group sizes (i.e., rang- may be a valuable tool for teacher evaluation
ing from single students to whole classes), and training is provided below.
trained observers use event recording to mea- The TPRA may be a useful tool for evaluat-
sure the accuracy of teachers’ antecedent and ing teaching because its primary measure,
postcedent presentations as well as the accu- learn units, not only builds on opportunity-to-
racy of students’ responses. The accuracy of respond, but may also explain what is impor-
the learn unit’s components and subsequent tant about academic engaged time. Both aca-
rate calculations of teachers’ and students’ demic engaged time and opportunity-to-
correct and incorrect learn units per minute respond have been shown to predict academic
are used to remediate instructional errors as- achievement – that is, higher amounts of time
sociated with the learn unit—that is, the ante- spent on tasks that promote academic success
cedents, behaviors, and consequences for and higher numbers of opportunities-to-re-
both teachers and students. spond are correlated with student achieve-
As previously mentioned, observations from ment and on-task behavior (Greenwood, Hart,
ecobehavioral assessment literature provide et al., 1994). However, research suggests that
one basis for why the TPRA may be valuable learn units produce more student learning
for changing teacher behavior. Specifically, than opportunities-to-respond alone (Heward,
420 / Education and Training in Developmental Disabilities-December 20051994; Ingham & Greer, 1992) and result in Skinner (pp. 71–112). Longmont, CO: Sopris
relatively high amounts of feedback for teach- West.
ers about student learning and in relatively Christenson, S. L., & Anderson, A. R. (2002). Com-
high amounts of on-task behaviors for stu- mentary: The centrality of the learningcontext for
students’ academic enabler skills. The School Psy-
dents (Heward). Academic engaged time, as a
chology Review, 31, 378 –393.
measure, uses time as a primary dimension,
Denner, P. R., Salzman, S. A., & Bangert, A. W.
but does not account for the number of learn (2001). Linking teacher assessment to student
units that teachers present during instruction performance: A benchmarking, generalizability,
(Heward). If learn units are the point of con- and validity study of the use of teacher work sam-
tact between teachers and students (Heward) ples. Journal of Personnel Evaluation in Education,
then the learn unit may be a basic measure of 15, 287–307.
teaching (Greer, 1994, 2002). As such, the Emurian, H. H., Hu, X., Wang, J., & Durham, D.
TPRA, which measures learn units within (2000). Learning JAVA: A programmed instruc-
time, is a valuable tool for measuring and tion approach using Applets. Computers in Human
effecting behaviors of teachers. Behavior, 16, 395– 422.
Emurian, H. H. (2004). A Programmed Instruction
There is still much to learn about measur-
Tutoring System for Java“: Consideration of
ing teacher-student interactions at the teacher Learning Performance and Software Self-Efficacy.
level. Over the past 20 years, an estimated Computers in Human Behavior, 20, 423– 459.
300,000 TPRA observations have been com- Fisher, C. W., Berliner, D. C., Filby, N. N., Marliave,
pleted across at least 20 schools involving R., Cahen, L. S., & Dishaw, M. M. (1980). Teach-
more than 500 teachers. As part of teacher ing behaviors, academic learning time, and stu-
training and observation in eight CABAS® dent achievement: An overview. In C. Denham &
schools in the United States, Ireland, and En- A. Lieberman (Eds.), Time to learn (pp. 7–32).
gland, more than 3.8 million data points are Washington, DC: U.S. Department of Education.
generated from TPRA observations and learn Greenwood, C. R., Carta, J. J., Kamps, D., Terry, B.,
& Delquadri, J. (1994). Development and valida-
unit data derived from instructional sessions
tion of standard classroom observation systems
with teachers, parents, students, and supervi-
for school practitioners: Ecobehavioral assess-
sor. Based on our experiences with this pro- ment systems software (EBASS). Exceptional Chil-
cedure, a primary benefit of the TPRA is that dren, 61, 197–210.
this observational tool is a simple procedure Greenwood, C. R., & Delquadri, J. (2002). Code for
that can provide in-class training for and eval- instructional structure and student academic re-
uation of teachers without costly, out-of-class sponse: CISSAR. In M. Hersen & A. S. Bellack
workshops or equipment (Ingham & Greer, (Eds.), Dictionary of behavioral assessment techniques
1992). (pp. 120 –122). New York: Percheron Press.
Greenwood, C., R., Hart, B., Walker, D. I., & Risley,
T. (1994). The opportunity to respond and aca-
References demic performance revisited: A behavioral theory
of developmental retardation and its prevention.
Albers, A. E., & Greer, R. D. (1991). Is the three- In R. Gardner, D. M. Sainato, J. O. Cooper, T. E.
term contingency trial a predictor of effective Heron, W. L. Heward, J. Eshelman, et al. (Eds.),
instruction? Journal of Behavioral Education, 1, 337– Behavior analysis in education: Focus on measurably
354. superior education (pp. 213–224). Pacific Grove,
Andrejko, L. (1998). The case for the teacher port- CA: Brooks/Cole.
folio. Journal of Staff Development, Staff 19, 45– 48. Greenwood, C. R., Horton, B. T., & Utley, C. A.
Bahadourian, A. (2000). The effects of learn units on (2002). Academic engagement: Current perspec-
student performance in two college classrooms. Unpub- tives on research and practice. The School Psychol-
lished doctoral dissertation. Teachers College, ogy Review, 31, 328 –349.
Columbia University. Greer, R. D. (1994). The measure of a teacher. In R.
Bahr, M. W., & Bahr, C. M. (1997). Educational Gardner, D. M. Sainato, J. O. Cooper, T. E.
assessment in the next millennium: Contribu- Heron, W. L. Heward, J. Eshelman, et al. (Eds.),
tions of technology. Preventing School Failure, 41, Behavior analysis in education: Focus on measurably
90 –94. superior education (pp. 161–171). Pacific Grove,
Becker, W. (1992). Direct instruction: A twenty-year CA: Brooks/Cole.
review. In R. West & L. Hammerlynck (Eds.), Greer, R. D. (1996). The education crisis. In M. A.
Design for educational excellence: The legacy of B. F. Mattaini & B. A. Thyer (Eds.), Finding solutions to
TPRA / 421social problems: Behavioral strategies for change (pp. effective approach for evaluating teachers? Clear- 113–146). Washington, DC: American Psycholog- ing House, 75, 238 –241. ical Association. Lago-Delello, E. (1998). Classroom dynamics and Greer, R. D. (2002). Designing teaching strategies: An the development of serious emotional distur- applied behavior analysis systems approach. New York: bance. Exceptional Children, 64, 479 – 492. Academic Press. Lamm, N., & Greer, R. D. (1991). A systematic Greer, R. D., McCorkle, N. P., & Williams, G. replication of CABAS in Italy. Journal of Behavioral (1989). A sustained analysis of the behaviors of Education, 1, 427– 444. schooling. Behavioral Residential Treatment, 4, 113– Linhart-Kelly, R., & Greer, R. D. (1997, May). Rate 141. mastery and the long-term maintenance of sight Greer, R. D., Keohane, D., & Healy, O. (2002). words. Paper presented at the 23rd Annual Inter- Quality and comprehensive applications of behav- national Conference of the Association for Behav- ior analysis to schooling. The Behavior Analyst To- ior Analysis, Chicago, IL. day, 3, 120 –132. Retrieved December 22, 2003, Logan, K. R., & Keefe, E. B. (1997). A comparison of from http://www.behavior-analyst-online.org/ instructional context, teacher behavior, and en- BAT/BAT-32.pdf gaged behavior for students with severe disabili- Hawkins, R. P. (1991). Is social validity what we are ties in general education and self-contained ele- interested in? Argument for a functional ap- mentary classrooms. The Journal of the Association proach. Journal of Applied Behavior Analysis, 24, for Persons with Severe Handicaps, 22, 16 –27. 205–213. Logan, K. R., & Malone, D. M. (1998). Instructional Hersen, M., & Bellack, A. S. (2002). Dictionary of contexts for students with moderate, severe, and behavioral assessment techniques. New York: profound intellectual disabilities in general edu- Percheron Press. cation elementary classrooms. Education and Heward, W. L. (1994). Three “low-tech” strategies Training in Mental Retardation and Developmental for increasing the frequency of active student Disabilities, 33, 62–75. response during group instruction. In R. Gard- Madsen, C. H., Jr., Becker, W. C., & Thomas, D. R. ner, D. M. Sainato, J. O. Cooper, T. E. Heron, (1968). Rules, praise, and ignoring: Elements of W. L. Heward, J. Eshelman, et al. (Eds.), Behavior elementary classroom control. Journal of Applied analysis in education: Focus on measurably superior Behavior Analysis, 1, 139 –150. education (pp. 283–320). Pacific Grove, CA: Maranzano, C. (2000). Music teacher performance Brooks/Cole. evaluation: A call for more inclusive models. Jour- Hogin, S. E. (1996). Essential contingencies in correc- nal of Personnel Evaluation in Education, 14, 267– tion procedures for increased learning in the context of 274. the learn unit. Unpublished doctoral dissertation. McDonnell, J., Thorson, N., McQuivey, C., & Kiefer- Columbia University, New York. O’Donnell, R. (1997). Academic engaged time of Holland, P. E., & Adams, P. (2002). Through the students with low-incidence disabilities in general horns of a dilemma between instructional super- education classrooms. Mental Retardation, 35, 18 – vision and the summative evaluation of teaching. 26. International Journal of Leadership in Education, 5, McDonough, S. H., & Greer, R. D. (1999). Is the 227–247. learn unit a fundamental measure of pedagogy? Howard, B. B., & McColskey, W. H. (2001). Evalu- The Behavior Analyst, 22, 5–16. ating experienced teachers. Educational Leader- Miller, D. L., & Kelley, M. L. (1994). The use of goal ship, 58, 48 –51. setting and contingency contracting for improv- Ingham, P., & Greer, R. D. (1992). Changes in ing children’s homework performance. Journal of student and teacher response in observed and Applied Behavior Analysis, 27, 73– 84. generalized settings as a function of supervisor Montague, M., & Rinaldi, C. (2001). Classroom dy- observations. Journal of Applied Behavior Analysis, namics and children at-risk: A follow-up. Learning 25, 153–164. Disability Quarterly, Learning Disability 24, 73– 84. Juniper Gardens Children’s Project. (2001). EBASS. Moore, J. (2003). Evaluating teaching is important. Retrieved December 22, 2003, from University of Journal of Chemical Education, 80, 119. Retrieved Kansas, Juniper Gardens Children’s Project October 23, 2003 from http://JchemEd.chem- Web site: http://www.jgcp.ku.edu/EBASS/ebass_ .wisc.edu Moore, Z., & Bond, N. (2002). The use descrp.htm of portfolios for in-service teacher assessment: A Keohane, D. (1997). A functional relationship between case study of foreign language middle-school teachers’ use of scientific rule-governed strategies and teachers in Texas. Foreign Language Annals, 35, student learning. Unpublished doctoral disserta- 85–92. tion, Columbia University, New York. Munby, H. (1999). Planning, implementing, and Kumrow, D., & Dahlen, B. (2002). Is peer review an evaluating a field-based teacher education pro- 422 / Education and Training in Developmental Disabilities-December 2005
gram: An introduction. Teacher Education Quar- tion of behavior analysis to schooling. Journal of
terly, 26, 5–9. Applied Behavior Analysis, 24, 107–117.
Muyskens, P., & Ysseldyke, J. E. (1998). Student Smith, P. L., Harris, C. M., & Sammons, L. (2001).
academic responding as a function of classroom Using multimedia portfolios to assess preservice
ecology and time of day. The Journal of Special teacher and P-12 student learning. Action in
Education, 31, 411– 424. Teacher Education, 22, 28 –39.
Poling, A., Methot, L. L., & LeSage, M. G. (1995). Spicuzza, R., Ysseldyke, J., Lemkuil, A., Kosciolek, S.,
Fundamentals of behavior analytic research. New Boys, C., & Teelucksingh, E. (2001). Effects of
York: Plenum Press. curriculum-based monitoring on classroom in-
Protheroe, N. (2002). Improving instruction
struction and math achievement. Journal of School
through teacher observation. Principal, 82, 48 –51.
Psychology, 39, 521–542.
Quinn, K. P., & McDougal, J. L. (1998). A mile wide
Vaughn, S., Levy, S., Coleman, M., & Bos, C. S.
and a mile deep: Comprehensive interventions
(2002). Reading instruction for students with LD
for children and youth with emotional and behav-
ioral disorders and their families. The School Psy- and EBD: A synthesis of observation studies. The
chology Review, 27, 191–203. Journal of Special Education, 36, 2–13.
Rauch, K., & Whittaker, C. R. (1999). Observation Wallace, T., Anderson, A. R., & Bartholomay, T.
and feedback during student teaching: learning (2002). An ecobehavioral examination of high
from peers. Action in Teacher Education, 21, 1999. school classrooms that include students with dis-
Schwartz, I. S., & Baer, D. M. (1991). Social validity abilities. Exceptional Children, 68, 345–359.
assessments: Is current practice state of the art?
Journal of Applied Behavior Analysis, 24, 189 –204. Received: 11 February 2004
Selinske, J., Greer, R. D., & Lodhi, S. (1991). A Initial Acceptance: 4 April 2004
functional analysis of the comprehensive applica- Final Acceptance: 10 October 2004
TPRA / 423You can also read
