Critique: Assessment and the Production of Learning

Critique: Assessment and the Production of
Learning

           ELISABETH SOEP
           University of California, Berkeley

Education researchers and classroom teachers have argued that the constant pressure
to measure and rank students makes it difficult to shape assessment as an episode of
learning. Yet we know little about how learning moves in and through assessment of
any kind. Building on two national multisited studies, the research reported here uses
ethnographic techniques to examine learning within critique. Critique is a form of
assessment through which young people jointly judge their own work and that of their
peers. The article focuses on episodes of critique within two nonschool sites for col-
laborative production involving ethnically and economically diverse groups of
youth—a community-based video project and an organization in which young people
create radio stories for local and national broadcast. Learning environments such as
these draw voluntary youth participation and are organized around sustained projects
released to outside audiences. Findings indicate that critique manifests itself as an
episode of learning by engaging young people in joint assessment events that are
improvisational, reciprocal, and oriented toward the future of the work under review.
Intra- and cross-site comparisons suggest that critique is likely to arise within specific
conditions: when stakes are intense, metastandards are subjected to review, account-
ability is mutual and interactively sustained, and interdisciplinary practice is man-
datory. The article reviews various ways to conceptualize learning and argues in the
end for a theory of learning as production, a way of making. Implications include
new ideas for research methodologies and new understandings of youth-adult col-
laborations in learning and production.

Inside any given classroom within the United States, disparate forces, in-
cluding state standards, federal policies, legally mandated procedures, and
commercial products, shape even tiny moments of student evaluation.
Criticisms of classroom-based evaluation conventions are well established in
the education literature. Too much emphasis centers on measurement and
ranking and too little on how assessment can serve as an episode of learning
that is consistent with the larger pedagogical goals and practices of a given
classroom, while also attuned to the needs and dispositions of individual
children and youth.

Teachers College Record Volume 108, Number 4, April 2006, pp. 748–777
Copyright r by Teachers College, Columbia University
0161-4681

Critique: Assessment and the Production of Learning   749

   Despite these criticisms, the current sense of urgency around school ac-
countability makes it very difficult for even the most sensitive educators to
modulate how they evaluate their own students. Standardized testing pro-
cedures privilege discrete bits of knowledge over performances that reveal
active modes of knowledgeability. Although theorists may conceptualize
learning as inquiry or a process of situated participation, arguing that minds
develop through active engagement with meaningful tasks in specific con-
texts, rarely do mandated high-stakes evaluations focus on actual partici-
pation in anything other than a testing situation. Many reforms talk of
assessment as instruction, but the fact remains that we know very little about
how learning moves in and through assessment of any kind. The research
reported here examines learning within critique, a form of assessment
through which young people jointly judge their own work and that of their
peers as they prepare that work for public release.

         ASSESSMENT AS A MOMENT-TO-MOMENT ACTIVITY
Part of the difficulty in assessment reform stems from the fact that complex
and collaborative learning tasks do not lend themselves to straightforward,
easily replicable evaluation methods; such tasks require the production of
new knowledge and experimentation with received ideas. It is not easy to
track and compare student learning when young people work together on
creative problem-solving projects. Evaluation under these conditions tran-
spires not only through external appraisal but also, crucially, by way of
ongoing self- and peer-assessment by participants in the activity itself. Per-
haps we should be looking, therefore, inside learning events, to the dis-
course of young people themselves, as we develop new strategies for
assessment. Support for this line of argument is revealed in research cen-
tered on assessment as a moment-to-moment collaborative affair. For ex-
ample, in a recent study of young people involved in group work
assignments, Cohen and her colleagues (2002) found that the more stu-
dents evaluated their own work as it developed, using criteria presented at
the beginning of the task, the more motivated and task focused was their
discourse, resulting in higher quality products and more sophisticated
written reflections.
   Empirical evidence supporting the value of self- and peer-assessment is
critical in efforts to reimagine the role of evaluation in education. Stated
simply, it appears that the act of assessment itself is too often not sufficiently
participatory. Participatory evaluation would regularly place young people
in situations in which they must evaluate their own work and that of their
peers to move the work toward completion and ultimate release. Research
on how assessment messages circulate in everyday discourse beyond

750   Teachers College Record

classrooms advances this view. In traditional classroom formats, teachers
tend to control assessment, with students playing at best a passive role
(Cazden, 2001; Filer, 2000). In everyday conversation, by contrast,
assessment is often deeply participatory (Goodwin & Goodwin, 1992;
Linde, 1997). When one speaker utters an assessment in the course of
conversation, subsequent speakers tend to position themselves with respect
to the appraisal that they have just heard, by conveying agreement or ex-
pressing dissent.1 This process can be highly energetic, evident in flurries of
overlapping, densely populated stretches of speech, as in a phenomenon
especially conducive to collaborative problem-solving that Tannock (1998)
identified as ‘‘swarming.’’ In this sense, self- and peer-assessment can invite
heightened participation and represent a real educational opportunity to
engage students more fully in their own learning.
   Young people themselves, when faced with certain kinds of learning
tasks—specifically, those aimed toward the release of original, collabora-
tively produced material to an outside audience—are constantly developing
the very criteria that they will then apply to their work as they evaluate its
merits. This activity can be described as a process of critique—the joint
assessment of original objects and performances by producers themselves.

                           CRITIQUE IN THE ARTS
Models for critique exist in every field: Scientists evaluate one another’s
theories as they prepare for meetings and clinical trials, just as architects
formally review design plans before pitching accounts, athletes screen game
tapes to recognize and learn from the lessons of past performance, and
political teams endlessly debate the nuances of single words when compos-
ing speeches and slogans. However, conventions for critique derived from
the arts are especially relevant, given the highly established, if underrec-
ognized, rituals for self- and peer-assessment that occupy a prominent place
in arts education. Nonschool spaces for arts production in particular reveal
how young people use critique in settings that draw voluntary youth par-
ticipation—contexts organized around sustained creative projects culmi-
nating in public release of young people’s work. A moment of critique in
these kinds of settings might involve a group of young people who are
producing a video and stop in the middle of a shoot to assess and radically
revise the choreography for a scene that is just not working as it was orig-
inally imagined. Or picture a group of teen radio producers preparing a
special series on youth, money, and marketing for a national show. They are
discussing the subtleties and controversies embedded within a spoken-word
poem written by a 17-year-old as they consider the poem as a possible
element in that series.

Critique: Assessment and the Production of Learning   751

   These specific moments of critique took place within two different sites
for collaborative arts production: a youth video project called Cutaway, in
which young people produce movies screened in cinemas and museum
spaces, and a nonprofit media program called Youth Radio, in which young
people collaborate with adult producers on stories for local and national
broadcast.2 Representative episodes of critique in these settings contain
broader implications for the design of participatory, assessment-rich learn-
ing environments. Analysis of these interactive moments begins with a
definition of assessment as a process of seeing and responding to a given
piece of work, and critique as a practice of mutual assessment that young
people use when they turn to one another as resources for judgment. After
a brief review of literature on formal peer critique conventions in arts ed-
ucation, and a clarification of the conception of learning being applied and
advanced here, the analysis is organized around four questions:
  1. How does critique manifest itself in the process of moment-to-
  moment interaction? This question centers on the ways in which critique
  provides access to specific learning opportunities by giving young people
  responsibility for developing and applying judgment.
  2. What are the implications of critique for learning practice? This ques-
  tion draws attention to the conditions that promote critique.
  3. What are the implications of critique for learning theory? Response to
  this question suggests a conception of learning as production.
  4. What are the implications of critique for methodologies of research on
  learning? This final line of inquiry suggests that models for critique
  among youth contain lessons for the conduct of participatory, collabo-
  rative research.

             RESEARCH ON ASSIGNED PEER ASSESSMENT
There is a strong, though underrecognized, tradition within visual arts ed-
ucation, especially on a postsecondary level, of including learners in the
assessment of their own projects and those of their peers through some-
thing called a crit.3 A crit is a period of time, usually several hours, set aside
during an art class when students display work and discuss with instructors
and one another the strengths and weaknesses of each piece. Any mention
of crits to a group of practicing artists is likely to invite a spirited response.
When told about my interest in crits as sites of learning and assessment, one
painter recalled his professor hurling a beer can at his painting during a
crit. Another watched in dismay as an instructor tossed student paintings
out of a high-rise building window. I have had my own difficult, although

752   Teachers College Record

admittedly less dramatic, experiences in crits—for example, waiting mis-
erably until my monoprint was picked last in a crit in which students de-
cided in what order to consider individual pieces of work. It felt like being
the last kid chosen for the kickball team in gym class.
    Crits evoke other, highly positive narratives as well. Research based in
postsecondary settings reveals that many artists credit crits for motivating
their productivity by introducing intense deadlines, exposing them to new
techniques and interpretations, forcing them to refine their habits of per-
ception as related to production, and building their fluency in vocabularies
of art (Soep, 2000). Unlike criticism, in which outsiders to an art-making
experience describe and evaluate a given piece, in crits, artists themselves
make judgments about their own evolving work, and they find language to
describe what they see, like, and dislike in the efforts of others who are
learning alongside them.
    There is relatively little attention, however, to crits in education
scholarship (Bulka, 1996). The research that does appear tends to be in-
strumental, highlighting features of effective peer critique. These studies
often focus on university settings, calling for crits marked by a number of
auspicious characteristics, including discussion driven by aspiration
rather than ideological position (Rogers, 1996); close focus on specific
qualities of the work under review (Roth 1999); a climate of ‘‘spontaneity,
empathy, and equality’’ (James, 1996, p. 153); and a commitment to
‘‘provisionalism’’ in commentary offered in critique (Wernik, 1985). When
these qualities are not in place, research suggests that crits fail to inform
artists’ practices and judgments in any meaningful way, while making par-
ticipants feel as if they are ‘‘being hung naked on the wall and ridiculed’’
(Wernik, p. 194).
    Even fewer researchers have published refereed studies of critique in
K–12 visual arts classrooms. Barrett (1988, 1994, 1997, 2000) is perhaps the
most notable exception (also relevant are Blythe, Allen, & Schieffelin Po-
well, 1999; Cotton, 1981; Ende-Saxe, 1990; House, 2001). Barrett said that
crits among young artists should have a purpose, invite participation, in-
clude positive and negative reasoned judgments, emphasize interpretation
over evaluation, and privilege the viewer as the most important participant.
Facilitators of crits should be prepared, said Barrett, to ask and elicit good
questions and to address fundamental issues pertaining to intent, content,
subject matter, form, relationship of media to materials, sources of artistic
influence, and social issues. Barrett’s work and that of others in the research
literature provide a useful conceptual framework and a practical set of
principles for educators who facilitate crits in their classrooms.4 The exist-
ing literature also exposes some gaps in our understanding of the processes
of self- and peer-assessment as a property of learning. Specifically, we
know very little about how critique materializes spontaneously within

Critique: Assessment and the Production of Learning   753

collaborative undertakings when it is neither scheduled, nor staged, nor
assigned by anyone but young people themselves out of some kind of tacit
recognition that joint assessment is in order.

      CONCEPTIONS OF LEARNING THAT INFORM AND ARISE
                      FROM CRITIQUE
Moving from assigned crits to emergent critique entails a shift in focus from
preplanned feedback sessions to moments of judgment brought on by the
demands of collaborative production. This shift requires a working concep-
tion of learning, or, more specifically, what Cohen and Ainley (2000) called a
‘‘theory of cultural learning’’ (p. 92). Such a theory accounts for how young
people study and develop ‘‘school knowledge’’ and how they ‘‘learn to
dance, or knit, or make love, or ride bicycles, or horses, or play football, or
write graffiti or poetry, or tell jokes, or tall stories, practice safe sex, use
computers, play musical instruments, conduct experiments’’ (p. 92).5
    One well-established tradition for theorizing learning is by way of ac-
quisition models, which frame learning as the internalization of discrete
information that is then transferred to new contexts. Students memorize
dates, study rules and patterns, and master terminology and habits asso-
ciated with a given task or field, and then demonstrate those achievements
through systematic examination. Although this view may be intuitively use-
ful, contemporary learning theorists have raised concerns about acquisition
models, arguing that they make learning passive and equate knowledge to
an exchangeable commodity. Nevertheless, it has been argued that the idea
that students acquire and eventually transfer information continues to
dominate the assessment literature, at least implicitly, to the extent that
researchers do not always fully develop alternative theories of learning that
drive their analyses of assessment conventions and proposals for change
(Delandshere, 2002).
    On the opposite end of a theoretical spectrum from acquisition models is
a view that learning takes place simply by virtue of being alive and adapting
and adjusting to new conditions. Learning from this perspective is an
everyday affair—a way of handling the world and what happens within it. It
is less a bounded phenomenon that can be neatly isolated and tested than it
is a natural product of being in the world, taking in new information and
responding to it, all the while registering the ‘‘lessons’’ that arise out of lived
experience. This view is useful to the extent that it frames learning as an
ordinary practice, providing an important corrective to the tendency
to locate learning exclusively within settings where obvious and deliberate
instruction is taking place. Perhaps this conception is therefore a necess-
ary step in opening researchers’ eyes to the existence and significance of

754   Teachers College Record

learning environments within young people’s families, peer groups, neigh-
borhood streets, and media consumption practices, and in school class-
rooms. This view is, however, so vague and far reaching as to have limited
analytical utility.
    A third view reconciles an interest in learning as a process of acquiring
information, with an interest in learning as a property of everyday expe-
rience, while at the same time offering ways to account for the creation of
new knowledge. This third view says that learning is a process of situated
participation, in which new information enables people to reorganize their
relationships with one another and with respect to their joint projects. In
other words, learning means shifting the conditions that shape minds, cir-
cumstances, actions, and products. In this situated view, knowledge is co-
constructed within communities of practice (Chaiklin & Lave, 1996;
Greeno, 1998; Lave & Wenger, 1991; Papert, 1991; Rogoff, 1994; Varenne
& McDermott, 1999; for recent critical discussions, see Anderson, Reder, &
Simon, 2000; Fox, 2001; Hacking, 2000; Sfard, 1998; Young, 2000). This
notion of learning is a useful starting point, although this view will not be
where the analysis ends. Study of critique pushes this situated participation
model in a new direction—toward a view of learning as production, a way of
making. Conceiving of learning as production provides new theoretical
tools to help us work through some of the questions that the situated par-
ticipation perspective leaves open—specifically, how learning migrates from
one situation to another or, more to the point, how a single situation can
itself contain multiple imagined contexts. Critique is a key process whereby
learners produce these imaginary contexts by conjuring hypothetical
scenes, responses, and consequences for the decisions made within the
process of production. In this sense, as we will see, assessment can do more
than measure learning. It can serve as the fuel, so to speak, that moves
learning forward beyond a single situated moment in time.

                     SITES AND SCENES OF CRITIQUE
This argument for a particular way of theorizing learning surfaces from
scenes of interaction within two nonschool sites for creative media produc-
tion, Cutaway and Youth Radio. A focus on settings such as these is con-
sistent with a growing interest within the field of education in learning,
assessment, and youth development beyond school walls (cf., Campbell,
2001; Heath & McLaughlin, 1993; Hogan, 2002; Hull & Schultz, 2001,
2002; Mahiri, 2003; Noam, Biancarosa, & Dechausay, 2002; Paley, 1995;
Soep, 2002). Research in this area acknowledges that young people spend
only a relatively small percentage of their time inside classrooms and that
the best community-based projects seem able to draw active, committed,

Critique: Assessment and the Production of Learning   755

and highly impressive performances of learning among youth. Language
development appears to extend well into later childhood and adolescence
through extended involvement with community-based projects over sus-
tained periods (Heath, 1998b).
    These observations do not, of course, mean that schools matter less,
only that life beyond school has the potential to reveal new insights relevant
to the design of learning environments likely to draw youth engagement
and high quality performance. The same students who might struggle
in a remedial English class write and perform spoken-word poems that
bring critical audiences to their feet. Young people with sporadic attendance
in classrooms or those who have dropped out of school entirely will
consistently attend nonschool projects such as media programs, theater
collectives, or aerosol art crews, moving into leadership positions and
making long-term commitments, advancing their own skill levels while
pushing the arts that they practice in new directions. Evidence of youth
engagement and achievement along these lines is reported in two national
studies that greatly inform the present analysis, through my own partic-
ipation in those studies as a researcher and ethnographer. The first
is Harvard University’s Project Co-Arts, a study of pedagogy, assessment,
and neighborhood-based development within community arts centers in
low-income settings (Davis, 1998; Davis, Soep, Maira, Remba, & Putnoi
1993). The second is a Stanford University-based study (1987–1997)6 of
youth learning and leadership in community-based organizations, which in
1995 began to focus on arts-based sites, given the especially notable contexts
and outcomes for learning that these settings appeared to provide (Heath,
1993, 2001; Heath & Ball, 1993; Heath & Soep, 1998; Soep, 1996, 2000,
2002, 2004, in press).
    The organizational and interactive features linked to effective
community-based youth organizations are well documented in the existing
literature—for example, opportunities for intensifying youth participation
and leadership, and sustained projects organized around cycles of planning,
preparation, practice, and performance (Heath & McLaughlin, 1994a,
1994b). Some investigators urge schoolteachers to forge relationships with
community projects, suggesting that classroom-based educators might
profitably model some aspects of the kinds of learning environments that
take hold more commonly beyond school walls. Others point to the con-
siderable structural barriers that make these kinds of relationships very
difficult, given the compulsory, highly regulated character of education in
school. Overall, it seems important, as we pay more and more attention to
community-based learning environments for youth, not to imply that an
impenetrable boundary separates them from schools (Hull & Schultz,
2002). Such a boundary does not always exist for young people themselves,
and innovative educators in both realms of practice very often share a

756   Teachers College Record

commitment, and also the capacity, to create conditions in which youth
serve as producers and judges of their own development (Weis & Fine,
2000).
    Moreover, researchers are increasingly reckoning with the dangers of
romanticizing nonschool learning environments as some kind of educational
panacea. It is all too easy to portray these places as ‘‘free’’ spaces where
young people can escape (or, more problematically, be rescued from) the
forces of inequality that they face in other institutional settings, including
school (Hull & Schultz, 2002). Hogan (2002) identified the ways in which
power dynamics can undermine community-based learning by shutting
youth out of meaningful involvement, limiting their participation to menial
tasks and potentially doing more harm than good, as yet another example of
an experience whose reality looks quite different from the ways in which it
was initially presented to youth participants. Trend (1997) considered arts-
based community projects in particular, pointing to the striking unevenness
among such projects in terms of product quality and the depth of youth
engagement; his analysis suggests that young people are often invited into a
project to provide autobiographical content and access, but they may have
little say over how, where, and to whom the work is actually presented. In
her study of a youth media project, Fleetwood (2005) sharpened and ex-
tended this point about the limits to youth involvement in community-based
learning, and she cautioned that such projects have a tendency to pursue the
fantasy of an authentic youth voice, which itself often embodies a sensa-
tionalized portrayal of racialized ‘‘urban’’ youth experience.
    What is striking in the literature highlighting the positive potential and
the pitfalls of nonschool learning environments is the extent to which as-
sessment seems to be a kind of pivot point. Effective community-based
projects engage young people in the process of judging their own work and
that of their peers, and indeed this level of participation in assessment is a
hallmark of high-quality sites (Heath & Soep, 1998). At the same time, many
of the problems that can arise with these projects are revealed in the ways in
which young people are excluded from the process of formulating judg-
ments and acting on them—when they are called upon, for example, to tell
their stories in graphic detail but then dismissed when it comes time to
make editorial decisions and transform confessional material into refined
works. It is therefore especially relevant, given these current debates in
educational practice and research, to look closely at the assessment practices
that take place within such projects, to illuminate how learning happens
there and how learning can be theorized in a broader sense. It is important
to recognize that critique finds no easy path into school processes and
products, for the institution is not set up to take assessment from students
into account within high-stakes measurements such as standardized tests.
Nevertheless, classroom educators who practice performance-based

Critique: Assessment and the Production of Learning   757

approaches to assessment within the everyday flow of their teaching regard
evaluation as an episode of learning. They create ‘‘real-world’’ events in
which young people perform ‘‘mastery’’ of a given subject for outside au-
diences and panels, and in preparing for these performances, students of-
ten find themselves in a position to critique their own work and that of their
peers (Darling-Hammond, Ancess, & Falk, 1995). Daily assessment inside
schools is a social practice in which students can be active agents (Filer,
2000); critique is therefore a powerful resource for teachers who see young
people as partners in the production of learning.
    The research on nonschool learning environments reported in this ar-
ticle has evolved through two phases. Building on the national studies noted
previously, the first phase, from 1991 to 2000, included study of youth
learning within more than 20 community-based organizations, including a
teen drum corps, an improvisational theater troupe, a mural and graphic
arts project, a zine-writing group, and a video production project (Cutaway)
in cities in New England, the southeastern United States, and on the West
Coast. These last two sites were studied in particular depth for comparative
purposes, including, in the case of Cutaway, field work over 1 year that
entailed more than 190 hours of intensive data collection and tape record-
ing to yield a language corpus for discourse analysis.7 This phase of the
research, although centered on projects facilitated by adult leaders, focused
especially on moments of critique among youth as they formulated judg-
ments of their own work and that of their peers. My own participation as a
researcher in these moments of critique was minimal because it would have
seemed problematic, given the nature of the research, for my own habits of
critique to shape the phenomenon central to the study. In the second phase
of the research, from 2000 to the present, this emphasis on observation with
little participation gave way to a different arrangement entirely. Phase two is
an action-research project within one youth media organization, Youth Ra-
dio, where I work actively with young people to produce radio stories for
local and national broadcast. My own voice is, therefore, one among many
in the exchange of critique. This phase of the research looks more closely at
the workings of critique between young people and adults who are mu-
tually, if asymmetrically, engaged in a project for which both parties will be
judged once the broadcast airs to significant audiences.
    The analysis offered here focuses on one episode of critique from each
phase of the research, the first based at Cutaway and the second at Youth
Radio. These episodes were selected for their representativeness; these
were not random, nor were they exceptional occurrences. There were, to
be sure, long stretches of time within both research sites when no critique
among youth could be heard: when instructors lectured on a particular
topic or demonstrated a new piece of equipment, or when young people
worked silently in a studio or edit suite. Episodes of critique such as the ones

758   Teachers College Record

that follow predictably arose when the process of production itself required
a moment of display or performance—when a young video maker had to
present her script to the actors who would be playing the parts, or when a
radio producer was recording his narration in an announce booth and the
peer who was engineering that session provided feedback on voicing tech-
nique. These ‘‘naturally occurring’’ assessment junctures took place over
the entire course of a production project, from the earliest stages of idea
generation, through the final hours before the work would be released, to
the aftermath of an exhibition or broadcast. It grew especially intense when
some kind of conflict arose—when participants needed to make a crucial
decision—and there was disagreement as to the best way to proceed. Within
these kinds of moments, critique made its strongest appearance as a re-
source for identifying and solving problems through a process of assess-
ment, in this sense significantly repositioning assessment as a practice that
fuels, rather than merely measures, learning. It is important to clarify,
however, that conflict is defined not necessarily as an acrimonious encoun-
ter, but instead as a moment when young people face some obstacle in
moving their work forward and when there is no immediate consensus on
how to proceed, nor is there an adult who single-handedly can make that
judgment on their behalf.

SCENARIO 1: CRITIQUE AMONG YOUTH AT CUTAWAY

Founded in 1995, Cutaway is located in a major West Coast city and involves
primarily working- and middle-class youth of color and led by a White artist
from a working-class family.8 In the specific Cutaway project highlighted
here, 8 young artists ranging in age from 14 to 18 spent one summer and
fall producing an original movie, acting and serving as crew members on
one another’s productions. Their individual videos were ultimately pro-
jected simultaneously in a video installation, meaning that they had to make
sense as standalone narratives and also as part of a total visual environment
in the gallery where the work premiered. While learning the basic technical
skills of video production, the young people spent several weeks brain-
storming ideas for their individual movies and narrative and visual themes
that would resonate across all the work, lending coherence to the overall
installation. Lila, the adult artist in charge, launched the project with a loose
suggestion that perhaps the group might want to do movies that related in
some way to their neighborhoods, but by the time the young people had
developed scripts and begun shooting, they had diverged considerably
from that initial point of departure. There was one mockumentary about
flirtation rituals between boys and girls, an experimental documentary on
public art, and an earnest, soap opera-like love story about romance across
race and class, to name a few. Every phase of the process that led up to these

Critique: Assessment and the Production of Learning   759

completed movies was punctuated by episodes of critique in which the
young people reviewed one another’s works, discussing and debating
strengths, weaknesses, and possibilities for change.
    One particularly heated instance of spontaneous critique arose during a
middle-of-the-night shoot, when Simon, a 17-year-old public high school
senior, was directing and starring in his own fictional movie about hostility
between two generations of Asian American youth. It was after midnight.
The group gathered in an urban alley, running extension cords out of
adjacent apartments, rigging lights, mics, and camera. Everyone in the
group was exhausted and frustrated. The day had started at 9:00 a.m., and
more than 15 hours later, it became apparent to the full group that Simon’s
fight scene was falling apart. It reached a point where Lila, the adult artist,
‘‘opted out,’’ as she said. She told the youth cast and crew that they were
on their own to figure out how to rework the scene and started packing
up the set.
    Within this tense environment, it came time to shoot the pivotal moment
in Simon’s piece: when his character gets into a physical struggle with an-
other character, played by Chanpory, at which point a gun accidentally fires,
killing Simon’s character’s brother (played by Simon’s real-life brother,
Bryant). While all this action was happening on camera, a great deal of work
was also taking place behind the scenes. Cleo scrutinized the action on a
field monitor, a portable television that allows video makers to see what
their footage looks like on screen. Claire operated the camera. Lydia, an-
other member of the group, watched and waited. Claire called action.
Simon and Chanpory lunged toward each other and began the struggle,
with Claire rolling tape until Cleo stopped everything abruptly, saying ‘‘I’m
telling you, Simon, it’s not working. This is not working.’’ Simon at that
point was still focused on Chanpory, telling him exactly how to pull his arm
to make him fall, but Cleo was insistent, calling Simon’s name repeatedly
until he finally turned to her, even as Lydia, in the background, muttered
under her breath, ‘‘I feel like screaming.’’ Finally with Simon’s attention,
Cleo began to explain what she had seen. ‘‘All right, when I was looking at
it, it looked really weird, because—okay, what do you want the people to
see? You wanna see all four of you guys, or just two?’’ Simon and Chanpory
immediately began answering simultaneously, and even after Cleo told
Chanpory to let Simon talk (it was his movie after all), Chanpory continued
to direct the scene from his own point of view. ‘‘No! I don’t want to hear
from you!’’ she said, putting her hand up over Chanpory’s face, saying ‘‘I
don’t even want to hear from you.’’ Chanpory laughed and backed off,
letting a surprised-sounding, ‘‘Sorry!’’ pass through his lips. Simon then
explained his original vision of the scene, a mix of medium shots and wide
shots, from various angles. Cleo repeated his descriptions, making sure she
understood what he had in mind and prompting him to go on:9

760    Teachers College Record

1. Cleo:                 Okay, then what? That’s okay. We’ll do that on the
                         close up, so then, this is what, when you guys are
                         fighting, you make sure Bryant is facing—
2.   Bryant:             [Yeah, I did. I turned him around.
3.   Cleo:               [these people, because I only saw your back.
4.   Bryant:             You have to get up though.
5.   Chanpory:           Yeah, you guys have to find a way to get up.
6.   Cleo:               And make it fast, okay?
7.   Bryant:             All right, [after you get shot, lay down on the floor
                         away from the camera.
8. Chanpory:             [Okay. When you come after me, focus on this hand,
                         and just put both hands—
9. Cleo:                 And you guys, you guys, you guys, you’re not fight-
                         ing. You’re acting. So let the other person take con-
                         trol, if that has to be, you know what I mean. Don’t be
                         struggling, cause that looks stupid.

        QUALITIES OF INTERACTION: ACCESS TO JUDGMENT

A fleeting exchange such as this might easily be missed by observers looking
for obvious or tightly organized evaluation events (Lee, 2001). And yet this
kind of episode, as representative of a process that took various forms across
the production cycle of the Cutaway project, reveals how critique manifests
itself within moment-to-moment interaction, providing access to specific
learning opportunities by giving young people responsibility for developing
and applying judgment.10
   First, this is an episode of joint assessment, meaning that young people
were turning to one another to see the work unfolding before their eyes in
new ways, evaluating its merits and pointing to its shortcomings. Critique is
often shot through with obvious evaluative terminology—what linguists call
assessment tokens—in this case, ‘‘It’s not working,’’ ‘‘It looked funny,’’ ‘‘That
looks stupid,’’ or even ‘‘I feel like screaming.’’ Youth in critique can also
convey assessment in less explicit ways—by silence, for example, or through
interpretations that imply respect for the work under review, or with the
unspoken ratification of uptake. In making these kinds of moves at this
juncture within Simon’s shoot, young people are constituting a circle of
decision makers at the precise moment when the adult in charge has stra-
tegically, if temporarily, vacated her position. The learning dynamic in place
here is fueled in part by the mutual understanding that ultimately the work
must be completed in a timely manner (before they are kicked out of the
alley), and that the finished product will be viewed and judged by a critical
outside audience.

Critique: Assessment and the Production of Learning   761

   Particularly striking in the data both within Cutaway and across research
sites were instances in which young people dramatized narratives invoking
anticipated audience members, and other imaginary interlocutors, enacting
how those hypothetical figures would respond to the work. They conjured
characters and then voiced their own assessments as if they came through
that character’s mouth. In this excerpt, Cleo strategically alluded to what
Simon wanted ‘‘the people’’ to see, aligning her own questions with a
broader audience view. In this sense, she amplified her voice, re-
contextualizing the present decision—which was being made among peers
and collaborators in an urban alley—by projecting the consequences of that
decision onto an exhibition that would ultimately be assessed by a much
wider public. The young people themselves carried responsibility for judg-
ing what they wanted those people, that projected audience, to see. It must
be stated, however, that it was only after months of working together that
the adult artist could leave the young people to resolve this crisis; they had
developed habits of critique and a shared understanding of one another’s
projects, which allowed them to persevere and see both weaknesses of the
work and possibilities for salvaging Simon’s vision.
   Second, critique entails moment-to-moment improvisation. Cleo, Simon,
and the others did not have the luxury of planning out in advance how to
manage this particular microcrisis of representation. With little time to
prepare what they would say, the young people improvised in the same way
that jazz musicians do on stage—by way of simultaneous composition and
performance, making ‘‘in-flight’’ decisions about problems without prefig-
ured correct answers (Eisner, 1992; Thibault, 1997; see also Lee, 2001).
They could also immediately sense how interlocutors were receiving their
feedback and spontaneously adjust their commentary. Critique is therefore
not only iterative but responsive, evolving out of the specifics of a given
group within a particular context. Enormous planning went into Simon’s
movie, but not every detail was predictable. The young people necessarily
had to improvise their work on the movie, in large part by exchanging and
acting on impromptu assessments. This particular situation was a kind of
worst-case scenario—Simon’s carefully choreographed vision did not trans-
late to the screen. Improvising one’s way through that kind of crisis without
an absolute authority who could come in and save the day meant calibrating
the critique to the dynamics of that precise moment. Never before in the
Cutaway process had someone summarily stopped someone else from talk-
ing (by inserting a hand in front of another person’s face) as Cleo did here.
But she apparently sensed that the urgency of the present situation neces-
sitated that move, and the ensuing interaction appears to suggest that she
was not mistaken. Along with this example operating on the level of in-
teraction, there were also technical skills that allowed these young people, in
this moment, to improvise their way through this instance of critique; their

762   Teachers College Record

mastery of vocabulary such as wide shots and medium shots allowed them
to speak in a kind of shorthand through which they could express, evaluate,
and finally execute the scene. Critique, in a broader sense, entails impro-
vising a course of action out of the often conflicting positions circulating
within the group. Youth work toward a standard that emerges from their
collaboration with a professional and their developing knowledge of models
of ‘‘good work.’’
    Reciprocity is a third defining feature of critique as an opportunity for
learning through the exercise of judgment. Within this moment during
Simon’s shoot, the young people exchanged perspectives, identified prob-
lems, and proposed solutions, not always harmoniously. Cleo edged into a
directorial role, even in a movie technically directed by one of her peers.
Once this episode passed, however, the dynamics of reciprocity shifted.
Later this same night, the group began shooting a scene from Cleo’s movie,
and soon they were critiquing her work. They energetically debated wheth-
er it would be clear that her cop character, who spouted antiyouth rhetoric,
was not expressing Cleo’s own views, but exposing the perspective that she
wanted to condemn.
    Reciprocity is what distinguishes critique from one-way structures of
evaluation: Someone from outside an experience passes judgment on the
work without that person’s own efforts also being subjected to feedback
from the group. Reciprocity also differentiates critique from criticism, in the
literary or aesthetic sense; reviewers compose evaluations of completed
work without ever actually having to come face to face with author or artist,
who may or may not even still be alive. All participants in critique subject
work to joint review; there is a mutuality of engagement and vulnerability to
the judgment of others.
    Fourth and finally, in critique there is an orientation toward emerging
work. Although participants in critique often (sometimes to a problematic
extent) look back toward the intentions that drive certain creative decisions,
they consider where the work comes from to determine where it needs to
go. Within this scene from Simon’s shoot, the young people repeatedly
projected a sense of future for the work and for their decisions even as they
entertained ‘‘past tense’’ considerations—including what Simon had orig-
inally envisioned for the piece and how specific actions registered on screen.
The youth projected what they could and could not do, forecasted how
certain moves would look, and asked questions to clarify planning for the
next take. By projecting an imaginary sense of future for the work, young
people in critique edge toward actual completion of a given project while
also developing a broader sense of how that which they have seen and
learned in one project might apply to new endeavors in related fields.
    The imprint of critique can be seen on finished products through the
adjustments that authors have actually made on the basis of peer feedback,

Critique: Assessment and the Production of Learning   763

and evidence in some cases that authors chose not to bend to every judg-
ment offered by colleagues and collaborators. In the case of Simon’s movie,
the change that came over the work as a result of critique was evident in the
contrast between the ways in which his peers initially reacted to his scenes,
with considerable negative feedback and frustration, and the response that
he received at a rough-cut screening, just days before the final exhibit,
attended by Cutaway youth and an adult professional editor consulting on
the project. When it came time to watch the fight scene and accidental
shooting, some in that audience were nearly speechless, others offered
emotional praise, and more than one person in that room actually teared
up, having been swept into the drama of Simon’s story playing out on the
screen.

SCENARIO 2: YOUTH AND ADULTS IN CRITIQUE AT YOUTH RADIO
The episode of critique drawn from Cutaway gives a picture of what
the process can look like and what learning opportunities can open up
when young people themselves manage the process of judgment. This
second example of critique shares the defining features identified above: It
is an instance of joint assessment, an improvisational, reciprocal process
oriented toward the future of the work under review. However, in this case,
the dynamics of critique include young people in a relationship with adults;
both parties depend on each other, in obvious and subtle ways, to complete
work that meets professional standards. Episodes of critique involving
young people and adults take place almost daily at Youth Radio, where
adult producers such as myself collaborate with youth reporters on stories
for local and national broadcast, most notably on National Public Radio.
Young people come into the program from San Francisco Bay Area public
schools, and 80% are working-class youth and youth of color—not the
voices typically dominating public airwaves, particularly as reporters, com-
mentators, cultural critics, and analysts. An ethnically and economically
diverse staff brings diverse professional experiences to the work, including
backgrounds in broadcast journalism and music programming, youth
development, activism, and education.
   The young people come to their first class at Youth Radio on a Wednes-
day, and by Friday, they are on the air for their live show in which they DJ
music segments, write and deliver commentaries, announce the news, and
produce public service announcements. Young people who are especially
interested in audio narrative pursue advanced training and eventually ar-
rive in Youth Radio’s newsroom, the primary source of the organization’s
national stories. Youth Radio has aired national stories in the last few years
on topics including sexuality in middle school, standardized testing boycotts

764   Teachers College Record

in low-income districts, minors who volunteer as subjects in medical re-
search, math requirements as a barrier to college retention, and commen-
taries from Palestinian and Israeli youth living through the crisis in the
Middle East. Youth reporters meet weekly in editorial meetings, where they
pitch story ideas and frame narratives around characters, ambient sound,
archived media, and music. The process of actually producing a story in-
volves gathering tape, writing a detailed script, recording narration, dig-
itizing these materials, and using editing software to mix the piece. Young
people carry out each of these phases with hands-on guidance from adult
producers. Peer critique is built into a place like Youth Radio on multiple
levels—in editorial meetings, for example, or when two people go out to-
gether to get tape for a story, or when one person is engineering another
person’s piece. There is also a longstanding model of peer teaching at Youth
Radio, where young people serve as editors, voice coaches, and studio
mentors to others just learning new skills. Within the apprenticing culture
of Youth Radio, these peer teachers are simply more experienced youth
who have been working for some time at Youth Radio, reaching the point at
which they are prepared to educate others; in some cases, their students are
older than they are.
    Recently, Youth Radio solicited a poem from 17-year-old Rafael Santiago
Casal as part of a series that the newsroom was producing for public
radio on youth, money, and marketing. Rafael writes and performs
with a spoken-word poetry program called Youth Speaks, for which
I serve on the education committee of the board. Rafael delivers his poem
in a head-spinning rush of words and images, peppered with sexual
metaphors and profanities (which are, at least to this listener, neither
gratuitous nor predictable). This particular poem is a full-frontal attack on
the style industry, which he says uses child labor to commodify and sell
youth culture back to kids at a price that they cannot afford. Rafael starts
with a mocking reference to ‘‘the man with fashion sense’’ who had the
bright idea to tell kids what they need, creating ‘‘a million martyrs’’
identically dressed in sagged jeans, major league team jerseys, silver chains,
and $200 shoes. Later in the piece, Rafael moves from ‘‘the man’’ as the
target of his condemnation to a corporation he genders as a ‘‘damn fine’’
female who offers sex with just this catch: ‘‘you got to brand it and
lavish and ravage your own/image after we ho ya and handle your own
every dollar/until everyone marches to the same beat.’’ He links these media
manipulations to a kind of ‘‘mental incarceration’’ not unrelated to the
forces that ‘‘ended in the unfortunate/substantial transformation of our
pentagon into a quadrilateral.’’ And finally Rafael concludes, in this final
passage, with an image of his own body controlled by the manipulations he
describes—an ironic message, given the penetrating critique he has just
produced:

Critique: Assessment and the Production of Learning   765

  From BET to MTV we got PYTs11 marketing sex appeal the
  all mighty breast appeal
  It sells, yes but reflects poorly on the women ya feel
  Got us thinking woman? Oh you mean woo-man here to woo
  the man from the man to the man just to do the man lie
  down dick ride and screw the man please she fuckin
  rules the man this is just the plan they planted
  embedded empowering them through the image in women
  and men til corporate America owns my dick, they tell
  it where to be what to do and how high to go so
  frequently that i don’t own it anymore I feel lied to
  We move to their currents like a tide pool
  A million martyrs marching to the same beat
  What an eye full

   Public radio listeners will know that this is not the kind of narrative most
often broadcast on those airwaves. Youth Radio producers were struck by
the power of the poet’s message and the intricacy and lyricism of his im-
agery. We also had to consider the broadcast standards we would face in
pitching the poem to our outlets’ editors, who sometimes traffic in more
straightforward reporting using established news formats. Through e-mail,
we shared our feedback with Rafael. He would need to consider shortening
the poem simply because of the slots allowed for ‘‘creative’’ segments such
as this one on a traditional public affairs show. Federal Communication
Commission guidelines prohibit cursing and graphic sexuality, and we sug-
gested that there were some passages whose subtlety might be hard to
follow for radio listeners on their afternoon commute.
   In response to this e-mailed critique and suggestions for revision, Rafael
basically declined to participate. Perhaps we had missed the message of the
poem, he said, which was about media manipulation of a personal truth.
Was that not what we were asking him to do by requesting significant edits?
The poet’s response set off an exchange of subsequent e-mails and face-to-
face conversations, whereby he and we figured out a way to move forward
with the pitch of his piece, given our editorial considerations and his
insistence on the integrity of the message.

766   Teachers College Record

   This episode begins to suggest the relationships between youth, adults,
youth-serving institutions, and public cultures that enter and help shape
critique. What drew our interest was the content of Rafael’s poem, from a
vantage point of a listener, as a curator of youth voices. But Rafael, through
his response to our critique, brought us inside his message, as he says—
forcing reflexivity about the practice of critique, its functions and limits. The
learning that arose here—for Rafael, and for the others involved in this
exchange—inserts adults inside the learning opportunities made possible
through critique. Young people and adults are jointly and reciprocally as-
sessing one another, through moment-to-moment improvisation, always
with an eye toward a future for the unfolding work. The adults involved—
their own habits, aims, and prospects—are very much embedded within
these conditions, in any context in which the logics of youth development
and professional production meet through the demands of mutual assess-
ment. The version of the poem Rafael ultimately recorded for Youth Radio,
which eventually aired on a weekly public affairs show, provides evidence of
the impact of this episode of critique. In this case, however, unlike the
Cutaway example, it is the absence of change in the finished product that is
most striking and significant. The substance of the piece did not shift, and
only the profanities, which would have disqualified the poem from broad-
cast on any network, were eliminated from the artist’s original.12 However,
the learning that developed for the young people and adults who partic-
ipated in these negotiations has lingered beyond this particular encounter,
informing subsequent moments when it has come time to make judgments
about work that might push listeners to listen, and think, and see their
worlds in new ways.

      IMPLICATIONS FOR LEARNING PRACTICE: CONDITIONS
                       FOR CRITIQUE
Episodes of mutual assessment drawn from Cutaway and Youth Radio
provide a sense of the distinct learning opportunities made possible
through critique, as evident in the nuances of interaction and in the details
of finished products. However, this kind of analysis cannot sidestep an im-
portant set of questions: What guarantee do we have that critique will steer
youth down the right path? Specifically, would it not have been better to
have a professional choreographer on hand to monitor Simon’s fight scene
and simply place those bodies where they belonged? Is it not possible to
argue that part of the frustration everyone felt during that midnight shoot
was that everything at that moment was up for joint debate? Likewise, in the
case of Youth Radio, would it not it have been much more efficient had the
young people and adults involved in the critique surrounding Rafael’s

Critique: Assessment and the Production of Learning   767

poem simply taken that piece as it was from the outset, without getting into
the negotiations that in fact ensued? At its worst, cannot critique devolve
into something like a focus group, where every artistic decision passes
through a consensus-generating process that ultimately compromises idi-
osyncratic approaches and brave departures of imagination?13
    The simple answer to these questions is yes. Critique can be misguiding,
whether young people are alone in critique or formulating judgments with
adults. The same can be said, of course, when adults are engaging in their
own joint creative pursuits. When scholars edit anthologies, or submit ar-
ticles to refereed journals, or coteach classes, or plan conferences, it is al-
ways the case that joint assessment might ultimately steer individuals and
groups in ill-fated directions or yield disparate and contradictory recom-
mendations. And yet as young people learn to negotiate episodes of joint
assessment, they develop habits and strategies for judging the quality of
their own and their peers’ work. They need practice in the exercise of
judgment if they are to realize that they need not rely exclusively on an
outside authority with the absolute power to evaluate their performances
even if ultimately, other people’s assessments matter deeply to them and
help determine the fate of their work.
    There is also a second, and more complicated, answer to the question of
how we know that critique will actually have a positive rather than negative
effect on a given project or learning environment: to identify the conditions
associated with critique as a necessary practice rather than contrived activity.
Ironically, these conditions evoke and invert some of the principles most
commonly associated with standardized testing in schools. Four conditions
emerged from my analysis across research sites where critique does and
does not take hold. Comparison involved systematically reviewing tran-
scripts and other records of interaction over the course of full production
cycles at the research sites to identify under what circumstances, within
specific programs and across different organizational models, critique tend-
ed to emerge among youth and between young people and adults. Critique
erupts as a resource for learning when:

1. STAKES ARE INTENSE

This observation is counterintuitive in the sense that we tend to position
nonschool activities, especially those based in the so-called expressive arts,
outside the academic subjects prioritized in school. English and math classes
are what really matter, the logic goes, and the beauty of the arts, and ‘‘ex-
tracurricular’’ environments in general, is precisely that they offer a kind of
oasis from the pressures that young people face in more rigorous school
subjects (Siegesmund, 2002; Soep, 2002). But in nonschool projects in
which young people choose to participate, the stakes are intense, and they