A Comparison of Chat Room Productivity: Robert Sanders 59
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Robert Sanders 59
A Comparison of Chat Room Productivity:
In-class Versus Out-of-class
ROBERT SANDERS
Portland State University
ABSTRACT
Language production of Spanish students using synchronous CMC (chat) dur-
ing class time was compared with that of students meeting outside of class. The
study included about 100 participants enrolled in 10 sections of a first-quarter
Spanish course at a US university. Students in the control group spent 30 minutes
of their weekly class time in computer labs completing a chat room assignment.
Students in the experimental group planned their own schedules for meeting 30
minutes each week, outside of class time, in chat rooms. Transcripts of the chat
sessions were analyzed for duration, turns, words, vocabulary, socially appropri-
ate comments, and comments off task. Production was greater when students met
with their own work groups outside of class. The implication of the study is that
greater student collaboration and responsibility result in greater production while
conserving class time and technology resources.
KEYWORDS
Chat, Computer-mediated Communication, Synchronous CMC, Distributed Education
INTRODUCTION
Several researchers have addressed the effectiveness of out-of-class email journ-
aling in comparison with in-class, paper dialogue journaling (see González-Bue-
no and Pérez, 2000; Pérez-Sotelo & González-Bueno, 2003) and with in-class
chat discussions (see Pérez, 2003). This study compares in-class and out-of-class
production in WebCT chat rooms by students enrolled in an introductory Spanish
course at a US university. The purpose of the study is to evaluate out-of-class,
electronic contact as an alternative to some face-to-face contact. Out-of-class par-
ticipation could provide significant cost savings through distributed resource uti-
lization. The university in question is struggling to provide students basic Spanish
instruction at a time when enrollment growth far outstrips available resources.
Its introductory Spanish program focuses on oral proficiency and uses class time
primarily for communication in the target language. Prior to implementation of
chat discussions, existing research was reviewed for indications of proper activity
design.
CALICO Journal, 24 (1), p-p 59-76. © 2006 CALICO Journal60 CALICO Journal, Vol. 24, No. 1 REVIEW OF LITERATURE Benefits of Computer-mediated Communication Communicative interaction and production have been repeatedly addressed in SLA research on computer-mediated communication (CMC). Many studies (e.g. Beauvois, 1992, 1997; Biesenbach-Lucas & Weasenforth, 2001; Blake, 2000; Kern, 1995; Sotillo, 2000; Warschauer, 1996) approach communication through measures of formal features that transcend oral and written modalities, such as t- units, clauses, cohesive devices, turns, text length, words, authority, collaboration, participation, equality, and opportunity. These approaches have proven effective for analysis of online L2 communication and have been greatly facilitated by the transcription features of most CMC software. Hypotheses about output (Swain, 1985), learner collaboration (Bayer, 1990), and learning communities (Meskill, 1999; Wenger, 1998) are frequently cited to explicate the role of communication in SLA. These hypotheses correlate increased contact, production, and practice to increased acquisition. Case studies have found that text-based CMC is effective for increasing student production by facilitating greater student participation (Beauvois, 1992; Biesen- bach-Lucas & Weasenforth, 2001; Chun, 1994; Kelm, 1992; Kern, 1995; Kiesler, Siegel, & McGuire, 1984; Warschauer, 1996), greater distribution of participation among students (Beauvois, 1992; Kern, 1995; Warschauer, 1996), greater partici- pation by “shy” students (Beauvois, 1992; Chun, 1994; Kelm, 1992), transfer of communicative authority to students (Beauvois, 1992; Chun, 1994; Darhower, 2002; Kern, 1995), reduced anxiety (Chun, 1994; Kern, 1995), reduced burden on working memory (Payne & Whitney, 2002), development of extended discus- sion more attuned to student motivation and ability (Weasenforth, Biesenbach- Lucas, & Meloni, 2002), increased attention to classmates (Chun, 1994), and col- laboration among students in the development of learner-learner support networks (Darhower, 2002; Feenberg, 1999; Kern, 1995). Again, the results of CMC are increased interaction and communication. Chun (1994) had suggested that the discursive cohesiveness and levels of in- teraction observed in synchronous CMC (SCMC) might transfer to oral conver- sation. In fact, CMC has been proven effective for increasing oral achievement and proficiency. In a research study of 83 students in fourth-semester French, Beauvois (1998) found a significant difference in achievement on oral exams of +11% for students who had discussed assigned reading texts (the subject of oral exams) only via SCMC. In a study of 20 ESL students, Warschauer (1996) report- ed increased English proficiency scores in correlation to increased participation in CMC. Payne and Whitney (2002), in a study of 58 students in a third-semester Spanish course, found that SCMC improved performance in oral production inter- views more than did in-class discussion. In a third-semester German course (Ittzes Abrams, 2003), in-class SCMC (chat) sessions led to significantly better oral per- formance in large group discussions than did typical course activities (reading, exercises, and interviews). The study included 33 control subjects and 32 subjects using SCMC. Improved oral performance was measured in c-units, or phrases
Robert Sanders 61 with communicative value. The students who completed all of the chat assign- ments produced an average of 58% more words during oral testing and 53% more c-units. While there was no significant effect on lexical richness, lexical density, or syntactic complexity, Ittzes Abrams emphasized the improvement in fluency among learners who could “access the necessary lexical items with greater facility and speed” (p. 164). Differences Between Synchronous and Asynchronous CMC Each modality of text-based CMC appears to offer different advantages over oral discussion and writing. Sotillo (2000) has addressed the differences in L2 lan- guage production in SCMC and asynchronous CMC (ACMC). Drawing upon Wei’s (2000) description of subsystems in lexical structure, Sotillo characterized interaction in SCMC as fluid and interpersonal, in which students were “operat- ing at the lexical-conceptual structural level, which conflates universally avail- able semantic and pragmatic information” (Sotillo, p. 105), whereas interaction in ACMC was more presentational in that students used more text with greater complexity, indicative of “operating at the predicative-argument structural level” (p. 106). In layman’s terms, transcripts of SCMC look like those of dynamic oral conversations, while transcripts of ACMC are similar to formal reports or written responses. Biesenbach-Lucas and Weasenforth (2001) performed an ESL study that also suggests two distinct discursive styles in CMC. When the ESL students used word processors in class they produced long documents with extensive con- textualization, whereas they assumed more context when using email in class. Electronic mail was not designed for and cannot fully facilitate SCMC, but simul- taneous use by students meeting together in a class room constitutes an arguably synchronous modality. The different styles evidenced in SCMC and ACMC correspond to Halliday’s (1993) basic distinction between communicative interaction and interpretation of experience. As the semiotic system of the child develops into that of an adult, these perspectives of meaning become associated with speaking and writing. Speaking corresponds to the dynamic aspect of reality, “reality as process” in Halliday’s words, while writing foregrounds “reality as object” (p. 111). Halliday points out that spoken language emphasizes verbs and adjectives, whereas in writ- ing “processes and properties are construed as nouns” (p. 111). This construal in- volves the complex subordination characteristic of ACMC that is generally absent in SCMC. Synchronous electronic communication elicits an interactive style set upon information exchange and social cohesion, whereas asynchronous electronic communication elicits formal analysis and synthesis of information. Darhower’s (2002) sociocultural study of electronic L2 communication found student nego- tiation of intersubjectivity, topics, and social cohesion to be emerging features of SCMC interaction. Knowledge construction was not a salient feature. The interactive and reflective functions evidenced by production in CMC are also indicated by the effects of synchronous and asynchronous CMC on oral com- munication. In a comparison of synchronous and asynchronous electronic dis-
62 CALICO Journal, Vol. 24, No. 1 course with face-to-face activities, Ittzes Abrams (2003) found that where SCMC facilitated significant production gains in follow-up oral discussions, ACMC did not. In fact, the ACMC (electronic bulletin board) treatment resulted in the low- est oral production of all when compared to SCMC and face-to-face preparation. In the follow-up discussion, students who had used the electronic bulletin board produced an average of 93% fewer words and 43% fewer c-units than the control group, despite the fact that they used more “subordinate, relative and infinitive clauses during their computer-mediated communication” (p.163). Cost Effectiveness and Distributed CMC Studies of the cost effectiveness of integrating technology in education are few and far between (see Atkin, 1970; Fletcher, 1995; Hawley, Fletcher, & Piele, 1986; Knight, 2000; Levin, 1984). Cost has not been a frequent variable in research on L2 curricula and is surprisingly missing in research on technology in SLA. Sala- berry’s (2001) retrospective study of Modern Language Journal articles on tech- nology exposes a history of mostly unsubstantiated expectations of greater learn- ing outcomes and cost effectiveness. Salaberry concludes that much skepticism is in order and recommends that “efficient use of human and material resources” (p. 51) become an important criterion for evaluating L2 technology. Cost effective- ness has, on occasion, appeared as an organizing theme in literature on second language curricula (see Garrett, 1991; Maxwell & Garrett, 2002; Meredith, 1983). The focus on effectiveness in studies of CALL may be justified given that robust gains in learning efficiency can come from improved competency outcomes and faster program completion,1 but the cost effectiveness of learner-centered CALL has not yet been systematically addressed. Egbert, Paulus, and Nakamichi (2002) identify limited technology resources as an obstacle to the effectiveness of CALL in the classroom and suggest that course designs incorporating technology should include ways to teach around this barrier (p. 122). One suggestion would be to make greater use of the resources students possess as well as those provided by the institution for the general student popula- tion. CMC activities that only involve students enrolled in the course (e.g. those that do not involve foreign partners) could be performed out-of-class, while in- class activities could focus on real speaking, cultural activities, face-to-face test- ing, and physical practice of L2 non-verbal communication. Since CMC is place and time independent (Warschauer, 1997), one should not expect that engaging in CMC in class is necessarily more effective than engaging in it out of class. González-Bueno and Pérez (2000) found that student production in out-of-class email dialogue journals (ACMC) exceeded that of in-class paper-and-pencil dia- logue journaling. They did not find any significant lexical or grammatical advan- tage in the electronic medium and suggested that dialogue journals and compo- sitions, regardless of the medium, be complemented with form-focused writing activities. Pérez-Sotelo and González-Bueno (2003) revisited the question in a more controlled study in which the paper-and-pencil dialogue journaling was also performed out-of-class. They found no significant differences between the elec- tronic and the paper-and-pencil method with respect to lexical accuracy and the
Robert Sanders 63 amount of language produced, but significantly greater grammatical accuracy for the control group. They recommend that the electronic journals not be used where grammatical accuracy is a principal goal but, instead, that it be exploited for other benefits. For example, Chun (1994) focused on “sociolinguistic and interactive competence” in evaluating the benefits of SCMC (p. 18). Interestingly, Pérez-So- telo and González-Bueno’s research study was instrumental in moving the open- ended writing assignment out of class. In a study on vocabulary growth among first-semester Spanish students using synchronous and asynchronous CMC, Pérez (2003) found no significant differ- ence between the effectiveness of instructor supervised in-class chat sessions and email journals completed at home, despite the fact that the instructor supplied new vocabulary to the students during the chat sessions. Pérez also contends that the subjects enjoyed writing email journals instead of attending class one day each week. Despite the potential for distributed resource utilization, most case studies of SCMC employ designs in which the activity is performed in classrooms or com- puter labs by an entire class, simultaneously, with the instructor present. Not only is it overly expensive to use CMC during traditional class meetings, it may be an underutilization of the teacher’s core competence. While most L2 students tend to value instructor presence very highly while using CMC (see Darhower, 2002; Pérez, 2003; Stepp-Greany, 2002), this may be a reflection of the general desire for personal guidance in most language study tasks. Students overwhelmingly value contact with a teacher as one of the most important and effective compo- nents of L2 study, but instructor assistance during CMC usually takes the form of technical aid, task explanation, and vocabulary and grammar clarification. Most of these needs can probably be met through student training, clear and simple ex- pectations, illustrative examples (see Weasenforth et al., 2002, p. 63), reference to the appropriate chapter of the textbook, and student-student collaboration. Facili- tation and guidance by the instructor is by no means limited to his or her presence during or at the physical interface with CMC; integration, implementation, and monitoring of CMC in the curriculum, along with abundant feedback, bring forth much more effectively and efficiently the instructor’s role of facilitator, trainer, motivator, and collaborator. RESEARCH QUESTION Is student productivity greater when SCMC is used in class, or when it is used out of class by students working collaboratively to schedule and complete their assignments? METHOD Procedure The differences between synchronous and asynchronous CMC discourse revealed by existing research suggest general guidelines for their implementation. Lee (2002) concludes that “CMC is an effective way for learners to negotiate both
64 CALICO Journal, Vol. 24, No. 1 meaning and form and reinforce their communication skills” (p. 20).2 Lee empha- sized the use of chat for completing communicative tasks that required personal information exchange (focus on meaning), student selection of form (linguistic and sociolinguistic competence), and that were related to a specific chapter of the course text (grammar and vocabulary reference). In the study described here, students were required to exchange personal infor- mation in groups and later report the information individually in formal composi- tions. The weekly topics were chosen from the corresponding textbook chapter. The control group completed chat room assignments during class time, under in- structor supervision, in self-selected groups, using a WebCT chat room (SCMC). In-class participation is normally assessed based on instructor perception, and in the control group instructors used a 10-point scoring guide (see the scoring guide in the Appendix). Transcripts were not used to grade the discussions be- cause doing so would have duplicated the instructors’ supervision of the activity. The following class day, control students turned in paper compositions reporting the information they had learned about their partners. Teachers responded to the compositions with written comments and a grade. Grades for the chat sessions were reported using participation sheets. Collaboration was harnessed in the experimental treatment by requiring stu- dents to form their own work groups and schedule their own chat sessions outside of class time. Most of them used their home computers. They used WebCT chat rooms for their discussions (SCMC) and posted their compositions in a WebCT electronic bulletin board (ACMC). Students had until the end of the week to com- plete the chat and composition assignments. Their chat discussions were graded the following week using transcripts and following the same 10-point scoring guide as in the control group. Grading participation with transcripts in the experi- mental sections took roughly the same amount of time (30 minutes) as immediate supervision of the control group. In the electronic bulletin board, students were able to see the compositions of other students. The teachers replied to the content of each composition with follow-up questions and reported grades for chat and compositions to students using a participation sheet. The instructors did not participate in the activity with the students, although the instructors for the control group were available to help students. Instructors from both groups intervened in class with direct instruction (e.g., presentation, drills, and/or communicative activities) whenever grammatical errors common to the majority of students were detected. Using synchronous communication to exchange interesting personal informa- tion that students will later synthesize and report gives the interaction a specific purpose, thus motivating students to stay on task. It also gives the instructor a meaningful context for evaluation in that incomplete reports are often the result of insufficient information exchange in SCMC. Developing the same topics in both CMC modalities allows for practice of form and vocabulary in both interpersonal and presentational modes. In this context students often use transcripts of their chat to help them recall the information and to monitor and correct form. It should be noted that this study is part of a greater research project investigat-
Robert Sanders 65 ing the effects of reduced seat time. The experimental treatment included reduced seat time as well as lower maximum section size. The maximum allowable en- rollment was 30 students per section in the control group and 25 per section in the experimental group. The control group spent 200 minutes per week in class (including 30 minutes in the computer lab completing the chat activity) while the experimental group spent 130 minutes per week in class (not including the out-of- class chat activity). Both groups completed a common set of weekly online gram- mar activities and quizzes, listening comprehension quizzes, and reading compre- hension quizzes. Additionally, the experimental group had one online vocabulary activity and quiz per week and one more online listening comprehension activity and quiz per week than the control group. The time to complete these activities was estimated at 40 minutes. In the control group, these additional assignments were administered as textbook-based in-class activities. Data Analysis In the first week of classes, students were surveyed to determine their prior experi- ence studying Spanish. Transcripts from weeks seven and eight were chosen for analysis because at that point the students and instructors were accustomed to per- forming the activity. The topic for week seven was a typical day in the student’s life, and week eight’s was about favorite places. The transcripts were analyzed for minutes of activity, turns, Spanish words, correctly spelled Spanish words (both accounting for accents and not accounting for accents), non-Spanish words, com- ments off task, and socially appropriate comments. Proper nouns, unintelligible words, numerals, and special chat symbols (emoticons) were not counted. Punc- tuation and capitalization were excluded. The discussions were not analyzed for grammatical quality. All results were recorded individually for each student. Non-Spanish words were typically English but sometimes included French or Italian words. All instances of non-Spanish words were tallied (including repeti- tions). “Socially appropriate comments” or questions were typically greetings, leave takings, and related inquiries. Darhower (2002) found that these comments were a significant feature of SCMC and greatly contributed to social cohesive- ness. Drawing on Chun’s (1994) recognition of “minimal sociolinguistic compe- tence” (p. 28), Meskill’s (1999) concept of communities of learners, and Wenger’s (1998) theory of communities of practice, Darhower (2002) characterized socially appropriate comments as part of “behaviors that led to the construction of an on- line discourse community of Spanish learners” (Darhower, p. 265). Four Spanish word counts were employed: (a) all instances of recognizable Spanish words, (b) all instances of correctly spelled Spanish words including ac- cents and diacritical marks, (c) all instances of correctly spelled Spanish words excluding accents and diacritical marks, and (d) correctly spelled, distinct Spanish vocabulary words, excluding accents and diacritical marks. The first three Span- ish word counts and the other features measured in the transcripts were tallied by a graduate research assistant who was not affiliated with the course. The fourth count—that of distinct Spanish vocabulary words—was calculated by the author using word lists generated by Concordance, a software program developed by
66 CALICO Journal, Vol. 24, No. 1 R. J. C. Watt. All distinct vocabulary words in the lists that appeared to be cor- rectly spelled Spanish words were tallied. To simplify the counting, accents and diacritical marks were excluded, and all misspelled words were omitted. None of the word counts is taken alone as a definitive indicator of quality, but together they suggest something about quantity and quality of production at the word level. The following fragment, taken from an experimental group discussion in week eight, can be used to demonstrate the analysis. There were in fact three partici- pants in this discussion, but the fragment is short and only includes two partici- pants. Student 1: Mi bar favorita es el Backroom. Student 1: Vamos a bailar el viernes Student 2: Mi bar favorito es el “Pub at the en of the Universe”. Student 1: ¿Donde es Pub at the end of the universe? Student 2: ¿Cuál es su balie favorito? This sample does not include any non-Spanish words or socially appropriate com- ments. Because the assigned topic was favorite places, none of the comments were considered to be “off task.” Proper nouns are excluded. The first word count tallies 12 instances of recognizable Spanish words for the first student and 10 for the second. The word balie is recognized as baile even though it is misspelled. The second count shows 11 instances of correctly spelled Spanish words for the first student and 9 for the second. The word donde is misspelled because of the lack of the diacritical mark. The word balie is discounted for misspelling, and favorita is accepted as a correctly spelled word even though it does not agree with the noun. (As stated above, this analysis does not concern grammatical accuracy.) The third count, which excludes accents and diacritical marks, shows 12 instances of correctly spelled Spanish words for the first student and 9 for the second. The fourth count—that of distinct and apparently properly spelled Spanish vocabulary words—shows 10 and 7 words, respectively. Student 1 repeated es and el. In the word list, donde appears to be properly spelled. Student 2 repeated favorito and es. Participants In week seven, 46 students from the control group completed the activity and con- sented to having their transcripts used for research. In week eight, 42 completed the activity and consented to the case study. The students had an average 2.0 years prior study of Spanish. Completion and consent for the experimental group included 54 students in week seven and 57 students in week eight.3 They had an average of 1.9 years prior study of Spanish. The average size of the experimental chat groups was 3.0 students in week seven and 3.2 students in week eight. In the control treatment, the average sizes were 3.6 and 3.4 students per chat group each week, respectively. The control and experimental group instructors were not the same. The control students were enrolled in four sections of Spanish 101 taught by three different
Robert Sanders 67
nonnative instructors. They had an average 1.7 years prior experience teaching
Spanish as a foreign language. The experimental treatment was given to six sec-
tions taught by three instructors and the course coordinator (author). The instruc-
tors had an average 1.7 years prior experience teaching Spanish, and one was a
native speaker. The course coordinator had 8 years prior experience. Because the
study was part of a greater investigation of reduced seat time instruction involving
15 sections and 12 teachers, the instructor variable could not be isolated. Although
the conditions for testing were not pristine, they reflect the type of practical imper-
fection experienced in many large foreign language programs.
RESULTS
All results were tested for significant differences using independent samples t
tests, with a two-tailed significance value of .05 as a standard. Care should be
taken in examining the results of the t tests since there were multiple dependent
variables, but in most comparisons the p value was well above or well below .05.
The students in the experimental group had significantly better results in both
weeks with respect to total minutes of activity (+45%), turns (+107%), instances
of Spanish words (+103%), instances of correctly spelled Spanish words when
accounting for accents (+91%), instances of correctly spelled Spanish words
when not accounting for accents (+93%), socially appropriate comments (+74%),
and original instances of Spanish vocabulary words (+60%; p < .01 in all com-
parisons). The total number of non-Spanish words was significantly higher for
students in the experimental group (+239%; p < .01 in both comparisons). The
results are summarized in Table 1.
Table 1
Mean Productivity
Week 7 (n = 46, 54) Week 8 (n = 42, 57)
Cont. Exp. p Cont. Exp. p
Minutes 23.67 33.09 .000* 21.74 32.93 .000*
Turns 17.65 33.56 .000* 16.41 36.77 .000*
Spanish words 75.57 144.91 .000* 70.12 150.58 .000*
Correctly spelled, w/ accents 52.02 92.74 .000* 44.86 91.95 .000*
Correctly spelled, w/o accents 59.33 105.59 .000* 50.67 106.33 .000*
Spanish vocabulary 40.85 62.39 .000* 37.05 62.16 .000*
Non-Spanish words 2.52 9.07 .001* 3.07 9.88 .000*
Social comments 3.41 7.04 .000* 5.07 7.74 .001*
Comments off task 0.00 0.06 .359 0.00 0.12 .188
* p < .0568 CALICO Journal, Vol. 24, No. 1
Social Interaction and Focus on Task
On the whole, the experimental group engaged electronically in more social com-
munication. This might be expected because these students were not in immediate
contact with each other, whereas students in the control group were in each other’s
presence in the computer laboratory during regularly scheduled class time.
Both groups appear to have been similarly attentive to the assigned task. The
results for comments off task did not reveal any significant differences. The off-
task comments were in fact made by only a few students: one student with three
off-task comments in week seven, three students with a total of seven off-task
comments in week eight. These results differed greatly from those found by Dar-
hower (2002) in a study of sociocultural aspects of SCMC. He discovered that
when students used pseudonyms and interacted anonymously, 15% to 48% of all
chat discussion was off task, with interlocutors usually leaving the assigned topic
as soon as the instructor left the chat room. The difference in outcomes between
these two studies suggests that assigning participation grades and follow-up ac-
tivities (reflective reports) is effective for keeping students on task. But it should
be noted that Darhower’s study also identified student interest and topic relevance
as prevalent determinants in student negotiation of new topics, indicating that to
stray from assigned tasks may actually be beneficial for L2 production (see also
Weasenforth et al., 2002).
Spelling Accuracy
The control and experimental groups had a similar degree of spelling accuracy.
This is suggested by comparing each student’s ratio of correctly spelled Spanish
words (all instances) to his or her recognizable Spanish words (all instances). The
ratios indicate the percentage of Spanish words that are correctly spelled (see
Table 2).
Table 2
Percentage of Spanish Words Spelled Correctly
Week 7 (n = 46, 54) Week 8 (n = 42, 57)
Cont. Exp. p Cont. Exp. p
Including accents 68% 64% .132 64% 62% .414
Excluding accents 79% 73% .043* 73% 72% .700
* p < .05
When excluding accents, the control group had an accuracy ratio 6% higher
than that of the experimental group in week seven and 1% higher in week eight.
When accounting for accents and diacritical marks, the control group had ratios
4% higher in week seven and 2% higher in week eight. Only one of the four com-
parisons showed a significant difference (t test).Robert Sanders 69
Production and Time on Task
Time on task probably affected the other dependent variables. The total number of
minutes active was different for each student and significantly different between
the two groups. Some insight can be gained by adjusting for this difference. The
results for each student were divided by his or her number of turns, therefore
producing a time-independent description of student production, and by his or her
number of minutes active, therefore indicating the students’ rates of production.
Although these calculations manipulate the original data, independent samples t
tests of the means for each student in the two groups resulted in p values that were,
for the most part, well above or well below .05.
Dividing each student’s production by his or her number of turns suggests only
one consistent difference between the two groups: the experimental treatment is
susceptible to greater use of non-Spanish words per turn. Results are summarized
in Table 3.
Table 3
Mean Productivity per Turn
Week 7 (n = 46, 54) Week 8 (n = 42, 57)
Cont. Exp. p Cont. Exp. p
Spanish words 3.99 4.44 .149 4.19 4.20 .980
Correctly spelled, w/ accents 2.69 2.86 .438 2.69 2.64 .843
Correctly spelled, w/o accents 3.08 3.26 .487 3.05 3.04 .948
Spanish vocabulary 2.37 2.05 .124 2.45 1.86 .002*
Non-Spanish words 0.12 0.26 .011* 0.19 0.25 .200
Social comments 0.22 0.23 .794 0.40 0.22 .001*
Comments off task 0.00 0.00 .359 0.00 0.00 .141
* p < .05
Independent samples t tests of the per-turn averages revealed significant differ-
ences in only 3 of 14 comparisons: the experimental group used more non-Span-
ish words in week seven (+113%, p = .011), had fewer original instances of Span-
ish vocabulary words in week eight (-24%, p = .002), and made fewer socially
appropriate comments in week eight (-44%, p = .001). For the most part, dividing
total student production by the number of turns produces mixed results.
The averages produced by dividing each student’s results by his or her minutes
of activity are much more suggestive of differences between the groups, and in-
dependent samples t tests of these results indicate significant differences in 11 of
the 16 possible comparisons. Results are summarized in Table 4.70 CALICO Journal, Vol. 24, No. 1
Table 4
Mean Productivity per Minute
Week 7 (n = 46, 54) Week 8 (n = 42, 57)
Cont. Exp. p Cont. Exp. p
Turns 0.75 1.08 .001* 0.81 1.17 .000*
Spanish words 3.03 4.58 .001* 3.44 4.69 .009*
Correctly spelled, w/ accents 2.08 2.91 .006* 2.21 2.87 .047*
Correctly spelled, w/o accents 2.37 3.33 .005* 2.51 3.34 .025*
Spanish vocabulary 1.69 2.00 .088 1.89 1.96 .727
Non-Spanish words 0.10 0.31 .002* 0.15 0.32 .006*
Social comments 0.15 0.23 .001* 0.28 0.25 .441
Comments off task 0.00 0.00 .359 0.00 0.00 .180
* p < .05
In both weeks, students in the experimental group had better results with re-
spect to the number of turns per minute (+44%); instances of Spanish words per
minute (+43%); and instances of correctly spelled Spanish words per minute,
both when accounting for accents (+35%) and when not accounting for accents
(+37%). These differences were significant (p < .05 in all comparisons). The ex-
perimental group also had more original instances of Spanish words per minute in
both weeks (+11%), but the differences were not significant. Dividing the number
of social comments per minute of activity produced mixed results. Students in the
experimental group had poorer results with regard to the number of non-Spanish
words used per minute (+152%), with an average of .3 per minute compared with
.1 for the control group (p < .05 in comparisons for both weeks). On the whole,
the experimental group outperformed the control group in 11 of the 16 “per-min-
ute” calculations. These results indicate greater production for the experimental
group.
Cost Effectiveness
The use of class time and university resources was considerably lower for the
experimental group. Computer labs were reserved for the control group, whereas
students in the experimental group used general access labs and home computers
to complete the assignment. For the experimental group, the chat room activ-
ity extended student-student communication beyond the limits of class time and
space.
DISCUSSION
Time on task was the most salient difference between the treatments. In the con-
trol group, grading was based on instructor perception of participation, as is typi-Robert Sanders 71
cal of traditional classroom management. Ironically, students may have perceived
the instructor to be responsible for their level of engagement. In the experimental
group, grading was done with transcripts that recorded time on task, and, more
important, the students alone were responsible for their level of activity. The fact
that students in the experimental group averaged 33.1 and 32.9 minutes actively
participating in weeks seven and eight—exceeding the requirement by 10%—
suggests that, given more responsibility and accountability for their participation,
students will spend more time on task. This suggestion was confirmed the follow-
ing year when out-of-class chat assignments were required in all sections of Span-
ish 101 at the university. The required length for the activity was reduced to 20
minutes, but this did not affect time on task. The average time in discussion with
other students (not simply logged in) was 29 minutes in week seven and 31 min-
utes in week eight, exceeding the requirement by an average of 50%. Apparently,
students are comfortable with the medium, interested in the topics, motivated to
communicate with each other, and they collaborate responsibly.
An obvious explanation for the increased rate of productivity and use of non-
Spanish words in the experimental group would be that students code switched
more often to bridge interlanguage gaps. After all, students in the experimental
group used, on average, 152% more non-Spanish words per minute. But the ratios
of non-Spanish words (all instances) to Spanish words (all instances) for each
group appear similar. The control group had ratios of 3% and 5% in weeks seven
and eight, respectively, compared to the experimental group’s 6% and 7% (see
Table 5).
Table 5
Ratio of Non-Spanish Words to All Instances of Spanish Words
Week 7 (n = 46, 54) Week 8 (n = 42, 57)
Cont. Exp. p Cont. Exp. p
.03 .06 .019* .05 .07 .136
p < .05
These findings seem to confirm other reports that code switching in SCMC is
minimal (see Beauvois, 1992, 1997; Darhower, 2002; see also Kelm, 1992). It is
unclear whether the increase in non-Spanish words was a cause or effect of faster
production. Additionally, many of the English words used by both groups were
introduced by students asking their partners for the Spanish equivalent. The ex-
perimental group may have used this technique more often because the instructor
and other students were not physically present to supply and clarify vocabulary.
Some researchers contend that peer advising on vocabulary is beneficial for SLA
(see Darhower, 2002; Kötter, 2003).
CONCLUSION
The experimental treatment was a more effective instructional model. Altogether,
the experimental group faired better in 23 out of 32 instances in which significant72 CALICO Journal, Vol. 24, No. 1
differences were found. Student collaboration outside of class resulted in more
student participation and production. The quality of production appears—at the
word level—to be similar. The main difference is quantity of production. Existing
research suggests that the increase in contact and practice should result in greater
acquisition.
Limitations of the Study and Suggestions for Further Research
This study did not include a grammatical analysis of the discourse produced.
A comparison of grammatical accuracy and syntactic complexity is warranted.
Analysis of student improvement during the course of a term is also needed.
While Ittzes Abrams (2003) reminds us that CMC is now an important mode of
communication in its own right, the results of out-of-class SCMC on oral perfor-
mance should be compared to those of in-class SCMC. Of greater need still is a
longitudinal study of the effects of CMC on learning and acquisition after several
years of use and on the best points within L2 curricula to utilize CMC.
NOTES
1
See Fletcher’s (1995) study of computer-assisted flight training.
2
On negotiation in CMC, see Smith (2000).
3
Three students in the experimental group were excluded from the study for failure to
comply with the requirements of the assignment. In week seven, one of these students
decided to write English and used 115 non-Spanish words during the session. The remain-
ing students in the experimental group used an average of 9 non-Spanish words in the as-
signment. In week eight, two students were excluded from the analysis because they used
348 and 233 non-Spanish (mainly English) words, respectively. The other students in the
experimental group used an average of 10 non-Spanish words during the activity.
REFERENCES
Atkin, M. C. (1970). Evaluating the cost-effectiveness of instructional programs. In M. C.
Wittrock & D. E. Wiley (Eds.), The evaluation of instruction: Issues and prob-
lems (pp. 221-238). New York: Holt, Rinehart and Winston.
Bayer, A. (1990). Collaborative-apprenticeship learning: Language and thinking across
the curriculum, K-12. Mountainview, CA: Mayfield.
Biesenbach-Lucas, S., & Weasenforth, D. (2001). E-mail and word processing in the
ESL classroom: How the medium affects the message. Language Learning &
Technology, 5 (1), 135-165. Retrieved April 18, 2006, from http://llt.msu.edu/
vol5num1/weasenforth/default.html
Blake, R. (2000). Computer mediated communication: A window on L2 Spanish interlan-
guage. Language Learning & Technology, 4 (1), 120-136. Retrieved April 18,
2006, from http://llt.msu.edu/vol4num1/blakeRobert Sanders 73
Beauvois, M. H. (1992). Computer-assisted classroom discussion in the foreign language
classroom: Conversation in slow motion. Foreign Language Annals, 25 (5), 455-
464.
Beauvois, M. H. (1997). Computer-mediated communication: Technology for improv-
ing speaking and writing. In M. D. Bush (Ed.), Technology-enhanced language
learning (pp. 165-184). Lincolnwood, IL: National Textbook Company.
Beauvois, M. H. (1998). Write to speak: The effects of electronic communication on the
oral achievement of fourth semester French students. In J. A. Muykens (Ed.),
New ways of learning and teaching: Focus on technology and foreign language
education (pp. 165-183). Boston: Heinle & Heinle.
Chun, D. (1994). Using computer networking to facilitate the acquisition of interactive
competence. System, 22 (1), 17-31.
Darhower, M. (2002). Instructional features of synchronous computer-mediated communi-
cation in the intermediate L2 class: A sociocultural case study. CALICO Journal,
19 (2), 249-278.
Egbert, J., Paulus, T. M., & Nakamichi, M. (2002). The impact of CALL instruction on
classroom computer use: A foundation for rethinking technology in teacher edu-
cation. Language Learning & Technology, 6 (3), 108-126. Retrieved April 18,
2006, from http://llt.msu.edu/vol6num3/egbert
Feenberg, A. (1999, Winter). Distance learning: Promise or threat? National CrossTalk, 7
(1). Retrieved April 18, 2006, from http://www.highereducation.org/crosstalk/
ct0199/opinion0199feenberg.shtml
Fletcher, J. D. (1995). [Computer-assisted military training]. In A. Melmed (Ed.), The
costs and effectiveness of educational technology: Proceedings of a workshop
(RAND, November 1995). Retrieved July 11, 2002, from http://www.ed.gov/
Technology/Plan/RAND/Costs/cover/html
Garrett, N. (1989). The synergism of technology and theory in classroom second language
acquisition research. In J. E. Alatis (Ed.), Georgetown University Round Table
on Languages and Linguistics (pp. 288-294). Washington, DC: Georgetown Uni-
versity Press.
Garrett, N. (1991). Technology in the service of language learners: Trends and issues. Mod-
ern Language Journal, 75 (1), 74-101.
González-Bueno, M., & Pérez, L. C. (2000). Electronic mail in foreign language writing:
A study of grammatical and lexical accuracy, and quantity of language. Foreign
Language Annals, 33 (2), 189-198.
Halliday, M. A. K. (1993). Towards a language-based theory of learning. Linguistics and
Education, 5 (1), 93-116.
Hawley, D. E., Fletcher, J. D., & Piele, P. K. (1986). Costs, effects, and utility of micro-
computer-assisted instruction (Tech. Rep. No. 1). Eugene, OR: Center for Ad-
vanced Technology in Education.
Ittzes Abrams, Z. (2003). The effect of synchronous and asynchronous CMC on oral per-
formance in German. Modern Language Journal, 87 (2), 157-167.
Kelm, O. R. (1992). The use of synchronous computer networks in second language in-
struction: A preliminary report. Foreign Language Annals, 25 (5), 441-454.74 CALICO Journal, Vol. 24, No. 1
Kern, R. G. (1995). Restructuring classroom interaction with networked computers: Ef-
fects on quantity and characteristics of language production. Modern Language
Journal, 79 (4), 457-476.
Kiesler, S., Siegel, J. & McGuire, T. W. (1984). Social psychological aspects of computer-
mediated communication. American Psychologist, 39 (10), 1123-1134.
Knight, Susan. (2000). Expanding enrollment while maintaining quality: A Spanish 101
experimental model. Hispania, 83 (4), 855-863.
Kötter, M. (2003). Negotiation of meaning and code switching in online tandems. Lan-
guage Learning & Technology, 7 (2), 145-172. Retrieved April 18, 2006, from
http://llt.msu.edu/vol7num2/kotter
Lee, L. (2002). Enhancing learners’ communication skills through synchronous electronic
interaction and task-based instruction. Foreign Language Annals, 35 (1), 16-24.
Levin, H. M. (1984). Costs and cost-effectiveness of computer assisted instruction (Project
Rep. No. 84-A21). Palo Alto, CA: Stanford University, School of Education.
Maxwell, D., & Garrett, N. (2002). The challenge to language learning in higher education.
Change, 34 (3), 22-28.
Mehan, H. (1985). The structure of classroom discourse. In T. A. Van Dijk (Ed.), Handbook
of discourse analysis (pp. 119-131). London: Academia.
Meredith, R. (1983). Materials and equipment: The next generation. Modern Language
Journal, 67 (4), 424-430.
Meskill, C. (1999). Computers as tools for sociocollaborative language learning. In K.
Cameron (Ed.), Computer-assisted language learning (CALL): Media, design
and applications (pp. 144-152). Exton, PA: Swets and Zeitlinger.
Payne, J. S., & Whitney, P. J. (2002). Developing L2 oral proficiency through synchro-
nous CMC: Output, working memory, and interlanguage development. CALICO
Journal, 20 (1), 7-32.
Pérez, L. C. (2003). Foreign language productivity in synchronous versus asynchronous
Computer-mediated communication. CALICO Journal, 21 (1), 89-104.
Pérez-Sotelo, L., & González-Bueno, M. (2003). Idea: Electronic writing in L2: Accuracy
vs. other outcomes. Hispania, 86 (4),869-873.
Salaberry, M. R. (2001). The use of technology for second language learning and teaching:
A retrospective. Modern Language Journal, 85 (1), 39-56.
Smith, B. (2000). Computer-mediated negotiated interaction: An expanded model. Modern
Language Journal, 87 (1), 38-57.
Sotillo, S. M. (2000). Discourse functions and syntactic complexity in synchronous and
asynchronous communication. Language Learning & Technology, 4 (1), 82-119.
Retrieved April 18, 2006, from http://llt.msu.edu/vol4num1/sotillo
Stepp-Greany, J. (2002). Student perceptions on language learning in a technological envi-
ronment: Implications for the new millennium. Language Learning & Technol-
ogy, 6 (1), 165-180. Retrieved April 18, 2006, from http://llt.msu.edu/vol6num1/
STEPPGREANY/default.html
Swain, M. (1985). Communicative competence: Some roles of comprehensible input and
comprehensible output in its development. In S. Gass & C. Madden (Eds.), Input
in second language acquisition (pp. 235-253). Rowley, MA: Newbury House.Robert Sanders 75
Warschauer, M. (1996). Comparing face to face and electronic discussion in the second
language classroom. CALICO Journal, 13 (2 & 3), 7-26.
Warschauer, M. (1997). Computer-mediated collaborative learning: Theory and practice.
Modern Language Journal, 81 (4), 470-481.
Weasenforth, D., Biesenbach-Lucas, S., & Meloni, C. (2002). Realizing constructivist
objectives through collaborative technologies: Threaded discussions. Language
Learning and Technology, 6 (3), 58-86. Retrieved April 18, 2006, from http://llt.
msu.edu/vol6num3/weasenforth
Wei, L. (2000). Unequal election of morphemes in adult second language. Applied Linguis-
tics, 21 (1), 106-140.
Wenger, E. (1998). Communities of practice: Learning, meaning and identity. New York:
Cambridge University Press.
APPENDIX
Participation Scoring Guide
8-10 = The student
a. communicates very well in discussion contexts
b. never uses English in discussion, pair or group activities, or when asking
questions
c. responds fully, voluntarily elaborates on answers, and elicits discussion
from partners by asking pertinent questions
7 = The student
a. communicates insufficiently in some discussion contexts
b. frequently uses English, with and without instructor permission
c. gives short answers, has difficulty elaborating when requested
1-6 = The student
a. is unable to communicate in most or all discussion contexts, is tardy or
leaves early
b. speaks mostly English in class
c. makes mostly errors with regard to materials covered
d. responds to contextual questions with isolated words or silence
0 = The student is absent, regardless of cause.
ACKNOWLEDGMENT
The author would like to acknowledge the assistance of Michelle Jurado, the
graduate research assistant who tallied from transcripts Spanish words, correctly
spelled Spanish words (including and excluding accents), non-Spanish words,
comments off task and socially appropriate comments. This work could not have
been performed out of context and was essential for the study.76 CALICO Journal, Vol. 24, No. 1 AUTHOR’S BIODATA Robert Sanders is Assistant Professor of Spanish in the Department of Foreign Languages and Literatures at Portland State university. He received his Ph.D. from the University of Arizona. His research interests include distributed use of technology for the enhancement of face-to-face instruction and learning opportu- nity, and the redefining of language teaching. AUTHOR’S ADDRESS Robert Sanders Portland State University Department of Foreign Languages and Literatures FLL - NH 393 PO Box 751 Portland, Oregon 97207 Phone: 503 725 5296 Fax: 503 7255276 Email: rsanders@pdx.edu
You can also read
