Properties of the Narrative Scoring Scheme Using Narrative Retells in Young School-Age Children
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Research
Properties of the Narrative Scoring Scheme
Using Narrative Retells in Young
School-Age Children
John Heilmann
East Carolina University, Greenville, NC
Jon F. Miller
Ann Nockerts
University of Wisconsin—Madison
Claudia Dunaway
San Diego Unified School District, CA
Purpose: To evaluate the clinical utility of the Conclusion: The NSS is an efficient and informa-
narrative scoring scheme (NSS) as an index of tive tool for documenting children’s development
narrative macrostructure for young school-age of narrative macrostructure. The relationship
children. between the NSS and microstructural measures
Method: Oral retells of a wordless picture book demonstrates that it is a robust measure of
were elicited from 129 typically developing chil- children’s overall oral narrative competence and a
dren, ages 5–7. A series of correlations and powerful tool for clinicians and researchers. The
hierarchical regression equations were completed unique relationship between lexical diversity and
using microstructural measures of vocabulary and the NSS confirmed that a special relationship
grammar to predict NSS scores. exists between vocabulary and narrative organi-
Results: While the NSS was significantly corre- zation skills in young school-age children.
lated with age and each of the microstructural
measures, the hierarchical regression analyses
revealed a unique relationship between vocabu- Key Words: narrative, language sample
lary and narrative macrostructure. analysis, story grammar, vocabulary
A
nalysis of oral narratives provides a rich source of Scott & Stokes, 1995). Measures of linguistic content are used
data that documents children’s language use in a to document children’s productive vocabulary skills, which
naturalistic context. Narrative analysis is a highly typically calculate children’s lexical diversity using measures
effective clinical and research tool, as it allows examiners to such as type–token ratio (Templin, 1957) and number of dif-
analyze multiple linguistic features simultaneously using a ferent words (Miller, 1987; Miller & Klee, 1995). While these
single short sample. Examiners have the opportunity to doc- measures are the most commonly cited microstructural anal-
ument children’s productive vocabulary and grammar using yses in the literature, this list is in no way exhaustive. Micro-
microstructural analyses as well as children’s broader text- structural analyses continue to be reviewed, critiqued, and
level narrative organization skills by utilizing macrostructural reanalyzed (see Justice et al., 2006, for a review and discussion).
analyses (see Westby, 2005). Macrostructural analyses, on the other hand, examine
Microstructural analyses primarily focus on children’s lin- children’s language skills beyond the utterance level and
guistic form and content, which are measured within individ- document children’s ability to relate concepts that transcend
ual utterances. Linguistic form is commonly assessed by the individual utterance. Most macrostructural analyses of
analyzing children’s grammatical and syntactic abilities using children’s narratives are rooted in the story grammar tradi-
mean length of utterance (e.g., Brown, 1973; Miller, 1981) tion, which proposes that all stories have a setting and episode
and various measures of sentence complexity (e.g., Nippold, system (Mandler & Johnson, 1977; Rumelhart, 1975; Stein &
Hesketh, Duthie, & Mansfield, 2005; Schuele & Tolbert, 2001; Glenn, 1979). The setting provides background information
154 American Journal of Speech-Language Pathology • Vol. 19 • 154–166 • May 2010 • A American Speech-Language-Hearing Associationon the characters and their environment, while the episode difficulties of children with language impairment suggest that
system includes three main components that occur in all their language deficits may be due to broader information-
stories: (a) a problem (initiating event and/or internal response), processing deficits, such as a reduced processing capacity
(b) attempts at solving the problem, and (c) consequences/ (Boudreau, 2007; Colozzo, Garcia, Megan, Gillam, & Johnston,
outcomes. To be a complete episode, a narrator must include 2006). Further study of the relationship between children’s
all three of these key components (see Strong, 1998, for a microstructural and macrostructural language skills will pro-
review). These settings and episodes can be combined in an vide a better understanding of the nature of the impairment.
infinite number of ways to create individual stories. Better understanding of children’s narrative skills in general,
and macrostructural skills in particular, also has substantial
clinical implications. Difficulty with narrative organization
Relationship Between Microstructural and
can have a dramatic impact on children with language impair-
Macrostructural Measures ment, as such discourse-level skills are required to effectively
Previous research has demonstrated that lexical and gram- communicate. While vocabulary and grammar deficits limit
matical gains play an important role in children’s acqui- children’s ability to produce fully competent utterances, im-
sition of narrative proficiency (Berman & Slobin, 1994; paired macrostructural skills affect children’s ability to gen-
Bishop & Donlan, 2005). In their seminal work, Berman and erate coherent and age-appropriate extended discourse.
Slobin (1994) documented the role of linguistic forms taking Furthermore, narratives are a major component of the school
on new functions that aid in the organization of narratives. curriculum, and children are expected to understand and use
This work described the trade-offs that occurred when differ- appropriate narrative form effectively. Therefore, narrative
ent microstructural features were used for different functions, macrostructure skills must be efficiently and accurately docu-
including the organization of narratives. Children’s narrative mented, and should be considered within an extensive assess-
organization skills were positively related to advances in use of ment protocol for children with language difficulties.
microstructural features, including grammatical forms (e.g.,
verb tense, aspect, and voice), lexical forms (e.g., lexical aspect Methods of Measuring Narrative Macrostructure
and manner/cause of verbs of motion), and lexico-grammatical
features (e.g., locative particles, prepositional phrases, and While most narrative macrostructure measures are rooted
connectives). This pattern of results is consistent with Slobin’s in the story grammar tradition and share the same underlying
long-standing claim that “new forms first express old func- principles, the coding of narrative macrostructure varies
tions, and new functions are first expressed by old forms” widely throughout the literature. Some narrative macro-
(Slobin, 1973, p. 184). Bishop and Donlan (2005) observed structure coding schemes documented children’s inclusion of
that children’s ability to encode and retell a story was more specific story grammar components (e.g., Strong, 1998) or
strongly related to linguistic ability than nonverbal IQ. Chil- identified the presence or absence of specific story grammar
dren’s microstructures, including complex syntax and relating components for a given story (e.g., Berman, 1988; Boudreau
causal concepts, were more influential in event memory and & Hedberg, 1999; Miles & Chapman, 2002; Reilly et al.,
story organization than their cognitive skills. These data docu- 2004). In these studies, children who produced more story
menting the unique relationship between microstructures and grammar components and/or more advanced story grammar
macrostructures are contrary to evidence demonstrating that features were thought to have stronger narrative organization
children’s narrative organization skills emerge from more skills. The second major class of macrostructural measures
general cognitive capabilities, including executive function used text-level judgments of children’s narrative proficiency
(van den Broek, 1997), and sociolinguistic processes (Eaton, (Applebee, 1978; Hedberg & Westby, 1993; Stein, 1988).
Collis, & Lewis, 1999; McCabe, 1997; Peterson & McCabe, Rather than identifying the presence or absence of specific
2004; Quasthoff, 1997a, 1997b). story grammar components, these measures required holistic
judgments by the examiner to rate the quality and develop-
mental level of the narrative.
Narrative Skills of Children With Language Impairment The major advantage of the simple story grammar coding
Given the strong relationship between microstructural and procedures is that it facilitates a relatively high level of ac-
macrostructural measures, it is not surprising that children curacy across coders. There is less room for differences across
with language impairment have substantial difficulty using coders, as the coder is only responsible for identifying the
appropriate vocabulary and grammar when telling stories presence or absence of specific story-related themes. The
(Boudreau & Hedberg, 1999; Gillam & Johnston, 1992; Pearce, major disadvantage of simple story grammar analyses is that
McCormack, & James, 2003; Reilly, Losh, Bellugi, & Wulfeck, they are potentially limited in their ability to account for the
2004) in addition to substantial difficulty with text-level orga- abstract interutterance concepts and qualitative aspects of the
nization of narratives (Boudreau & Hedberg, 1999; Manhardt narrative, or the story’s “sparkle” (Peterson & McCabe, 1983).
& Rescorla, 2002; Merritt & Liles, 1987; Pearce et al., 2003; McFadden and Gillam (1996) demonstrated that holistic rat-
Reilly et al., 2004). The presence of both microstructural ings of children’s narratives capture the more refined aspects of
and macrostructural deficits in children with language impair- narratives, such as charm and depth, and that these ratings were
ment counters theories that identify the primary cause of better than simple story grammar analyses for documenting
language impairment as deficiencies with grammatical com- differences between children with language impairment and
petence (e.g., Clahsen, 1989; Gopnik & Crago, 1991; Rice, their typically developing peers between the ages of 9;0 (years;
Wexler, & Cleave, 1995). Alternatively, the global narrative months) and 11;7.
Heilmann et al.: Properties of the Narrative Scoring Scheme 155Additional Skills Required to Tell an Effective Story: for school-age children to effectively tell a coherent and in-
Beyond Story Grammar teresting story (see Appendix). To extend beyond simple story
grammar analyses, the NSS incorporates multiple aspects of
The development of narratives in children and adults has the narrative process into a single scoring rubric and provides
been studied extensively and has revealed additional areas of an overall impression of the child’s narrative ability. This
advancement beyond inclusion of story grammar features, metric combines both the basic features of the story grammar
including children’s use of literate language and cohesive de- approaches as well as the higher level narrative skills that
vices (Bamberg & Damrad-Frye, 1991; Halliday & Hasan, continue to develop through the school-age years. In addition
1976; Hedberg & Westby, 1993; Wigglesworth, 1997). Use of to adding higher level narrative skills in the scoring scheme,
literate language occurs when children use abstract language the NSS uses a combination of discrete coding criteria and
features commonly used by teachers and found in the curri- examiner judgment. The NSS was created to improve on the
culum (Westby, 2005). Some key literate language features simple story grammar measures by requiring the examiners to
related to narrative competence include use of metacognitive make interutterance text-level judgments, which have been
verbs (e.g., think or know), metalinguistic verbs (e.g., say or shown to be more effective than discrete coding schemes in
talk), and elaborated noun phrases (e.g., the boy in the restau- identifying children with language impairment (McFadden &
rant with the frog; see Nippold, 2007, and Westby, 2005, for a Gillam, 1996). By breaking the judgments into seven skill
review). Bamberg and Damrad-Frye (1991) identified that areas, examiners have the opportunity to reflect on each com-
more sophisticated and later developing narratives included ponent of the narrative process and judge the child’s profi-
abstract language such as metacognitive and metalinguistic ciency in that area. This combination of explicit scoring
verbs. In their analysis of productions of Frog, Where Are guidelines and flexibility to allow for examiner judgment
You? (Mayer, 1969), Bamberg and Damrad-Frye identified reflects the hybrid nature of the NSS. The scores from the
that these abstract language skills emerged at age 5 years, seven NSS categories are combined to provide a single com-
demonstrated robust development through adulthood, and posite score, which allows the examiner to generate an index
were essential for relating the hierarchical relationships be- of children’s overall narrative ability.
tween events in complex narrative productions. Additional The first step in developing the NSS was establishing the
studies have identified that literate language skills were pres- key components from the story grammar literature, which
ent in children’s oral narratives during the preschool years included the story’s introduction, the major conflicts and res-
(Curenton & Justice, 2004), developed through the school olutions (conflict resolution), and a conclusion. To document
years and into adolescence (Greenhalgh & Strong, 2001; children’s use of literate language, the NSS includes catego-
Nippold, 2007; Pelligrini, Galda, Bartini, & Charak, 1998), ries that assess metacognitive and metalinguistic verbs. The
and were used less frequently by children with language im- mental states component documents children’s abilities to use
pairment (Greenhalgh & Strong, 2001). metacognitive verbs (e.g., think and know) to describe the
Another high-level narrative feature that continues to develop characters’ thoughts and feelings. The character development
through the school years is the cohesiveness of children’s nar- component of the NSS also documents children’s literate
rative productions (Halliday & Hasan, 1976; Wigglesworth, language skills by measuring the ability to use metalinguistic
1997). To tell a story, narrators must effectively use cohesive verbs (e.g., talk and say), differentiate between main and sup-
devices to carry concepts across individual utterances. Three porting characters, and talk in the first person to depict the
major categories of cohesive devices are (a) referential cohe- characters in the story. The NSS evaluates two separate as-
sion, which allows a narrator to maintain appropriate refer- pects of cohesive ties that were adapted from Halliday and
ence to the characters, objects, and locations across utterances Hasan (1976). The referencing component measures aspects
using both noun phrases and pronouns; (b) conjunctive cohe- of referential cohesion, including appropriate use of pronouns
sion, which allows a narrator to sustain concepts across phrases and antecedents. The cohesion component documents the
and utterances using conjunctive words and phrases (e.g., conjunctive and lexical aspects of cohesion, including appro-
and, but, besides, on the other hand, finally, in addition); and priate ordering, emphasis of critical events, and transitions
(c) lexical cohesion, which allows a narrator to effectively use between events.
vocabulary to link concepts across utterances. Measures of
cohesiveness can be a sensitive index of language use, as
children with language impairment have more difficulty with Goals of the Study
correct use of cohesive ties (Hedberg & Westby, 1993; Liles, Children’s performance on the NSS has been reported in
1985; Strong & Shaver, 1991). While cohesion is often con- studies that examined the narrative organization skills of na-
sidered a microstructural measure, we treated it as a macro- tive Spanish-speaking children who were learning English
structural measure because aspects of cohesion transcend as a second language (Miller et al., 2006). The goal of the
individual utterances and are necessary for producing coher- present study was to describe the NSS from a clinical perspec-
ent narratives. tive and to further analyze the linguistic properties of the
measure in a group of children who were fluent in English.
Furthermore, the literature has revealed that there is a special
The Narrative Scoring Scheme: A Comprehensive
relationship between children’s microstructural and macro-
Measure of Narrative Proficiency structural language skills (Berman & Slobin, 1994; Bishop &
Our goal in developing the narrative scoring scheme (NSS) Donlan, 2005). To better understand the linguistic properties of
was to create a metric that documents the range of skills required the NSS and to extend our understanding of the relationship
156 American Journal of Speech-Language Pathology • Vol. 19 • 154–166 • May 2010between microstructural and macrostructural measures, a TABLE 1. Demographic data for all participants.
second goal was to document the relationship between chil-
dren’s vocabulary, grammar, and narrative organization skills. Variable n
To achieve this, we examined the relationship between mea-
Grade
sures of vocabulary, grammar, and the NSS. These analyses Preschool 3
further clarified the key constructs that the NSS is measuring Kindergarten 79
and also provided additional evidence for the role of vocabu- First grade 47
lary and grammar in the development of young school-age Sex
children’s narrative organization skills. This study addressed Female 69
the following questions: Male 60
Race/ethnicity
1. Are age and measures of vocabulary and grammar signifi- White 87
cantly correlated with NSS scores in narrative retells of Hispanic 16
young school-age children? Othera 15
African American 7
2. Are measures of vocabulary uniquely related to NSS scores No data 4
in narrative retells of young school-age children?
Note. Ethnicity data were collected for children who were Hispanic or
3. Are measures of grammar uniquely related to NSS scores Latino. Race data are provided for all children who were non-Hispanic/
in narrative retells of young school-age children? non-Latino.
a
Other races/ethnicities included Arabic (2), Chinese (3), Japanese (2),
Korean (1), Filipino (5), Portuguese (1), and Samoan (1).
Method
Participants review of the race and ethnicity data revealed that the sample
A total of 129 typically developing children age 5–7 years is a relatively heterogeneous group. Socioeconomic status
were recruited for this study. The children were recruited from was measured by calculating the highest number of years
public schools in the San Diego (CA) City School and Cajon of education that the child’s mother completed. On average,
Valley School Districts. Administrators from the two districts mothers completed 14.4 years of education (SD = 2.5), with
assisted with obtaining informed consent from each of the a range of 9–20 years. The majority of the mothers completed
children’s primary caregivers. The pool of potential partici- at least some college, while only eight of the children’s
pants was reviewed by the school’s speech-language pathol- mothers did not complete high school.
ogists (SLPs) and classroom teachers to identify children who
qualified for the study. To participate in the study, children Procedure
were required to have average scores on all summative class- Each participant completed a narrative retell of the word-
room, district, and state assessments. The classroom teachers less picture book Frog, Where Are You? (Mayer, 1969). The
reviewed the records for each student to identify him or her purpose of collecting the narratives was to establish a nor-
as average performing. The child’s academic record was also mative database reflecting typically developing children’s
reviewed to identify that he or she had no history of language oral narrative skills and to further our understanding of chil-
impairment and/or learning disability. While academic data dren’s developing narrative competence. The head SLP from
were used as inclusionary criteria for the participants, test each district was responsible for training 18 school-based
scores and descriptions of performance were not recorded and SLPs to elicit the oral narratives. The head SLP and clinicians
were not available for further analysis. The majority of the met on three separate occasions and had the opportunity to
participants were native English speakers. A small percentage practice the protocol with each other several times. The clini-
of the children were Spanish/English bilingual and designated cians read the scripted instructions to the students and cued
as “fluent English.” This designation was made by the child’s them to listen to a taped version of the story while following
respective school district and was based on a passing grade on along with the pictures in the book. The students then retold the
an English proficiency test and grade-level academics. The story using the book as an aid. Examiner prompts were limited
SLPs and teachers confirmed that each child met the inclu- to encouragement to begin the story and open-ended cues to
sionary criteria and enrolled the eligible children in the study. continue the retell. The scripted instructions and audiotaped
The individual children participating in the study provided story script were adapted from the Strong Narrative Assess-
verbal assent prior to completing the protocol. ment Procedure (Strong, 1998) and were used to facilitate
Table 1 summarizes the demographic data for the partici- high fidelity among the numerous examiners completing the
pants. Sixty-one percent of the participants were in kinder- language sample elicitation.
garten, 36% of the children were in first grade, and 2% were
in preschool. The numbers of male and female participants
were roughly equal, with slightly fewer boys than girls. The Transcription and Coding
school SLPs attempted to recruit students who reflected the The children’s narrative productions were digitally recorded
racial and ethnic diversity of their schools. The SLPs first and sent to the Language Analysis Lab at the University of
identified the children’s ethnicity (Hispanic or not Hispanic) Wisconsin—Madison, where they were transcribed by trained
and then documented the non-Hispanic children’s race. A research assistants who had at least 10 hr of transcription
Heilmann et al.: Properties of the Narrative Scoring Scheme 157experience using standard coding conventions for Systematic SALT (Miller & Iglesias, 2008), which produced a rectan-
Analysis of Language Transcripts (SALT) software (Miller & gular data file summarizing each dependent measure for each
Iglesias, 2008). Utterances were segmented into communi- of the transcripts. This file was formatted for statistical anal-
cation units (C-units; Labov & Waletzky, 1967), which in- ysis using SPSS Version 16.0.
cluded a main clause and all dependent clauses. The transcripts
began and ended with the child’s first and last utterance, re-
spectively. See Miller and Iglesias (2008) for a full review of Agreement
the transcription conventions. Accuracy of the transcription and coding process was ex-
After completing the orthographic transcription, the research amined at three levels. Protocol accuracy was calculated by
assistant reviewed the transcript and completed the NSS. To the principal investigator, who reviewed 10% of the written
score the NSS, the transcriber carefully reviewed the narrative transcripts to identify whether the transcribers were adhering
transcript and assigned a score of 0–5 for each of the seven to the transcription conventions. Percentage agreement be-
categories summarized in the Appendix. Categories that could tween the transcribers and principal investigator was 98%
not be scored received a score of zero or NA. Scores of zero for segmentation rules, 99% for word-level codes, and 98%
were given if the child did something that precluded the ex- for coding of reduplications and reformulations (i.e., mazes).
aminer from scoring a section of the NSS, such as skipping To determine transcription accuracy and coding agreement,
a part of the story or refusing to complete the task. If there was 10% of the narratives were independently transcribed and
an error on the part of the examiner (e.g., not following the coded for the NSS and SI by a second research assistant.
protocol or problems with the recording), sections of the NSS Transcription accuracy was calculated by comparing the in-
that were affected were not scored, and the examiner coded the dependent transcripts at the word and morpheme level (94%
section as not applicable for analysis (i.e., NA). For all other agreement), utterance segmentation decisions (98% agree-
sections, scores of 1 reflected minimal presence/immature ment), placement of mazes (93% agreement), and utterance
performance, scores of 3 reflected emerging skills, and scores types (100% agreement).
of 5 reflected proficient performance. Transcribers also had Calculating agreement for the two coding schemes pre-
the opportunity to assign scores of 2 and 4 if performance was sented a greater challenge. Simple interrater agreement scores
judged to be between the major anchors. To improve the can be misleading, as small differences between coders (e.g.,
accuracy of the scoring procedure, specific guidelines were NSS scores of 24 and 25) are treated the same as large dif-
provided for scores of 1, 3, and 5 (see the Appendix). These ferences between coders (e.g., NSS scores of 16 and 28).
guidelines assisted the transcriber in assigning an accurate Therefore, agreement for the NSS and SI coding was calcu-
score that reflected the child’s performance in each compo- lated using Krippendorff’s alpha, which accounted for both
nent of the narrative process and reduced the abstractness of chance agreement and the degree of difference between tran-
the narrative macrostructure concepts. The scoring across the scribers (Krippendorff, 1980). Alpha values accounting for
seven categories received equal weighting because the liter- differences in ordinal data were calculated using the summed
ature revealed that each of these seven narrative aspects is NSS and SI scores that were calculated by the two independent
necessary for telling a well-developed story. Furthermore, transcribers (a = .92 for SI; a = .79 for NSS). Krippendorff
keeping the scoring rules straightforward and consistent established benchmarks for alpha values, with ≥.80 reflecting
facilitates simple and accurate scoring. A comprehensive adequate agreement and values between .67 and .80 reflect-
training procedure is available on the SALT Web site (www. ing acceptable agreement for exploratory research and draw-
saltsoftware.com /training / handcoded /) that includes an ing tentative conclusions. For a review of the accuracy and
overview of the NSS; scoring tips; a description of how to agreement process and a further discussion of Krippendorff’s
enter NSS scores into a SALT file; excerpts from samples alpha, see Heilmann et al. (2008).
demonstrating minimal/immature, emerging, and proficient
performance across each section of the NSS; and a set of
practice transcripts. Language Sample Measures
In addition to the NSS, the transcribers completed coding To test the relationship between microstructural measures
for the subordination index (SI; Scott & Stokes, 1995; Strong, and the NSS, we used the following language sample mea-
1998). The transcribers added a code to each C-unit that sum- sures that repeatedly have been found to be robust and devel-
marized the number of independent and dependent clauses. opmentally sensitive to the population and context used in the
C-units that were incomplete, unintelligible, or nonverbal, or present study (i.e., typically developing 5–7-year-olds who
that had an error at the utterance level, were excluded from produced oral narratives):
the analysis. Utterance-level errors included incorrect word Length/productivity. Number of total words (NTW) is a
order, omission of more than two words in an utterance, and measure of productivity that documents the amount of infor-
utterances that simply did not make sense. Elliptical responses to mation provided in the story (Allen, Kertoy, Sherblom, &
examiner questions were also excluded from the SI analysis. Pettit, 1994; Paul & Smith, 1993). In addition, NTW was used
Utterances in which the child inappropriately omitted the sub- to assist with statistically controlling sample length, which has
ject or copula were coded and received a score of zero. After the potential to affect additional language sample measures.
each C-unit was coded, the SI was generated by dividing Vocabulary. Number of different words (NDW) is a mea-
the total number of clauses (both main and subordinate) by sure of lexical diversity that provides a robust estimate of
the total number of C-units. After all the narratives were children’s productive vocabulary (Klee, 1992; Miller, 1987;
transcribed and coded, the transcripts were analyzed using Miller & Klee, 1995) and has been widely used as an index of
158 American Journal of Speech-Language Pathology • Vol. 19 • 154–166 • May 2010vocabulary skills in studies examining children’s oral nar- TABLE 2. Descriptive statistics for language sample measures
rative skills (Gazella & Stockman, 2003; Humphries, Cardy, and correlations with the narrative scoring scheme (NSS).
Worling, & Peets, 2004; Swanson, Fey, Mills, & Hood, 2005;
Correlation
Uccelli & Páez, 2007). Measure M SD Range with NSS
Grammar. Mean length of C-unit (MLCU) measures the
average number of morphemes that children use per C-unit Age 6.0 0.7 5.0–7.0 .30*
and is an index of general grammatical skills that increases with Narrative macrostructure 19.0 3.0 11.0–26.0 —
age through the school-age years, particularly when analyzed (NSS)
Productivity (NTW) 264.1 77.2 133.0–608.0 .50**
using narratives and expositories (Leadholm & Miller, 1992; Vocabulary (NDW) 92.5 19.0 47.0–150.0 .58**
Nippold et al., 2005). Also, the SI, a measure of clausal density, Grammar (MLCU) 7.0 0.9 4.8–9.6 .44**
indicates the average number of subordinate clauses produced Grammar (SI) 1.1 0.1 1.0–1.3 .35**
per C-unit. Use of subordinate clauses emerges during the
preschool years (Diessel & Tomasello, 2000) and continues Note. NTW = number of total words; NDW = number of different
words; MLCU = mean length of C-unit; SI = subordination index.
to develop through the school-age years (Nippold, 2007).
Because the productions of oral narratives were relatively *p = .001. **p < .001.
short, issues of sample length required special consideration.
For conversational language samples, it is common practice to
control sample length across children by using a consistent also summarizes the bivariate correlations between the NSS,
number of utterances, words, or elapsed time (e.g., 50 utter- children’s age, and each of the microstructures. All correla-
ances; Miller, 1981). In the present study, children produced tions were significant and were moderate in strength (Cohen,
between 12 and 77 utterances. Using a standard transcrip- 1988).
tion cut of 50 utterances would limit the analyses to 11 tran- To further explore the covariance structure between the
scripts. If we chose to maintain 75% of the data, 97 transcripts variables and to identify unique relationships between the
would remain in the analysis, and each transcript would be microstructures and the NSS, two separate hierarchical re-
cut at 29 utterances and would ultimately withhold 955 utter- gression equations were completed. Hierarchical regressions
ances from the data set. Our goal was to maintain the maxi- allow examination of variance that is uniquely explained by a
mum amount of data available by maintaining the entire sample given variable. The first hierarchical regression equation is
and to use statistical procedures to account for differences summarized in Table 3. Sample length was first controlled by
in transcript length. Furthermore, using the entire sample for entering NTW into Model 1. Length was controlled because
linguistic analyses, including NDW, is common practice in measures of lexical diversity are inevitably influenced by the
contemporary studies of children’s oral narratives (Gazella & NTW in the sample (Malvern & Richards, 2002). That is, the
Stockman, 2003; Humphries et al., 2004; Swanson et al., more total words that a child produces, the more opportunity
2005; Uccelli & Páez, 2007). he or she has to produce different words.
Model 2 identified the unique relationship between vocab-
Results ulary and NSS after controlling for length. Taken together,
NTW and NDW were significantly correlated with NSS scores
The NSS scores were first reviewed to identify how many (r = .58). Adding NDW in Model 2 increased the explained
child and examiner errors precluded scores to be adminis- variance from 24% to 33%, a net increase of 9%. A one-way
tered. In the present study, a total of 903 scores were completed analysis of variance was completed to determine whether the
using the children’s narrative transcripts (seven categories were increase in explained variance was significant, and an f 2 sta-
scored across 129 transcripts). Across the 903 sections, only tistic was calculated to estimate its effect size. According to
six sections received scores of zero (four stories lacked an in- Cohen (1988), effect sizes of 0.02, 0.15, and 0.35 are con-
troduction, and two children omitted a conclusion), and only sidered small, medium, and large, respectively. The 9% increase
one of the 903 sections received a score of NA; 99% of the in explained variance was significant, F(1, 124) = 16.5,
NSS sections were able to be scored correctly, confirming that p < .001, f 2 = 0.12, demonstrating that NDW was uniquely
the training and elicitation procedures facilitated a high level
of child compliance and examiner fidelity during the elici-
tation process. TABLE 3. Summary of hierarchical regression analysis with
The skewness statistic was calculated for the NSS to de- grammatical measures uniquely predicting NSS scores in
termine if there was an unequal distribution of NSS scores Model 3.
across the sample (see Coolican, 2004, for a review). Skew-
ness measures of zero indicate a perfectly normal distribution, Model Predictors r Adjusted r 2 r 2 change
while skewness values below –0.8 or above 0.8 have been 1 Productivity (NTW) .50 .24*
described as “noticeably skewed” (Bourque & Clark, 1992, 2 Productivity (NTW) .58 .33* .09*
p. 69). The skewness statistic for the NSS was –0.5, indicat- Vocabulary (NDW)
ing that scores were more concentrated toward the ceiling 3 Productivity (NTW) .61 .35* .02
but were not noticeably skewed according to the criteria Vocabulary (NDW)
Grammar (MLCU, SI)
established by Bourque and Clark.
Descriptive statistics for age, the NSS, and each of the *Significant at p < .01.
microstructural measures are presented in Table 2. Table 2
Heilmann et al.: Properties of the Narrative Scoring Scheme 159related to children’s NSS scores above and beyond the vari- literate language, to measure a wider range of skills than tradi-
ability explained by NTW. tional story grammar analyses.
Model 3 was completed to identify if there was a unique To determine whether the NSS was developmentally ap-
relationship between the two grammatical measures (MLCU propriate for the children in this study, the skewness statistic
and SI) and NSS scores, after controlling for NTW and NDW. was calculated. This analysis revealed that the data were not
The measures in this final model were significantly correlated noticeably skewed according to Borque and Clark’s (1992)
with the NSS (r = .61). Adding MLCU and SI to the third criteria and that the NSS appeared to be a sensitive measure for
regression equation added an additional 2% prediction to NSS school-age children who produced an oral retell. (We are cur-
scores, which was not significant, F(2, 122) = 2.9, p = .060, rently completing a more thorough investigation of the de-
f 2 = 0.03. In this first hierarchical regression analysis, NDW velopmental sensitivity of the NSS as compared to alternative
uniquely predicted NSS scores after controlling for length methods of documenting narrative macrostructure skills.)
using NTW. The two grammatical measures, however, did To document the relationship between the microstructural
not add unique prediction of children’s NSS scores. To con- features of language samples and children’s performance on
firm that NDW was the major unique predictor of NSS scores, a the NSS, a series of correlation and hierarchical regression
second hierarchical regression equation was completed and is analyses were completed. These analyses confirmed that a close
summarized in Table 4. Again, sample length was controlled relationship existed between children’s productivity, vocab-
by entering NTW in the first model. The two grammatical ulary, grammar, and narrative macrostructure skills. The
measures were next entered in the second equation, which correlation analyses documented that age and each of the
resulted in a combined correlation of r = .57. Adding SI and microstructural measures (NTW, NDW, MLCU, and SI)
MLCU to NTW increased the explained variance from .24 were significantly correlated with children’s narrative organi-
to .30, documenting that the grammatical measures explain zation skills. It is noteworthy that the correlation between age
6% of the variance in NSS after controlling for sample length. and NSS was the weakest observed correlation. The socio-
This increase was significant, F(2, 123) = 6.0, p = .002, cultural theory of narrative macrostructure development pro-
f 2 = 0.08. NDW was added to the third model to test the poses that children who have more experience with stories
unique prediction of vocabulary on NSS scores. Adding NDW will have greater narrative competence (e.g., Eaton et al., 1999;
in the third model resulted in a 5% increase in explained Stein & Glenn, 1979). The children in the present sample
variance between the microstructures and NSS, which was spanned 2 years in age. The older children in this sample likely
significant, F(1, 122) = 8.7, p = .003, f 2 = 0.06. had more experience listening to and telling stories. How-
In sum, this series of hierarchical regression equations ever, it was the children’s vocabulary and grammar skills that
documented that children’s use of vocabulary is the major were most strongly related to their narrative macrostructure
significant and unique microstructural variable in predicting scores. While this study did not directly control for the amount
their story organization skills as measured by the NSS. Chil- of experience the children had with narratives, the data pro-
dren’s productive grammar, while significantly correlated vide some additional evidence for the importance of chil-
with NSS scores, did not provide unique prediction of the dren’s linguistic proficiency in predicting narrative organization
children’s narrative macrostructure ability. skills (Berman & Slobin, 1994; Bishop & Donlan, 2005).
Two separate hierarchical regression analyses were com-
pleted to identify the unique relationships between each of
Discussion the microstructures and NSS scores. After controlling for
Upon establishing a set of reference databases for chil- length, the unique relationship between the NSS and mea-
dren’s narrative retells, the Language Analysis Lab at the sures of vocabulary and grammar was calculated. The anal-
University of Wisconsin—Madison set out to identify a clini- yses showed that children’s productive vocabulary skills were
cally useful measure of children’s narrative organization skills. the only unique predictor of narrative organization skills.
The NSS was created to bring together the benefits of con- Grammatical measures, on the other hand, provided no unique
crete scoring criteria combined with judgment of text-level prediction of NSS scores. The unique importance of vocab-
constructs. The NSS also incorporated higher level narrative ulary in predicting narrative organization skills was a novel
components, including cohesive markers and measures of finding. Bishop and Donlan (2005) documented that children’s
use of complex syntax and expression of causal concepts
TABLE 4. Summary of hierarchical regression analysis with uniquely predicted children’s ability to organize their oral
vocabulary measures uniquely predicting NSS scores in narratives. Bishop and Donlan examined children between
Model 3. 7 and 9 years of age, while the present study investigated
children age 5–7 years. The children in the present study were
Model Predictors r Adjusted r 2 r 2 change using minimal complex syntax. As observed in Table 2, SI
1 Productivity (NTW) .50 .24*
values averaged 1.1, showing that children produce approx-
2 Productivity (NTW) .57 .30* .06* imately one subordinate clause every 10 utterances. The
Grammar (MLCU, SI) children’s use of subordination may have been influenced by
3 Productivity (NTW) .61 .35* .05* the story used in this study; the children’s SI values, on aver-
Grammar (MLCU, SI) age, were just slightly lower than the SI value from the story
Vocabulary (NDW)
script (SI = 1.15). These low levels of subordination could
*Significant at p < .01. explain, in part, the modest correlations and lack of a unique
relationship between the grammatical measures and the NSS.
160 American Journal of Speech-Language Pathology • Vol. 19 • 154–166 • May 2010Relationship Between Vocabulary and the NSS NSS alpha was not as high as the alpha for the SI. The dif-
The present study revealed that there is a special and im- ference in alphas between the NSS and SI was not surprising,
portant relationship between narrative organization and vo- however, as the NSS requires much greater individual judg-
cabulary skills that emerges prior to children becoming fully ment when compared to the relatively straightforward scoring
literate. This study demonstrated that the development of rules for the SI. However, recall that these subjective ratings
story schema and vocabulary acquisition is developing along are often the most sensitive when identifying children with
a similar path. The importance of vocabulary in narrative language impairment (e.g., McFadden & Gillam, 1996). Our
organization skills is not surprising given the literature de- goal is to continue developing new training methods and
scribing the development of narrative form. Preschool and complete additional research identifying ways to increase
young school-age children typically produce narratives that coding accuracy for the NSS and other macrostructural anal-
simply chain sequences of events in temporal order (Berman, yses of children’s natural language use.
1988) and provide simple descriptive sequences (Stein & Upon scoring the NSS, clinicians are provided with a clini-
Glenn, 1979). It is not until the later school-age years that cally useful benchmark for children’s overall narrative profi-
children hierarchically organize the events in their narrative ciency. The data described in this study used the composite
productions (Berman, 1988) and take multiple perspectives to NSS score, which was the summed score for all seven sec-
relate the events (Stein & Glenn, 1979). To produce these tions of the NSS. We proposed that the NSS composite score
more advanced narratives, children must use complex syntax provided an index of children’s overall narrative organization
(Bamberg & Damrad-Frye, 1991; Bishop & Donlan, 2005). skills. The NSS data described in this study are available as
Before children are proficient in using complex syntax, they part of the SALT Narrative Story Retell database and can be
likely have to rely on their vocabulary skills to organize their downloaded free of charge from the SALT Web site. SALT
narrative productions. Furthermore, the emerging literacy software, used to access the data, can compare a target child’s
literature has documented the importance of vocabulary in the NSS scores with those of age-matched peers. In addition to
development of children’s narrative and comprehension skills. comparing NSS scores to the SALT database, composite NSS
There is a well-documented relationship between children’s scores can be useful for monitoring progress and documenting
vocabulary skills and reading comprehension (see Scarborough, treatment outcomes by collecting multiple narrative samples
2001). Furthermore, preschool and young school-age chil- from a child and documenting changes in NSS scores over
dren acquire new vocabulary through repeated exposures to time.
narrative form (Robbins & Ehri, 1994; Senechal & Cornell, The NSS also provides examiners with the opportunity to
1993) and by receiving adult scaffolding that highlights identify specific aspects of the narrative process that are dif-
the story’s structure (Hargrave & Senechal, 2000; Penno, ficult for a child. Because the NSS separately judges seven
Wilkinson, & Moore, 2002). aspects of the narrative process, examiners can evaluate per-
formance on each section of the NSS to identify areas of
strength and areas that require intervention. Compared with
Limitations narrative macrostructure measures that make holistic text-
The results from the hierarchical regression equations level judgments of narrative proficiency, there is greater spe-
could have been affected by the sampling context. Because all cificity in the NSS scoring procedure. Having a detailed
of the measures were collected from a single language sam- narrative performance profile facilitates a more accurate de-
ple, high internal validity was achieved. While each of the scription of the child’s performance and can assist in the
language sample measures theoretically and empirically re- development of treatment goals. For example, a child who
flects its respective linguistic domain, strong intercorrelations performed poorly on the referencing and cohesion sections of
between language sample measures have been observed (Miller the NSS but did well on the other sections likely has difficulty
& Klee, 1995). The hierarchical regression equations did af- using cohesive devices. Treatment goals could include im-
ford better understanding of the covariance structure. How- proving the child’s use of referential, lexical, and conjunctive
ever, vocabulary and grammatical measures acquired from cohesion.
alternate tasks may provide a more informative test of the Increasing our understanding of the relationships between
relationship between vocabulary, grammar, and narrative vocabulary, grammar, and narrative macrostructure has im-
macrostructure. portant clinical implications for documenting functional out-
comes and identifying treatment goals. One goal of language
intervention programs is for the therapy tasks to generalize
Clinical Implications to functional tasks. Telling a well-formed narrative is a func-
The NSS was created as a clinically useful comprehensive tional task that is important to children’s academic success
narrative macrostructure measure. To be clinically feasible, (Bishop & Edmundson, 1987; Griffin, Hemphill, Camp, &
an assessment tool must be able to be completed in a short Wolf, 2004; O’Neill, Pearce, & Pick, 2004). The data from the
amount of time to accommodate the busy schedule of the SLP. present study demonstrated that vocabulary skills were uniquely
In our lab, trained transcribers could complete the NSS using related to children’s story organization skills. Therefore, a
a narrative transcript in approximately 3 min. In addition to treatment procedure that increases a child’s vocabulary skills
efficient scoring, the NSS was developed to facilitate accurate would have broader, more functional outcomes if concurrent
scoring both within and between examiners. The Krippen- increases in his or her narrative macrostructure skills were
dorff alpha analyses revealed that the NSS had lower inter- documented. Similarly, we may expect that treatments ad-
rater agreement than we would ideally observe and that the dressing narrative macrostructure skills could also result in
Heilmann et al.: Properties of the Narrative Scoring Scheme 161concurrent increases in vocabulary skills. Clearer understand- their typically developing peers. American Journal of Speech-
ing of the relationship between microstructures and macro- Language Pathology, 8, 249–260.
structures will facilitate selection of appropriate treatment Bourque, L., & Clark, V. (1992). Processing data: The survey
goals. Understanding these relationships can aid in identifying example. Thousand Oaks, CA: Sage.
the appropriate microstructures to address when implement- Brown, R. (1973). A first language: The early stages. Cambridge,
MA: Harvard University Press.
ing interventions that focus on linguistic macrostructures, such
Clahsen, H. (1989). The grammatical characterization of develop-
as narrative organization. It is important to note that the data mental dysphasia. Linguistics, 27, 897–920.
in this study are purely correlational and that further research Cohen, J. (1988). Statistical power analysis for the behavioral
is needed before causal relationships between vocabulary, sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
grammar, and narrative macrostructure can be identified. Colozzo, P., Garcia, R., Megan, C., Gillam, R., & Johnston, J.
In sum, narrative language assessment is an effective (2006, June). Narrative assessment in SLI: Exploring interac-
method for documenting children’s language skills. The NSS tions between content and form. Poster session presented at
was developed by the Language Analysis Lab as a clinically the annual meeting of the Symposium on Research in Child
useful index of children’s narrative organization skills. Given Language Disorders, Madison, WI.
its clinical feasibility and its robust relationship with other Coolican, H. (2004). Research methods and statistics in psychol-
linguistic measures, the NSS provides clinicians and re- ogy. London, England: Hodder & Stoughton.
searchers with an additional tool to document children’s global Curenton, S., & Justice, L. (2004). African American and Cauca-
sian preschoolers’ use of decontextualized language: Literate
language skills using a functional and curriculum-oriented
language features in oral narratives. Language, Speech, and
task. Hearing Services in Schools, 35, 240–253.
Diessel, H., & Tomasello, M. (2000). The development of relative
clauses in spontaneous child speech. Cognitive Linguistics, 11,
Acknowledgments 131–151.
Eaton, J., Collis, G., & Lewis, V. (1999). Evaluative explanations
This research was supported by National Institutes of Health in children’s narratives of a video sequence without dialogue.
Grant 5 T32 DC005459 (“Interdisciplinary Research Training in Journal of Child Language, 26, 699–720.
Speech-Language Disorders”) and by the Language Analysis Lab at
Gazella, J., & Stockman, I. (2003). Children’s story retelling
the University of Wisconsin—Madison. This work was also made
under different modality and task conditions: Implications for
possible through close collaboration with two public school districts
standardizing language sampling procedures. American Journal
in San Diego County: San Diego City Schools and Cajon Valley
of Speech-Language Pathology, 12, 61–72.
Union Schools. The authors express appreciation to Karen Andriacchi
Gillam, R., & Johnston, J. (1992). Spoken and written language
and the research assistants from the Language Analysis Lab for their
relationships in language/learning-impaired and normally achiev-
assistance with transcription and organization of the data. Portions
ing school-age children. Journal of Speech and Hearing Research,
of this article were presented at the 2006 American Speech-Language-
35, 1303–1315.
Hearing Association Annual Convention in Miami, FL.
Gopnik, M., & Crago, M. (1991). Familial aggregation of a devel-
opmental language disorder. Cognition, 39, 1–50.
Greenhalgh, K., & Strong, C. (2001). Literate language features in
References spoken narratives of children with typical language and children
Allen, M., Kertoy, M., Sherblom, J., & Pettit, J. (1994). Chil- with language impairments. Language, Speech, and Hearing
dren’s narrative productions: A comparison of personal event Services in Schools, 32, 114–125.
and fictional stories. Applied Psycholinguistics, 15, 149–176. Griffin, T., Hemphill, L., Camp, L., & Wolf, D. (2004). Oral
Applebee, A. (1978). The child’s concept of story: Ages two to discourse in the preschool years and later literacy skills. First
seven. Chicago, IL: University of Chicago Press. Language, 24, 123–147.
Bamberg, M., & Damrad-Frye, R. (1991). On the ability to pro- Halliday, M., & Hasan, R. (1976). Cohesion in English. London,
vide evaluative comments: Further explorations of children’s England: Longman.
narrative competencies. Journal of Child Language, 18, 689–710. Hargrave, A., & Senechal, M. (2000). A book reading interven-
Berman, R. (1988). On the ability to relate events in narrative. tion with preschool children who have limited vocabularies: The
Discourse Processes, 11, 469–497. benefits of regular reading and dialogic reading. Early Child-
Berman, R., & Slobin, D. (Eds.). (1994). Relating events in nar- hood Research Quarterly, 15, 75–90.
rative: A crosslinguistic developmental study. Hillsdale, NJ: Hedberg, N., & Westby, C. (1993). Analyzing storytelling skills:
Erlbaum. Theory to practice. Tucson, AZ: Communication Skill Builders.
Bishop, D., & Donlan, C. (2005). The role of syntax in encoding Heilmann, J., Miller, J., Iglesias, A., Fabiano-Smith, L.,
and recall of pictorial narratives: Evidence from specific lan- Nockerts, A., & Andriacchi, K. (2008). Narrative transcription
guage impairment. British Journal of Developmental Psychol- accuracy and reliability in two languages. Topics in Language
ogy, 23, 25–46. Disorders, 28, 178–188.
Bishop, D., & Edmundson, A. (1987). Language-impaired 4-year- Humphries, T., Cardy, J., Worling, D., & Peets, K. (2004).
olds: Distinguishing transient from persistent impairment. Narrative comprehension and retelling abilities of children
Journal of Speech & Hearing Disorders, 52, 156–173. with nonverbal learning disabilities. Brain and Cognition, 56,
Boudreau, D. (2007). Narrative abilities in children with language 77–88.
impairment. In R. Paul (Ed.), Language disorders from a devel- Justice, L., Bowles, R., Kaderavek, J., Ukrainetz, T., Eisenberg,
opmental perspective: Essays in honor of Robin S. Chapman S., & Gillam, R. (2006). The Index of Narrative Microstructure:
(pp. 331–356). Mahwah, NJ: Erlbaum. A clinical tool for analyzing school-age children’s narrative
Boudreau, D., & Hedberg, N. (1999). A comparison of early performances. American Journal of Speech-Language Pathol-
literacy skills in children with specific language impairment and ogy, 15, 177–191.
162 American Journal of Speech-Language Pathology • Vol. 19 • 154–166 • May 2010Klee, T. (1992). Developmental and diagnostic characteristics of early narrative and later mathematical ability. First Language,
quantitative measures of children’s language production. Topics 24, 149–183.
in Language Disorders, 12, 28–41. Paul, R., & Smith, R. (1993). Narrative skills in 4-year-olds with
Krippendorff, K. (1980). Content analysis: An introduction to its normal, impaired, and late-developing language. Journal of
methodology. Thousand Oaks, CA: Sage. Speech and Hearing Research, 36, 592–598.
Labov, W., & Waletzky, J. (1967). Narrative analysis: Oral ver- Pearce, W., McCormack, P., & James, D. (2003). Exploring
sions of personal experience. In J. Helm (Ed.), Essays on the the boundaries of SLI: Findings from morphosyntactic and
verbal and visual arts (pp. 12–44). Seattle: University of story grammar analyses. Clinical Linguistics & Phonetics, 17,
Washington Press. 325–334.
Leadholm, B., & Miller, J. F. (1992). Language sample analysis: Pelligrini, A., Galda, L., Bartini, M., & Charak, D. (1998). Oral
The Wisconsin guide. Madison, WI: Department of Public language and literacy learning in context: The role of social
Instruction. relationships. Merrill-Palmer Quarterly, 44, 38–54.
Liles, B. (1985). Cohesion in the narratives of normal and language- Penno, J., Wilkinson, I., & Moore, D. (2002). Vocabulary acquisi-
disordered children. Journal of Speech and Hearing Research, tion from teacher explanation and repeated listening to stories:
28, 123–133. Do they overcome the Matthew effect? Journal of Educational
Malvern, D., & Richards, B. (2002). Investigating accommoda- Psychology, 94, 23–33.
tion in language proficiency interviews using a new measure of Peterson, C., & McCabe, A. (1983). Developmental psycholin-
lexical diversity. Language Testing, 19, 85–104. guistics: Three ways of looking at a child’ s narrative. New York,
Mandler, J., & Johnson, N. (1977). Remembrance of things parsed: NY: Plenum Press.
Story structure and recall. Cognitive Psychology, 9, 111–151. Peterson, C., & McCabe, A. (2004). Echoing our parents: Parental
Manhardt, J., & Rescorla, L. (2002). Oral narrative skills of late influences on children’s narration. In M. Pratt & B. Fiese (Eds.),
talkers at ages 8 and 9. Applied Psycholinguistics, 23, 1–21. Family stories and the life course: Across time and generations
Mayer, M. (1969). Frog, where are you? New York, NY: Dial (pp. 27–54). Mahwah, NJ: Erlbaum.
Press. Quasthoff, U. (1997a). An interactive approach to narrative devel-
McCabe, A. (1997). Developmental and cross-cultural aspects of opment. In M. Bamberg (Ed.), Narrative development: Six
children’s narration. In G. Bamberg (Ed.), Narrative develop- approaches (pp. 51–83). Mahwah, NJ: Erlbaum.
ment: Six approaches (pp. 137–174). Mahwah, NJ: Erlbaum. Quasthoff, U. (1997b). Were you ever in a situation where you
McFadden, T., & Gillam, R. (1996). An examination of the quality were in serious danger of being killed? Narrator-listener inter-
of narratives produced by children with language disorders. action in Labov and Waletzky’s narratives. Journal of Narrative
Language, Speech, and Hearing Services in Schools, 27, 48–56. & Life History, 7, 121–128.
Merritt, D., & Liles, B. (1987). Story grammar ability in children Reilly, J., Losh, M., Bellugi, U., & Wulfeck, B. (2004). Frog,
with and without language disorder: Story generation, story Where are you? Narratives in children with specific language
retelling, and story comprehension. Journal of Speech and impairment, early focal brain injury and Williams syndrome.
Hearing Research, 30, 539–552. Brain & Language, 88, 229–247.
Miles, S., & Chapman, R. S. (2002). Narrative content as described Rice, M. L., Wexler, K., & Cleave, P. L. (1995). Specific language
by individuals with Down syndrome and typically developing impairment as a period of extended optional infinitive. Journal
children. Journal of Speech, Language, and Hearing Research, of Speech and Hearing Research, 38, 850–863.
45, 175–189. Robbins, C., & Ehri, L. (1994). Reading storybooks to kinder-
Miller, J. F. (1981). Assessing language production in children. garteners helps them learn new vocabulary words. Journal of
Baltimore, MD: University Park Press. Educational Psychology, 86, 56–64.
Miller, J. F. (1987). A grammatical characterization of language Rumelhart, D. (1975). Notes on a schema for stories. In D. G.
disorder. In J. Martin, P. Fletcher, R. Grunwell, & D. Hall (Eds.), Bobrow & A. M. Collins (Eds.), Representation and under-
Proceedings of the First International Symposium on Specific standing: Studies in cognitive science (pp. 211–236). New
Speech and Language Disorders in Children (pp. 100–113). York, NY: Academic Press.
London, England: Association for All Speech Impaired Children. Scarborough, H. (2001). Connecting early language and literacy to
Miller, J., Heilmann, J., Nockerts, A., Iglesias, A., Fabiano, L., later reading (dis)abilities: Evidence, theory, and practice. In
& Francis, D. (2006). Oral language and reading in bilingual S. Neuman & D. Dickinson (Eds.), Handbook for research in
children. Learning Disabilities Research & Practice, 21, early literacy (pp. 97–110). New York, NY: Guilford Press.
30–43. Schuele, C., & Tolbert, L. (2001). Omissions of obligatory relative
Miller, J. F., & Iglesias, A. (2008). Systematic Analysis of Lan- markers in children with specific language impairment. Clinical
guage Transcripts (Research Version 9.1) [Computer software]. Linguistics & Phonetics, 15, 257–274.
Middleton, WI: SALT Software. Scott, C., & Stokes, S. (1995). Measures of syntax in school-age
Miller, J., & Klee, T. (1995). Computational approaches to the anal- children and adolescents. Language, Speech, and Hearing Services
ysis of language impairment. In P. Fletcher & B. MacWhinney in the Schools, 26, 309–319.
(Eds.), The handbook of child language (pp. 545–572). Oxford, Senechal, M., & Cornell, E. (1993). Vocabulary acquisition through
England: Blackwell. shared reading experiences. Reading Research Quarterly, 28,
Nippold, M. (2007). Later language development: School-age 360–374.
children, adolescents, and young adults (3rd ed.). Austin, TX: Slobin, D. (1973). Cognitive prerequisites for the development of
Pro-Ed. grammar. In C. Ferguson & D. Slobin (Eds.), Studies of child
Nippold, M., Hesketh, L., Duthie, J., & Mansfield, T. (2005). language development (pp. 175–208). New York, NY: Aca-
Conversational versus expository discourse: A study of syntactic demic Press.
development in children, adolescents, and adults. Journal of Stein, N. L. (1988). The development of storytelling skill. In
Speech, Language, and Hearing Research, 48, 1048–1064. M. Franklin & S. Barten (Eds.), Child language: A reader
O’Neill, D., Pearce, M., & Pick, J. (2004). Preschool children’s (pp. 282–297). New York, NY: Oxford University Press.
narratives and performance on the Peabody Individualized Stein, N., & Glenn, C. (1979). An analysis for story comprehen-
Achievement Test—Revised: Evidence of a relation between sion in elementary school. In R. Freedle (Ed.), New directions
Heilmann et al.: Properties of the Narrative Scoring Scheme 163You can also read
