Do Strikes and Work-to-Rule Campaigns Change Elementary School Assessment Results?
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Do Strikes and Work-to-Rule Do Strikes and Work-to-Rule Campaigns Change Elementary School Assessment Results? 479 Campaigns Change Elementary https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26 School Assessment Results? David R. Johnson School of Business and Economics Wilfrid Laurier University Waterloo, Ontario Dans cet article, à partir de données portant sur l’évaluation des élèves ontariens de la 3e année à la 6e année, j’analyse le lien qu’il est possible d’établir entre le rendement scolaire et les grèves et grèves du zèle des enseignants. De 1998-1999 à 2003-2004, environ 5 % des écoles ontariennes ont été touchées par une grève (et un lock-out), et environ 10 % par une grève du zèle (c’est-à-dire le refus des enseignants de participer à certaines activités scolaires auxquelles ils collaborent normalement). Les résultats indiquent qu’il y a eu en 3e année et en 6e année dans les écoles de milieux défavorisés une baisse marquée des résultats scolaires pendant les grèves, ainsi qu’une baisse, mais moins importante, pendant les grèves du zèle. Les baisses observées pendant les grèves du zèle varient selon les écoles : les plus élevées concernent les élèves de 3e année des écoles de milieux défavorisés, et les moins élevées les élèves de 6e année des écoles de milieux favorisés. Mots clés : grève des enseignants, rendement scolaire, grève du zèle This paper uses data from the Grade 3 and 6 assessments conducted in Ontario schools to examine the as- sociation between student achievement and both strikes and work-to-rule campaigns by teachers. Between 1998/99 and 2003/04 approximately 5 percent of schools in Ontario were exposed to strikes (including one lockout) and over 10 percent of schools experienced a work-to-rule campaign (the withdrawal of many school activities normally carried out by teachers). This study finds large reductions in academic achievement associated with strikes at disadvantaged schools in both Grade 3 and Grade 6, and substantial reductions in results associated with work-to-rule campaigns. The reductions associated with work-to-rule campaigns vary across schools with the largest reductions at disadvantaged schools in Grade 3 and at ad- vantaged schools in Grade 6. Keywords: strikes by teachers, student achievement, work-to-rule campaigns by teachers Introduction unionized. Yet the legal ability of teachers and boards to strike varies. In Canada, teachers in T eachers in Ontario, as in all Canadian jurisdictions, in many American jurisdictions, and throughout much of the developed world are Manitoba and Prince Edward Island do not have the right to strike while in Nova Scotia and Quebec the right to strike is restricted (Wright 2004). In Canadian Public Policy – Analyse de politiques, vol . xxxvii , no . 4 2011 CPPDec2011Inside.indb 479 29/11/11 4:07 PM
480 David R. Johnson the United States, only ten states allow teachers to this paper finds that work-to-rule campaigns have strike (Wright 2004). This paper compares student strong, negative effects on student achievement. assessment results at elementary schools that have https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26 experienced strikes or work-to-rule campaigns The analysis in this paper also shows that, in all (collectively called labour relations disruptions) to cases, the length of the labour relations disruption assessment results at the same elementary schools matters: the longer the disruption, the larger the in other years without a disruption. A work-to-rule predicted reduction in achievement. For strikes, the campaign is a common strategy where teachers come lost instructional days create a learning gap between to school but do not work beyond the limited duties schools that experienced stoppages and schools specified in their contracts. Typical duties withdrawn that did not experience stoppages. The quantity include running extracurricular activities, meeting of instructional time matters. In a similar way, a with parents, or attending administrative meetings. work-to-rule campaign may affect the quantity of In the period of analysis there were 15 strikes instructional time if the normal instructional time (including one lockout treated here as a strike) and goes beyond the contract instructional time. A work- 20 work-to-rule campaigns in Ontario. The key to-rule may also affect the quality of instructional question: Does academic performance, as measured time. Assessment results are lower after a work-to- by student assessment results, fall in the year of rule campaign in both Grade 3 and Grade 6, which these labour relations disruptions? suggests that teachers cannot teach at least some groups of students as effectively within the contract The average observed change in the assessment rules enforced by a work-to-rule campaign. results associated with disruptions across all schools is negative but often small and not statistically signifi- There is a large American literature, reviewed in cant. The largest reductions in results are observed in Carini (2008), attempting to understand if there is a the mathematics assessment. There is a large reduction relationship between school outcomes and unioniza- in reading and mathematics results associated with tion at the level of state, district, or student. A small strikes in both Grade 3 and Grade 6 for schools drawn portion of the American literature on the effect of from the lowest parts of the distribution of parental unionization includes various measurements asking education. Strikes reduce learning outcomes by the if strikes are associated with lower academic out- largest amount at precisely the schools where children comes. In an early review of the literature relating arrive with other disadvantages. strikes to achievement, Zirkel (1992) concluded that there was little evidence that teacher strikes affected A second important result is that work-to-rule attitude, attendance, or achievement of public school campaigns reduce learning outcomes on average. students. Since the Zirkel survey, two American The reductions are larger at disadvantaged schools in studies and one Canadian study have been conducted Grade 3 and at advantaged schools in Grade 6. This that are similar to the study reported here. is a controversial result. Work-to-rule campaigns, which are a common feature of labour relations dis- Thornicroft (1994) uses achievement results on ruptions in Ontario, might be considered to have no a battery of standardized tests administered only in detrimental effects on student outcomes. Patterson, the 1989/90 academic year in 612 Ohio districts. a union official, is quoted with respect to a work- Between 1984/85 and 1991/92, there were 61 strikes to-rule: “That is the action we see as appropriate, as averaging ten days in duration in these districts. A it brings pressure on the board without having any comparison of average achievement results between impact on the classrooms or the students” (Kingston strike and no-strike districts found few significant Whig-Standard, 10 October 2002). The evidence in differences. If there were any differences, districts Canadian Public Policy – Analyse de politiques, vol . xxxvii , no . 4 2011 CPPDec2011Inside.indb 480 29/11/11 4:07 PM
Do Strikes and Work-to-Rule Campaigns Change Elementary School Assessment Results? 481 with strikes had better results. A regression model Baker (2011) studies strikes in the Ontario school of achievement that included a variety of variables system over the same period and with much of the related to strikes found no significant relationships same data that I use. Baker’s primary strategy is to https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26 between achievement and the strike measures. The compare, across schools, the growth in test scores strike measures included an indicator for any strike between a Grade 3 and a Grade 6 “cohort” at schools in the entire six-year period, the number of strike with and without strikes. The cohorts are not exact days over the six-year period, a strike in the assess- because, as in my study, the data are at the school ment year, number of strike days in the assessment level and students enter and leave schools. Baker year, a strike indicator for each year in the data, and presents a large number of estimates that examine the number of days on strike for each year in the the sensitivity of his results to different choices data. There were a series of control variables in the for the group of schools without strikes, which regressions. The most important variables measure function as the control group. Baker finds that the social and economic differences across districts in largest effects are found in a reduction of Grade 6 income although there are some measures of school mathematics test score growth with a long strike, spending and teacher quality. About half the varia- defined as a strike of ten days or more that occurred tion in achievement across districts is associated in Grade 5 or Grade 6 of the Grade 6 class assessed. with variation in the control variables. Thornicroft I also find the largest effects of current-year strikes (1994) concludes strikes do not affect achievement. on mathematics results in Grade 6. Baker did not find that a long-ago strike in Grade 2 or 3 had any In a more recent study, Zwerling (2008) compares significant impact on the growth of test scores be- outcomes on a series of annual standardized tests tween Grade 3 and Grade 6 for the cohort exposed in many grades across districts in Pennsylvania to that strike. with and without strikes. There are three measures that describe students: the percentage who receive The paper proceeds as follows. The next section a free or reduced-price lunch, the percentage who outlines the labour relations system in place in On- are black or Hispanic, and the percentage who have tario between 1998/99 and 2003/04 and presents the individual education plans (both disabled and gift- data on strikes and work-to-rule activity. The third ed). Other other control variables describe district section presents the data on achievement, some characteristics relating to teachers and expenditures. preliminary data describing outcomes, and the social There are two measures of strike activity: a simple and economic characteristics of schools with and indicator variable if there is a strike in the district in without disruptions. The fourth section describes the test year or in the previous year, and a variable the methodology used to analyze the relationships measuring the number of days of the strike. Neither between disruptions and outcomes more carefully. of the two strike variables has a clear association The fifth section presents the results, followed by with achievement outcomes. The two strike vari- conclusions. ables are also interacted with the three variables describing students. Coefficients on the interaction terms are not statistically significant. Zwerling Education Labour Relations Disruptions (2008) concludes there is no evidence that strikes in O ntario 1998/99 to 2003/04 affect achievement. In his survey of the interaction of education and collective bargaining, Carini In the period studied, 75 individual school boards (2008) suggests that the effects of strikes should negotiated separate two-year contracts with locals be investigated more carefully at a lower level of of the provincewide union of teachers. The boards disaggregation, ideally with student-level data. received roughly equal funding each year per student Canadian Public Policy – Analyse de politiques, vol . xxxvii , no . 4 2011 CPPDec2011Inside.indb 481 29/11/11 4:07 PM
482 David R. Johnson from the provincial government (with small adjust- alternate years of the data there are no disruptions ments for items like second-language instruction because contracts run for two years. For example, and rural transportation). Boards had no access in the first year of the data, contracts ran from 1 https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26 to a local tax base to pay increased wages. Thus a September 1998 to 31 August 2000 at all boards. “successful” (from the union point of view) labour The two-year cycle is repeated over the remaining relations disruption could not increase total financial four years of the data (Rose 2002). resources at a board. In spite of these constraints, elementary teach- Labour Relations Disruptions and ers at 14 boards went on strike. In addition, there the A cademic P erformance of was one lockout over the six years studied (which I Elementary Students in Ontario: include in the analysis and label as a strike).1 Table Preliminaries 1 presents the basic facts on the 15 strikes and 20 work-to-rule campaigns in 1998/99, 2000/01, and The Measurement of Academic Performance 2002/03. Two boards experienced both a strike and The provincewide data on student learning out- a work-to-rule campaign. In the first year of the data comes, by school for all schools with 15 or more almost 500 schools (about 15 percent) had a strike. students in the relevant grade, are drawn from the There were fewer strikes in 2000/01 and even fewer Grade 3 and Grade 6 Education Quality Account- in 2002/03. Strikes ranged in length from 1 day to 16 ability Office (EQAO) assessments.2 Assessments days of school. Work-to-rule campaigns were much were first written in 1996/97, and results for most longer in duration. In 2002/03, almost 40 percent of schools in the province are available beginning in schools experienced a work-to-rule campaign. In the 1998/99. The usual measure of school success is Table 1 The Incidence of Labour Relations Disruptions in Ontario Elementary Schools: 1998–2003 Academic Year and Type Number Number Numbers of Instructional Percentage Average Standard Deviation of Labour Relations of Boards of Schools Days Affected by Disruptions of Schools Length of of Disruption Disturbance with This Affected (shortest to longest) in Province Disruption over Length over All Disruption Affected All Schools Schools 1998/99 Strikes 8 484 1, 1, 5, 8, 9, 11, 11, 15 14.9 1.1 3.2 2000/01 Strikes 4 205 3, 10, 14, 16 6.8 0.8 3.3 2000/01 Work-to-rule 6 828 6, 18, 22, 23, 25, 27 23.5 5.4 10.1 2002/03 Strikes 3 247 5, 11, 12 8.5 0.7 2.8 2002/03 Work-to-rule 14 1,316 7, 9, 11, 13, 18, 19, 21, 24, 38.4 9.7 14.6 28, 29, 29, 30, 31, 41 Notes: There were no labour relations disruptions in academic years 1999/00, 2001/02, and 2003/04. Contracts were two years in length, and negotiations and thus disturbances took place every other academic year. There were no work-to-rule campaigns associated with negotiations during the 1998/99 academic year. Source: Author’s compilation from data provided by the Ontario Education Relations Commission. Canadian Public Policy – Analyse de politiques, vol . xxxvii , no . 4 2011 CPPDec2011Inside.indb 482 29/11/11 4:07 PM
Do Strikes and Work-to-Rule Campaigns Change Elementary School Assessment Results? 483 the percentage of all students who obtain a Level 3 Table 3 (and later Table 4) make it clear that (meets expectations) or a Level 4 (exceeds expecta- while it is important to take into account the varia- tions) result. For convenience this is called the pass tion by year and type of test in trying to explain the https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26 rate. There is a reading assessment, a writing assess- association between labour relations disruptions ment, and a mathematics assessment. At a school and achievement, it is not enough. The top panel with both Grade 3 and Grade 6 classes in all six of Table 3 considers groups of schools with and years of the study period, there are 36 assessment without either type of labour relations disruption results, 18 in each grade. in the year of those assessments, the middle panel compares schools with and without strikes, and the Table 2 presents distributional statistics for pass lower panel compares schools with and without rates, by year and by type of assessment, in Grade work-to-rule campaigns. The table presents uncon- 3 and Grade 6. The key aspect is that the various ditional differences in average pass rates for groups measures of distributional spread (more specific- of schools on those assessments in a specific year. ally, the standard deviation and interquartile range) remain very similar within a test across years while This table reinforces ideas already presented. the average pass rate is clearly different in different Large differences within a year in average pass years on the same assessment. Because the assess- rates on different assessments are clear. Large dif- ments are designed to be comparable across years, ferences between average pass rates on the same four (at least) non-mutually exclusive interpretations assessment in different years are clear. The most of the changes over time in the average pass rate on interesting aspect of Table 3 is that if we simply a specific assessment are consistent with the data in compared the pass rate within a year and assess- Table 2. Ontario elementary students could be get- ment type at schools with and without different ting better results over time with this gain attributed labour relations disruptions using a test of equality to improved teaching and/or a better curriculum. of the mean pass rate for the two groups, we might Ontario teachers could be learning, over time, to conclude disruptions raise results significantly (as deliver the curriculum that is being assessed rather they do in 27 cases), lower results significantly (as than some other curriculum previously taught. The they do in 15 cases), or leave results unchanged (as third possibility is that the assessments could vary they do in 6 cases). How can this be? The answer in difficulty across years. Finally, it is possible that, is straightforward. Labour relations disruptions af- even if the assessments are equal in difficulty, the fect all schools within a board. Boards and schools same answers on an assessment of equal difficulty within boards have different social and economic are graded differently. Either of the last two reasons characteristics. These differences must be taken could explain the declines in average provincial into account to properly measure the association of results between pairs of adjacent years on the same labour relations disruptions and pass rates. assessment. I find such large changes between adja- cent years in the quality of either all the teachers or The Social and Economic Background of all the students in the cohorts assessed implausible Elementary Students and conclude it seems very likely that the assessment Table 4 presents summary statistics of the social and instruments, in spite of all the effort put into compar- economic characteristics of students at all schools ability, are not always comparable. For the purposes in Ontario, and then uses as groups the schools that of this paper, Table 2 shows that an accurate estimate did and not experience labour relations disruptions of the effects of labour relations breakdowns must at any point in the six years studied. A full explana- take into account the variation in average assessment tion of the methods used to measure these social results by year and by type of assessment. and economic characteristics is found in Johnson Canadian Public Policy – Analyse de politiques, vol . xxxvii , no . 4 2011 CPPDec2011Inside.indb 483 29/11/11 4:07 PM
484 David R. Johnson Table 2 Descriptive Statistics for Pass Rates on EQAO Assessments 1998/99 to 2002/03 https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26 Year Mean Standard Deviation Median 25th Percentile 75th Percentile Interquartile Range Grade 3 reading assessment 1998/99 43.7 17.0 43 32 56 24 1999/00 48.0 18.0 48 35 61 26 2000/01 47.7 18.0 48 36 60 24 2001/02 49.3 19.0 49 36 62 26 2002/03 49.5 17.8 50 37 62 25 2003/04 53.3 18.0 54 41 66 25 Grade 3 writing assessment 1998/99 50.8 16.2 52 40 62 22 1999/00 50.7 16.9 51 39 62 23 2000/01 50.2 17.3 51 39 62 23 2001/02 54.0 16.7 54 43 66 23 2002/03 54.6 15.6 55 44 66 22 2003/04 57.4 15.7 58 47 69 22 Grade 3 mathematics assessment 1998/99 54.2 20.2 55 40 69 29 1999/00 55.8 21.3 56 40 72 32 2000/01 58.4 21.4 60 44 75 31 2001/02 56.8 21.4 58 42 72 30 2002/03 55.8 21.0 56 41 72 31 2003/04 63.3 19.6 65 50 78 28 Grade 6 reading assessment 1998/99 48.5 17.1 48 37 60 23 1999/00 50.6 17.9 50 38 63 25 2000/01 53.7 17.7 54 42 66 24 2001/02 54.3 17.6 55 43 67 24 2002/03 55.4 16.9 56 44 67 23 2003/04 57.7 17.0 58 46 70 24 Grade 6 writing assessment 1998/99 47.7 16.5 48 37 59 22 1999/00 48.0 17.2 48 36 59 23 2000/01 51.8 16.9 52 41 63 22 2001/02 52.5 16.4 53 42 64 22 2002/03 53.1 15.8 53 43 65 22 2003/04 53.9 16.1 54 43 65 22 Grade 6 mathematics assessment 1998/99 46.4 19.6 46 32 59 22 1999/00 51.5 19.9 52 37 59 23 2000/01 53.9 20.0 54 40 63 22 2001/02 53.2 20.1 54 40 64 22 2002/03 53.7 19.9 54 40 65 22 2003-04 57.9 19.8 59 44 65 22 Notes: EQAO = Education Quality Accountability Office. Source: Author’s calculations from EQAO results. Canadian Public Policy – Analyse de politiques, vol . xxxvii , no . 4 2011 CPPDec2011Inside.indb 484 29/11/11 4:07 PM
Do Strikes and Work-to-Rule Campaigns Change Elementary School Assessment Results? 485 Table 3 Comparing Average Pass Rates (percentage of students at Level 3 or Level 4) at Schools with and without Labour Relations https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26 Disruptions within a Year of an Assessment Test Year Grade 3 Grade 6 No Disruption Disruption Diff.a No Disruption Disruption Diff.a A labour relations disruption of any type Reading 1998 44.3 (.33) 47.3 (.72) 3.3*** 49.0 (.33) 50.5 (.76) 1.5** Writing 1998 51.5 (.31) 54.4 (.70) 3.3*** 48.1 (.32) 49.7 (.77) 1.6** Math 1998 55.2 (.38) 58.8 (.83) 3.5*** 46.8 (.39) 48.3 (.90) 1.5* Reading 2000 48.0 (.36) 49.0 (.65) 1.0* 53.7 (.37) 56.7 (.67) 3.0*** Writing 2000 50.4 (.35) 52.8 (.62) 2.4*** 51.9 (.35) 54.2 (.65) 2.3*** Math 2000 58.1 (.44) 62.4 (.72) 4.3*** 53.4 (.41) 57.7 (.79) 4.3*** Reading 2002 50.4 (.41) 48.7 (.54) –1.7*** 57.0 (.40) 53.5 (.53) –3.5*** Writing 2002 54.8 (.36) 54.9 (.46) 0.1 54.0 (.37) 52.0 (.48) –2.0*** Math 2002 56.2 (.49) 56.0 (.61) –0.2 55.0 (.47) 51.9 (.61) –3.1** A strike or lockout Reading 1998 44.2 (.33) 47.3 (.73) 3.1*** 49.0 (.33) 50.5 (.76) 1.5** Writing 1998 51.5 (.31) 54.3 (.71) 2.8*** 48.1 (.32) 49.7 (.77) 1.6** Math 1998 55.2 (.38) 58.7 (.83) 3.5*** 46.8 (.39) 48.3 (.90) 1.5** Reading 2000 48.6 (.32) 43.6 (1.4) –5.0*** 54.5 (.33) 49.7 (1.7) –4.8*** Writing 2000 51.3 (.31) 45.7 (1.5) –5.4*** 52.7 (.31) 47.7 (1.8) –5.0*** Math 2000 59.6 (.38) 54.5 (1.9) –5.1*** 54.9 (.37) 45.2 (2.0) –9.7*** Reading 2002 49.7 (.34) 50.5 (1.2) 0.8 55.6 (.33) 57.2 (1.1) 1.6* Writing 2002 54.7 (.30) 58.0 (1.0) 3.2*** 53.1 (.33) 56.7 (1.0) 3.6*** Math 2002 56.1 (.40) 55.9 (1.4) –0.6 53.9 (.39) 51.9 (1.4) –2.0* A work-to-rule campaign Reading 2000 48.0 (.36) 49.4 (.69) 1.4** 53.6 (.36) 57.2 (.70) 3.6*** Writing 2000 50.4 (.34) 53.1 (.67) 2.7*** 51.9 (.34) 54.7 (.69) 2.8*** Math 2000 58.0 (.43) 63.2 (.76) 5.2*** 53.2 (.41) 58.8 (.81) 5.6*** Reading 2002 50.4 (.41) 48.6 (.54) –1.8*** 57.1 (.39) 53.4 (.53) –3.3*** Writing 2002 55.0 (.36) 54.8 (.47) –0.2 54.3 (.37) 51.9 (.49) –2.4*** Math 2002 56.3 (.49) 55.9 (.62) –0.4 55.1 (.47) 51.7 (.61) –3.4*** Notes: These schools have results for all 18 Grade 3 or Grade 6 assessments over the six-year period studied and belong to school boards with at least 15 schools. This is the sample used in the remaining tables. aDiff. is the difference between mean pass rates at disrupted schools and non-disrupted schools. A positive Diff. indicates mean pass rates are higher at disrupted schools and a negative Diff indicates mean pass rates are lower at disrupted schools. *Indicates that the null hypothesis of equal assessment results across schools with and without the labour market disruption would be rejected at a 10 percent level of significance (one sided); **at 5 percent; and ***at 1 percent. Source: Author’s calculations. (2005). To summarize, the Ministry of Education characteristics of a 20 percent sample of households provided three years (from the six years studied) in those areas. Using both the 1996 and 2001 cen- of postal codes of all students at all elementary suses, a weighted average of the characteristics of schools in Ontario. Postal codes place students in adult persons associated with these schools (living very small geographic areas with known average in the same geographic areas as the children who Canadian Public Policy – Analyse de politiques, vol . xxxvii , no . 4 2011 CPPDec2011Inside.indb 485 29/11/11 4:07 PM
486 David R. Johnson attended the school) is constructed. Social variables addresses of the students writing the assessment in include, for example, the percentage of lone parents each year. There is only one meaningful set of social or the percentage of adults with some university and economic characteristics for each school over https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26 education. There are also economic variables, for the entire six-year period.3 There are no data for the example, average income and the unemployment individual groups of students writing the assessment rate of the school’s population. It is quite import- at each school in each year. Thus the school-level ant that these data are from the postal codes of the data are used to represent, with error, the character- actual students who attend a school in order to istics of students writing the assessments. As long represent the characteristics of the adults from the as the measurement error is unrelated to the strike communities in which the students live. However, and work-to-rule occurrences, this does not raise the data are from the addresses of all students at- problems in analyzing the effects of labour relations tending the school in three different years, not the disturbances. Table 4 Summary Information on Social and Economic Characteristics of Schools in Ontario and Labour Relations Disruptions Variable Mean for Schools with Labour Schools without Labour Difference in Means: All Schools Relations Disruptions Relations Disruptions Labour Disruption Minus (SD) Mean (SD) Mean (SD) No Labour Disruptions Single parent (percent) 22.2 23.2 20.8 2.4*** (8.5) (8.9) (8.0) English or French as 91.2 88.7 94.2 –5.5*** home language (percent) (9.6) (11.2) (6.0) Aboriginal status (percent) 1.3 1.2 1.4 –.2*** (1.8) (1.7) (1.9) Recent immigrants – 4.6 5.9 3.0 2.9*** in Canada < 5 years (5.7) (6.7) (3.8) (percent) Percentage of dwellings 64.7 61.8 68.4 –6.6*** that are detached (23.3) (25.1) (20.2) Average household 63,519 64,227 62,659 1708*** income (2000 dollars) (18,886) (21,238) (15,529) Unemployment rate of 6.4 6.9 5.9 1.0*** persons with children (3.4) (3.7) (3.0) (percent) Adults with some 22.7 23.8 21.5 2.3*** university education (11.3) (11.9) (10.4) (percent) Number of observations 3,098 1,698 1,400 NA Note: *Indicates that the null hypothesis of equal assessment results across schools with and without the labour relations disruption would be rejected at a 10 percent level of significance (one sided); **at 5 percent; and ***at 1 percent. Source: Author’s calculations. Canadian Public Policy – Analyse de politiques, vol . xxxvii , no . 4 2011 CPPDec2011Inside.indb 486 29/11/11 4:07 PM
Do Strikes and Work-to-Rule Campaigns Change Elementary School Assessment Results? 487 The left-hand column of Table 4 presents the aver- would be estimated. Pi,S,t is the pass rate at school age characteristics of all schools in the province used S on assessment i in year t. There is, for each ob- in the analysis. Put simply, schools are very different servation at a school in a year, a vector of dummy https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26 in their social and economic composition. This illus- variables Dt where the value of Dt is unity in year t trates why comparing the unconditional difference and zero otherwise. LS,t is a variable measuring if in average pass rates will generate a biased measure there is a labour relations disruption and εi,S,t is a of the impact of labour relations disruptions. The random error term. The coefficient vector δt captures middle columns present average social and economic the variation in pass rates across schools between characteristics for schools with and without labour years on the same assessment i, the variation ob- relations disruptions at any time in the six years served in Tables 2 and 3. The term in equation (1), of data. Schools without labour disruptions have βi XG,S,t , would capture the association of the social fewer lone parents, more French or English speakers, and economic characteristics X of the grade-cohort more Aboriginals, fewer immigrants, more detached with pass rates on assessments of type i. The char- homes, lower household incomes, lower unemploy- acteristics of the grade-cohort writing the different ment rates, and a less educated adult population as assessments are the same in each year at each school measured by the percentage of adults in the school but the relationship between characteristic and result community with some university education. The could vary by type of assessment. The coefficient most important source of these differences is that the vector βi would inform us that students who come two very large boards covering Metropolitan Toronto from families with a higher level of parental edu- are the only two entirely urban boards in the province cation are predicted to perform better on a specific and both Toronto boards experienced labour relations assessment i, say the reading assessment. disruptions. When school pass rates, averaged over three years, are predicted for a cross-section using The term in the equation denoted μi,S is a measure the set of social and economic variables in Table 4, of the average quality of the school over the six about 40 percent of the variation in school pass rates years of assessments. If at one school μi,S is posi- is associated with variation in the social and economic tive and at another school μi,S is negative, then at background of students. When the differences between the two schools with the same social and economic schools in social and economic background are ignored characteristics, results are different. Johnson (2005) in the analysis of disruptions, as they are in Table 3, labels the school with the higher value of μ i,S a then the analysis of the association between disrup- higher-quality school. These school quality effects tions and student achievement is incorrect. A better are not presented in this paper.4 methodology is needed. Finally, the association of the labour relations disruption at the school in a year on assessment of Labour Relations Disruptions and type i is measured by the coefficient λi on a measure the A cademic P erformance of of labour relations disruption at that school in that Elementary Students in Ontario: year LS,t. The value of this coefficient varies by type Methodology of assessment; it may be that strikes do not reduce writing results but reduce mathematics results. If we had data on the social and economic character- istics of the members of each grade (3 or 6) cohort As already stated, the school social and eco- writing an assessment in each year at each school, nomic characteristics, now denoted XS, vary only the panel regression equation by school, that is, the actual data have no time dimension. The data do not describe the specific Pi,S,t = α + δt Dt + βi XG,S,t + μi,S + λi LS,t + εi,S,t (1) characteristics of the students writing the assessment Canadian Public Policy – Analyse de politiques, vol . xxxvii , no . 4 2011 CPPDec2011Inside.indb 487 29/11/11 4:07 PM
488 David R. Johnson at the school in a year, but only the average social This specification tests for a non-linear association and economic characteristics of all students at- between assessment results and the length of the tending a school over the six-year period. Thus in disruption. https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26 the panel setting of equation (1), it is not possible to estimate equation (1) directly. An indirect approach The fourth measure of disruptions is designed is taken to obtain the estimates of λi, the coefficients to ask if the association between disruptions and of interest. achievement varies by social and economic status of the school. Here equation (2) is augmented as The econometric model estimated that represents equation (1) is Pi,S,t = α + δt Dt + γi Si + λi1 LS,t + λi2 LS,t ((EDS–mean(ED))/sd(ED)) Pi,S,t = α + δt Dt + γi Si + λi LS,t + εi,S,t. (2) + εi,S,t. (3) Equation (2) is estimated separately for each assess- The effect of a disruption now varies with the ment in Grade 3 and Grade 6 with standard errors position of the school in the social and economic clustered at the school level. This is a regression gradient. To be very specific, the social and eco- with school fixed effects (Si) and year fixed effects nomic status of the school is measured by the (Dt). The coefficients on the school fixed effects standardized logarithm of the percentage of adults capture both the variation across schools in the so- in the school community who have some university cial and economic background of students and the education, denoted ((EDS–mean(ED))/sd(ED)). This school quality effect; the two cannot be identified variable is approximately normally distributed. For a separately with the available data. The fixed effects, school at the mean value of this variable (zero), the as- both year and school, are allowed to vary by type of sociation of disruption and performance on assessment assessment i. A school could, for example, have an of type i is measured by λi1. For a school two standard exceptional Grade 6 mathematics teacher. When L S,t deviations above (below), the association of disruption is non-zero, there is a labour relations disruption in and performance takes the value λi1+ 2 λi2 (λi1– 2 λi2). that year at that school. Equation (2) estimates the The predictions and standard errors are reported at the effects of labour relations disruptions by compar- mean and one and two standard deviations above and ing results at disrupted schools with results at those below the mean. same schools in years without a disruption. There are various measures of the labour relations disruptions. Labour Relations Disruptions and The first measure of a labour relations disruption the A cademic P erformance of is a simple indicator variable that takes a value of Elementary Students in Ontario: one where there is a disruption in that year. The Results second measure enters the number of days of the disruption in that year as the value of LS,t. The co- Table 5 presents various measures of the association efficient on this variable represents the effects on of labour relations disruptions and outcomes using the pass rate per day of the strike or the work-to- the framework of equation (2). The first row presents rule campaign. If this coefficient were negative, the coefficients for each type of assessment—reading, product of the coefficient and a longer strike would writing, and mathematics—on an indicator variable be larger in absolute value than for a shorter strike. that a school experienced either type of disruption, a A second model is estimated where the number of strike or a work-to-rule, in that year. The coefficients days of the disruption as well as the square of the are all negative. Only the coefficient measuring number of days of the disruption enters the equation. the association of Grade 6 mathematics results and Canadian Public Policy – Analyse de politiques, vol . xxxvii , no . 4 2011 CPPDec2011Inside.indb 488 29/11/11 4:07 PM
https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26 CPPDec2011Inside.indb 489 Table 5 The Association of Labour Relations Disruptions and Pass Rates in the Year of the Disruption by Grade, Type of Assessment, and Measure of Disruption Variable Measuring the Labour Relations Grade 3 Grade 6 Disruption Reading Writing Mathematics Reading Writing Mathematics All labour relations disruptions Indicator variable for disrupted school –0.52 (.35) 0.09 (.33) –0.40 (.41) –.31 (.34) –.27 (.34) –.87 (.41)** Strikes or lockouts Indicator variable for strike –.09 (.55) –.19 (.55) –.04 (.64) .10 (.49) .25 (.52) –1.0 (.65) Number of days of strike –.034 (.048) –.029 (.31) –.030 (.05) .013 (.04) .047 (.04) –.076 (.05) Number of days of strike .211 (.22) .010 (.20) –.136 (.26) .015 (.19) –.055 (.20) –.107 (.26) Number of days of strike squared –.018 (.01) –.002 (.01) .007 (.02) –.0001 (.01) .007 (.01) .002 (.01) Work-to-rule campaigns Indicator variable for work-to-rule –.89 (.42)** –.13 (.38) –.95 (.48)** –.26 (.40) –.21 (.39) –.90 (.47)** Number of days of work-to-rule –.041 (.01)*** .011 (.01) –.034 (.01)** –.027 (.01)* –.011 (.01) –.051 (.01)*** Canadian Public Policy – Analyse de politiques, Number of days of work-to-rule –.024 (.05) –.093 (.04)** –.064 (.057) .061 (.04) –.073 (.04) .095 (.05)* Number of days of work-to-rule squared –.0004 (.001) .003 (.001)*** .0009 (.001) –.002 (.001)* .001 (.001) –.004 (.001)*** Note: This table presents the coefficients on labour relations disruption variables using equation (2). School and year fixed effects are included in the estimating equation. The values in parenthe- ses are standard errors. Do Strikes and Work-to-Rule Campaigns Change Elementary School Assessment Results? *Indicates that the null hypothesis of a zero coefficient is rejected at 10 percent; **at 5 percent; and ***at 1 percent. Source: Author’s calculations. vol . xxxvii , no . 4 2011 489 29/11/11 4:07 PM
490 David R. Johnson labour market disturbances is significantly differ- the reduction in the pass rate with an additional day ent from zero. It is quite a large coefficient, and it of work-to-rule are quite precisely estimated for means that a school experiencing a labour relations the reading and mathematics assessments. In those https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26 disruption, either strike or work-to-rule, has a pass assessments, an additional day of work-to-rule re- rate 0.87 percentage points lower in the year of duces assessment results by about 0.04 percentage the disruption relative to other years at the same points per day. Thus a work-to-rule campaign that school. The other coefficients are not statistically is ten days in length, a fairly ordinary work-to-rule significant. campaign (see Table 1), is predicted to reduce pass rates by 0.4 percentage points. This is considerable The next row presents coefficients on an indica- evidence that work-to-rule campaigns are associated tor variable related to the presence of a strike at with lower assessment results, even for the aggre- the school in that year. None of the coefficients are gate of schools. The last row of the table measures statistically significant. Is this a surprising result? work-to-rule disruptions by the number of days and Recall from Table 1 that several of the strikes are the number of days squared. Some of the coefficients very short and it might be difficult to observe any on the quadratic terms are significant and others are impact of all strikes when strikes of such different not. There may be more complex interactions of the length are treated as equivalent in impact. length of work-to-rule campaigns and assessment results to be further considered. The next rows of results address the issue of strike length in two ways. The first strike measure is The most interesting results of this study are the number of days of the strike and the coefficient found in Table 6, which presents the results of the is the effect on the pass rate per day of a strike. specification found in equation (3). Consider the co- Once again none of the coefficients are statistic- efficient -0.191 in the upper left corner under Grade ally significant, but four of the six coefficients are 3 reading. This coefficient means that a school with negative. The coefficient on Grade 6 mathematics the level of the normalized logarithm of parental is the largest coefficient. The next row presents a education at two standard deviations below the mean pair of coefficients for each assessment, where in will experience a reduction of the pass rate of 0.191 equation (2) there are now two variables. The upper percentage points per day of strike. For a strike of coefficient is on the number of days of the strike ten days in length, this is 1.91 percent of the class and the lower coefficient is on the squared number predicted to fail who would otherwise not fail. The of days of the strike. Again there are no significant average percentage of adults in a school commun- coefficients. Non-linear effects in strike length do ity with some university education is 22 percent in not appear to be important. Table 4. Schools two standard deviations below the mean of the logarithm of the education variable have The lower portion of Table 5 repeats the analy- about 7 percent of adults in the school community sis for work-to-rule campaigns, the more common with some university education, a school two stan- way for schools to experience labour relations dard deviations above the mean of the logarithmic disruptions. There is some evidence that work- variable in Table 6 has about 50 percent of the adult to-rule campaigns reduce assessment results. All population with some university education. The as- coefficients on the indicator variables are negative. sociation of strike days and reduction in results falls Three of the coefficients on the simple indicator dramatically as schools move up the socioeconomic variable are statistically significant, including both gradient. The negative effects of strike days in Grade coefficients on the mathematics results. When the 3 are strongest on the mathematics assessments, analysis is repeated using the number of days of then the reading assessments. The relationship of work-to-rule campaigns, the coefficients measuring strike days to writing results is not statistically Canadian Public Policy – Analyse de politiques, vol . xxxvii , no . 4 2011 CPPDec2011Inside.indb 490 29/11/11 4:07 PM
https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26 Table 6 The Change in the Predicted Pass Rate in the Year of the Disruption by Grade and Type of Assessment at Different Values of the Percentage of Adults Associated with the School with Some University Education CPPDec2011Inside.indb 491 Standard Deviations (approximate Grade 3 Grade 6 percentage with some university education) Reading Writing Mathematics Reading Writing Mathematics Effects per strike day evaluated at different points in distribution of logarithm of education –2 (7%) –.191 (.105)* –.070 (.096) –.285 (.124)** –.097 (.109) –.091 (.120) –.410 ( .147)*** –1 (12%) –.116 (.071)* –.051 (.064) –.164 (.083)** –.047 (.071) –.029 (.078) –.261 (.097)*** 0 (20%) –.041 (.049) –.031 (.044) –.043 (.056) .001 (.044) .032 (.047) –.112 (.060)* +1 (32%) .032 (.056) –.011 (.055) .077 (.063) .051 (.051) .094 (.051)* .037 (.067) +2 (52%) .107 (.086) –.008 (.085) .198 (.097)** .101 (.083) .156 (.086)* .186 (.110)* Effects per work–to–rule day evaluated at different points in distribution of logarithm of education –2 (7%) –.072 (.036)** –.038 (.032) –.097 (.041)** .013 (.038) –.003 (.038) –.033 (.043) –1 (12%) –.059 (.024)** –.017 (.022) –.071 (.029)** –.003 (.026) –.006 (.025) –.040 (.030) 0 (20%) –.046 (.016)*** .002 (.014) –.045 (.019)** –.020 (.017) –.010 (.015) –.048 (.019)* +1 (32%) –.033 (.016)** .023 (.014)* –.019 (.017) –.037 (.017)** –.013 (.017) –.056 (.017)*** Canadian Public Policy – Analyse de politiques, +2 (52%) –.020 (.024) .043 (.021)** .006 (.025) –.054 (.027)** –.016 (.028) –.064 (.026)** Note: The values in parentheses are standard errors. Education level at the school is measured by the normalized logarithm of the percentage of households with some university education for students attending that school. School and year fixed effects are included in the estimating equation. Do Strikes and Work-to-Rule Campaigns Change Elementary School Assessment Results? *Indicates that the null hypothesis of a zero coefficient is rejected at 10 percent; **at 5 percent; and ***at 1 percent. Source: Author’s calculations. vol . xxxvii , no . 4 2011 491 29/11/11 4:07 PM
492 David R. Johnson significant. In Grade 6 the relationship of strike campaign. The reasons for this reversal of pattern days to mathematics pass rates is the only significant in Grade 6 remain to be understood. coefficient. But the effect is very strong. A school https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26 two standard deviations below the mean of the normalized education variable is predicted to have Conclusions a 4.1 percentage point reduction in the mathemat- ics pass rate after a ten-day strike. Coefficients two The paper measures the differences in assessment standard deviations below the mean, one standard results between schools in Ontario that have experi- deviation below the mean, and at the mean are all enced labour relations disruptions (strikes, lockouts, statistically significant and large in absolute value. and work-to-rule campaigns) and schools that have Strikes reduce Grade 6 mathematics results for the not experienced these disruptions. majority of Grade 6 students. The results show that both strikes and work- The lower part of Table 6 uses equation (3) to to-rule campaigns are associated with reduced investigate the association of assessment results assessment results in Grade 3 reading and math- and work-to-rule campaigns across the social and ematics. The predicted reduction in assessment economic gradient. There are three noteworthy results is larger per day of strike than per day of results. Where both the coefficient on the number work-to-rule. In Grade 3, the largest reductions in results are found at schools where children arrive of days of strikes and the coefficient on the num- with other disadvantages as measured by the level ber of days of work-to-rule are negative, then the of parental education. coefficient on the strike days is larger in absolute value. Strikes do have a larger impact than work-to- The picture in Grade 6 is more complicated. rule campaigns. There is evidence in Grade 3 that Additional strike days at disadvantaged schools work-to-rule campaigns are associated with lower are associated with reduced results in mathematics reading and lower mathematics results, and these in Grade 6; the effect is very large and extends to effects are quite large at schools where parental the midpoint of the social and economic gradient of education is low. Even at a school with the mean schools. Additional days of a work-to-rule campaign value of the gradient variable, an additional day in Grade 6 are associated with lower assessment of a work-to-rule campaign is associated with an results in reading and mathematics at the schools estimated 0.046 percentage point reduction in the where parental education is highest, not lowest. Grade 3 reading pass rate and a 0.045 percentage These effects are smaller. point reduction in the Grade 3 mathematics pass rate per day of work-to-rule. The coefficients are There is further work to be done. Using student- quite precisely estimated, perhaps because there level data would increase our understanding of what were numerous work-to-rule campaigns of widely students are most hurt by strikes and work-to-rule varying lengths. The third observation from Table 6 campaigns. Student-level data would help us to is that the association of work-to-rule campaigns and understand whether the reductions in academic results for Grade 6 is counterintuitive. There is no performance are temporary or permanent if stu- association between results and the number of days dents were tracked over time. In the period studied, of a work-to-rule campaign in Grade 6 at the schools there are no individual student data. Finally, more in the lower part of the education gradient. However, research needs to be done to understand the mech- at schools one standard deviation and two standard anism through which strikes are associated with deviations above the mean of the education gradient, the larger reduction in Grade 3 and 6 pass rates at assessment results fall when there is a work-to-rule disadvantaged schools while in work-to-rule cam- Canadian Public Policy – Analyse de politiques, vol . xxxvii , no . 4 2011 CPPDec2011Inside.indb 492 29/11/11 4:07 PM
Do Strikes and Work-to-Rule Campaigns Change Elementary School Assessment Results? 493 4 The paigns results fall more at disadvantaged schools in residual εS in Pi,S = α + βi XS + εS , a modified Grade 3 and more at advantaged schools in Grade 6. version of equation (1), provides an estimate of school quality. The equation is estimated over a cross-section of https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26 schools with test results averaged over a number of years. A positive residual is interpreted as a better quality school, Notes that is, a school where for similar social and economic Useful comments on this paper were received at the Target characteristics, assessment results are better. Conference at the University of British Columbia, from 5 The “school quality” terminology implicitly assumes the C.D. Howe Institute, from my colleagues at Laurier that resources are similar across schools, a reasonable and from the referees of this journal. assumption for schools in Ontario where teacher quali- 1 The period of this study followed a decade of con- fications are provincewide and base funding to boards siderable change in the Ontario education system. Even is also provincewide. Hence the same average resources before the period studied, there were strikes and work- are available to each student at a board. There is no fur- to-rule campaigns. In the years from 1992/93 to 1997/98, ther information about resource allocation to individual there were three stoppages and nine work-to-rule cam- schools in Ontario in this period. There are provincially paigns at elementary schools as well as seven stoppages imposed rules on the length of the instructional day, the and 29 work-to-rule campaigns at secondary schools. number of instructional days and, for part of the period, Neither stoppages nor strikes were new to the Ontario on class sizes. In the studies of Ohio and Pennsylvania education system. Campoleti, Hebdon, and Hyatt (2005) cited above, districts did vary in overall resources avail- report some evidence on the incidence of teacher work able in a meaningful way. stoppages relative to other unions in Ontario. The effect of labour relations disruptions on Ontario students cannot be measured prior to 1998/99 because there are no data References on student results at either the individual or school level. Baker, M. 2011. “Industrial Actions in Schools: Strikes 2 Thesedata are explored in detail in Johnson (2005). and Student Achievement.” Working Paper 16846. They cover roughly 3,500 of 4,000 elementary schools in National Bureau of Economic Research, New York. Ontario. During the period studied, Ontario students did Campolieti, M., R. Hebdon, and D. Hyatt. 2005. “Strike not have tracking numbers that allow study of individual Incidence and Strike Duration: Some New Evidence student data. from Ontario.” Industrial and Labor Relations Review 58(4):610-30. 3The postal codes do not identify the grade of the stu- Carini, R.M. 2008. “New Directions for the Study of Col- dent at the school. The information is simply the number lective Bargaining in Schools.” Journal of Collective of students at that school who live at a given postal code in Negotiations 32(4):317-28. that academic year. The social and economic characteristics Johnson, D.R. 2005. Signposts of Success: Interpreting are calculated for the average student at the school, not the Ontario’s Elementary School Test Scores. Policy Study students actually participating in the assessment. When this No. 40. Toronto, ON: C.D. Howe Institute. average is calculated for three of the six years studied, using Rose, J.B. 2002. “The Assault on School Teacher Bar- the census closest to the year of enrolment data, the average gaining in Ontario.” Relations Industrielles/Industrial characteristics of a school change by only very small amounts. Relations 57(1):100-128. A very small amount of demographic data is collected directly Thornicroft, K.W. 1994. “Teachers Strikes and Student from and about the students who write the assessment. These Achievement: Evidence from Ohio.” Journal of Col- variables, available only since 2000/01, are three slightly lective Negotiations 23(1):27-40. different language variables and the percentage of students Wright, D. 2004. “Voice, Accountability and Dialogue: receiving special education support in that grade. Johnson Recommendations for an Improved Collective Bar- (2005) shows that this set of variables is not very useful in gaining System for Teacher Contracts in British equations like (1), and the data based on postal codes and Columbia.” Commission to Review Teacher Collective the census are more useful. Bargaining, Don Wright, Commissioner, December. Canadian Public Policy – Analyse de politiques, vol . xxxvii , no . 4 2011 CPPDec2011Inside.indb 493 29/11/11 4:07 PM
494 David R. Johnson Zirkel, P. 1992. “The Academic Effects of Teacher Zwerling, H.L. 2008. “Pennsylvania Teachers’ Strikes Strikes.” Journal of Collective Negotiations in the and Academic Performance.” Journal of Collective Public Sector 21(2):123-38. Negotiations 32(2):151-72. https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26 Canadian Public Policy – Analyse de politiques, vol . xxxvii , no . 4 2011 CPPDec2011Inside.indb 494 29/11/11 4:07 PM
You can also read