Do Strikes and Work-to-Rule Campaigns Change Elementary School Assessment Results?

Page created by Phillip Pratt
 
CONTINUE READING
Do Strikes and Work-to-Rule
                                                                                                                                                                                          Do Strikes and Work-to-Rule Campaigns Change Elementary School Assessment Results?                  479

                                                                                                                                                                               Campaigns Change Elementary
https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26

                                                                                                                                                                               School Assessment Results?
                                                                                                                                                                               David R. Johnson
                                                                                                                                                                               School of Business and Economics
                                                                                                                                                                               Wilfrid Laurier University
                                                                                                                                                                               Waterloo, Ontario

                                                                                                                                                                               Dans cet article, à partir de données portant sur l’évaluation des élèves ontariens de la 3e année à la 6e ­année,
                                                                                                                                                                               j’analyse le lien qu’il est possible d’établir entre le rendement scolaire et les grèves et grèves du zèle des
                                                                                                                                                                               enseignants. De 1998-1999 à 2003-2004, environ 5 % des écoles ontariennes ont été touchées par une grève
                                                                                                                                                                               (et un lock-out), et environ 10 % par une grève du zèle (c’est-à-dire le refus des enseignants de participer
                                                                                                                                                                               à certaines activités scolaires auxquelles ils collaborent normalement). Les résultats indiquent qu’il y a eu
                                                                                                                                                                               en 3e année et en 6e année dans les écoles de milieux défavorisés une baisse marquée des résultats scolaires
                                                                                                                                                                               pendant les grèves, ainsi qu’une baisse, mais moins importante, pendant les grèves du zèle. Les baisses
                                                                                                                                                                               observées pendant les grèves du zèle varient selon les écoles : les plus élevées concernent les élèves de 3e
                                                                                                                                                                               année des écoles de milieux défavorisés, et les moins élevées les élèves de 6e année des écoles de milieux
                                                                                                                                                                               favorisés.

                                                                                                                                                                               Mots clés : grève des enseignants, rendement scolaire, grève du zèle

                                                                                                                                                                               This paper uses data from the Grade 3 and 6 assessments conducted in Ontario schools to examine the as-
                                                                                                                                                                               sociation between student achievement and both strikes and work-to-rule campaigns by teachers. Between
                                                                                                                                                                               1998/99 and 2003/04 approximately 5 percent of schools in Ontario were exposed to strikes (including
                                                                                                                                                                               one lockout) and over 10 percent of schools experienced a work-to-rule campaign (the withdrawal of
                                                                                                                                                                               many school activities normally carried out by teachers). This study finds large reductions in academic
                                                                                                                                                                               achievement associated with strikes at disadvantaged schools in both Grade 3 and Grade 6, and substantial
                                                                                                                                                                               reductions in results associated with work-to-rule campaigns. The reductions associated with work-to-rule
                                                                                                                                                                               campaigns vary across schools with the largest reductions at disadvantaged schools in Grade 3 and at ad-
                                                                                                                                                                               vantaged schools in Grade 6.

                                                                                                                                                                               Keywords: strikes by teachers, student achievement, work-to-rule campaigns by teachers

                                                                                                                                                                               Introduction                                               unionized. Yet the legal ability of teachers and
                                                                                                                                                                                                                                          boards to strike varies. In Canada, teachers in

                                                                                                                                                                               T   eachers in Ontario, as in all Canadian
                                                                                                                                                                                   jurisdictions, in many American jurisdictions,
                                                                                                                                                                               and throughout much of the developed world are
                                                                                                                                                                                                                                          Manitoba and Prince Edward Island do not have
                                                                                                                                                                                                                                          the right to strike while in Nova Scotia and Quebec
                                                                                                                                                                                                                                          the right to strike is restricted (Wright 2004). In

                                                                                                                                                                                                                  Canadian Public Policy – Analyse de politiques,         vol . xxxvii , no . 4 2011

                                                                                                                                                                    CPPDec2011Inside.indb 479                                                                                                     29/11/11 4:07 PM
480 David R. Johnson

                                                                                                                                                                               the United States, only ten states allow teachers to              this paper finds that work-to-rule campaigns have
                                                                                                                                                                               strike (Wright 2004). This paper compares student                 strong, negative effects on student achievement.
                                                                                                                                                                               assessment results at elementary schools that have
https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26

                                                                                                                                                                               experienced strikes or work-to-rule campaigns                        The analysis in this paper also shows that, in all
                                                                                                                                                                               (collectively called labour relations disruptions) to             cases, the length of the labour relations disruption
                                                                                                                                                                               assessment results at the same elementary schools                 matters: the longer the disruption, the larger the
                                                                                                                                                                               in other years without a disruption. A work-to-rule               predicted reduction in achievement. For strikes, the
                                                                                                                                                                               campaign is a common strategy where teachers come                 lost instructional days create a learning gap between
                                                                                                                                                                               to school but do not work beyond the limited duties               schools that experienced stoppages and schools
                                                                                                                                                                               specified in their contracts. Typical duties withdrawn            that did not experience stoppages. The quantity
                                                                                                                                                                               include running extracurricular activities, meeting               of instructional time matters. In a similar way, a
                                                                                                                                                                               with parents, or attending administrative meetings.               work-to-rule campaign may affect the quantity of
                                                                                                                                                                               In the period of analysis there were 15 strikes                   instructional time if the normal instructional time
                                                                                                                                                                               (including one lockout treated here as a strike) and              goes beyond the contract instructional time. A work-
                                                                                                                                                                               20 work-to-rule campaigns in Ontario. The key                     to-rule may also affect the quality of instructional
                                                                                                                                                                               question: Does academic performance, as measured                  time. Assessment results are lower after a work-to-
                                                                                                                                                                               by student assessment results, fall in the year of                rule campaign in both Grade 3 and Grade 6, which
                                                                                                                                                                               these labour relations disruptions?                               suggests that teachers cannot teach at least some
                                                                                                                                                                                                                                                 groups of students as effectively within the contract
                                                                                                                                                                                   The average observed change in the assessment                 rules enforced by a work-to-rule campaign.
                                                                                                                                                                               results associated with disruptions across all schools
                                                                                                                                                                               is negative but often small and not statistically signifi-            There is a large American literature, reviewed in
                                                                                                                                                                               cant. The largest reductions in results are observed in           Carini (2008), attempting to understand if there is a
                                                                                                                                                                               the mathematics assessment. There is a large reduction            relationship between school outcomes and unioniza-
                                                                                                                                                                               in reading and mathematics results associated with                tion at the level of state, district, or student. A small
                                                                                                                                                                               strikes in both Grade 3 and Grade 6 for schools drawn             portion of the American literature on the effect of
                                                                                                                                                                               from the lowest parts of the distribution of parental             unionization includes various measurements asking
                                                                                                                                                                               education. Strikes reduce learning outcomes by the                if strikes are associated with lower academic out-
                                                                                                                                                                               largest amount at precisely the schools where children            comes. In an early review of the literature relating
                                                                                                                                                                               arrive with other disadvantages.                                  strikes to achievement, Zirkel (1992) concluded that
                                                                                                                                                                                                                                                 there was little evidence that teacher strikes affected
                                                                                                                                                                                   A second important result is that work-to-rule                attitude, attendance, or achievement of public school
                                                                                                                                                                               campaigns reduce learning outcomes on average.                    students. Since the Zirkel survey, two American
                                                                                                                                                                               The reductions are larger at disadvantaged schools in             studies and one Canadian study have been conducted
                                                                                                                                                                               Grade 3 and at advantaged schools in Grade 6. This                that are similar to the study reported here.
                                                                                                                                                                               is a controversial result. Work-to-rule campaigns,
                                                                                                                                                                               which are a common feature of labour relations dis-                  Thornicroft (1994) uses achievement results on
                                                                                                                                                                               ruptions in Ontario, might be considered to have no               a battery of standardized tests administered only in
                                                                                                                                                                               detrimental effects on student outcomes. Patterson,               the 1989/90 academic year in 612 Ohio districts.
                                                                                                                                                                               a union official, is quoted with respect to a work-               Between 1984/85 and 1991/92, there were 61 strikes
                                                                                                                                                                               to-rule: “That is the action we see as appropriate, as            averaging ten days in duration in these districts. A
                                                                                                                                                                               it brings pressure on the board without having any                comparison of average achievement results between
                                                                                                                                                                               impact on the classrooms or the students” (Kingston               strike and no-strike districts found few significant
                                                                                                                                                                               Whig-Standard, 10 October 2002). The evidence in                  differences. If there were any differences, districts

                                                                                                                                                                               Canadian Public Policy – Analyse de politiques,              vol . xxxvii , no . 4 2011

                                                                                                                                                                    CPPDec2011Inside.indb 480                                                                                                           29/11/11 4:07 PM
Do Strikes and Work-to-Rule Campaigns Change Elementary School Assessment Results?                   481

                                                                                                                                                                               with strikes had better results. A regression model             Baker (2011) studies strikes in the Ontario school
                                                                                                                                                                               of achievement that included a variety of variables          system over the same period and with much of the
                                                                                                                                                                               related to strikes found no significant relationships        same data that I use. Baker’s primary strategy is to
https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26

                                                                                                                                                                               between achievement and the strike measures. The             compare, across schools, the growth in test scores
                                                                                                                                                                               strike measures included an indicator for any strike         between a Grade 3 and a Grade 6 “cohort” at schools
                                                                                                                                                                               in the entire six-year period, the number of strike          with and without strikes. The cohorts are not exact
                                                                                                                                                                               days over the six-year period, a strike in the assess-       because, as in my study, the data are at the school
                                                                                                                                                                               ment year, number of strike days in the assessment           level and students enter and leave schools. Baker
                                                                                                                                                                               year, a strike indicator for each year in the data, and      presents a large number of estimates that examine
                                                                                                                                                                               the number of days on strike for each year in the            the sensitivity of his results to different choices
                                                                                                                                                                               data. There were a series of control variables in the        for the group of schools without strikes, which
                                                                                                                                                                               regressions. The most important variables measure            function as the control group. Baker finds that the
                                                                                                                                                                               social and economic differences across districts in          largest effects are found in a reduction of Grade 6
                                                                                                                                                                               income although there are some measures of school            mathematics test score growth with a long strike,
                                                                                                                                                                               spending and teacher quality. About half the varia-          defined as a strike of ten days or more that occurred
                                                                                                                                                                               tion in achievement across districts is associated           in Grade 5 or Grade 6 of the Grade 6 class assessed.
                                                                                                                                                                               with variation in the control variables. Thornicroft         I also find the largest effects of current-year strikes
                                                                                                                                                                               (1994) concludes strikes do not affect achievement.          on mathematics results in Grade 6. Baker did not
                                                                                                                                                                                                                                            find that a long-ago strike in Grade 2 or 3 had any
                                                                                                                                                                                  In a more recent study, Zwerling (2008) compares          significant impact on the growth of test scores be-
                                                                                                                                                                               outcomes on a series of annual standardized tests            tween Grade 3 and Grade 6 for the cohort exposed
                                                                                                                                                                               in many grades across districts in Pennsylvania              to that strike.
                                                                                                                                                                               with and without strikes. There are three measures
                                                                                                                                                                               that describe students: the percentage who receive              The paper proceeds as follows. The next section
                                                                                                                                                                               a free or reduced-price lunch, the percentage who            outlines the labour relations system in place in On-
                                                                                                                                                                               are black or Hispanic, and the percentage who have           tario between 1998/99 and 2003/04 and presents the
                                                                                                                                                                               individual education plans (both disabled and gift-          data on strikes and work-to-rule activity. The third
                                                                                                                                                                               ed). Other other control variables describe district         section presents the data on achievement, some
                                                                                                                                                                               characteristics relating to teachers and expenditures.       preliminary data describing outcomes, and the social
                                                                                                                                                                               There are two measures of strike activity: a simple          and economic characteristics of schools with and
                                                                                                                                                                               indicator variable if there is a strike in the district in   without disruptions. The fourth section describes
                                                                                                                                                                               the test year or in the previous year, and a variable        the methodology used to analyze the relationships
                                                                                                                                                                               measuring the number of days of the strike. Neither          between disruptions and outcomes more carefully.
                                                                                                                                                                               of the two strike variables has a clear association          The fifth section presents the results, followed by
                                                                                                                                                                               with achievement outcomes. The two strike vari-              conclusions.
                                                                                                                                                                               ables are also interacted with the three variables
                                                                                                                                                                               describing students. Coefficients on the interaction
                                                                                                                                                                               terms are not statistically significant. Zwerling            Education Labour Relations Disruptions
                                                                                                                                                                               (2008) concludes there is no evidence that strikes           in O ntario 1998/99 to 2003/04
                                                                                                                                                                               affect achievement. In his survey of the interaction
                                                                                                                                                                               of education and collective bargaining, Carini               In the period studied, 75 individual school boards
                                                                                                                                                                               (2008) suggests that the effects of strikes should           negotiated separate two-year contracts with locals
                                                                                                                                                                               be investigated more carefully at a lower level of           of the provincewide union of teachers. The boards
                                                                                                                                                                               disaggregation, ideally with student-level data.             received roughly equal funding each year per student

                                                                                                                                                                                                                     Canadian Public Policy – Analyse de politiques,       vol . xxxvii , no . 4 2011

                                                                                                                                                                    CPPDec2011Inside.indb 481                                                                                                      29/11/11 4:07 PM
482 David R. Johnson

                                                                                                                                                                               from the provincial government (with small adjust-                        alternate years of the data there are no disruptions
                                                                                                                                                                               ments for items like second-language instruction                          because contracts run for two years. For example,
                                                                                                                                                                               and rural transportation). Boards had no access                           in the first year of the data, contracts ran from 1
https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26

                                                                                                                                                                               to a local tax base to pay increased wages. Thus a                        September 1998 to 31 August 2000 at all boards.
                                                                                                                                                                               “successful” (from the union point of view) labour                        The two-year cycle is repeated over the remaining
                                                                                                                                                                               relations disruption could not increase total financial                   four years of the data (Rose 2002).
                                                                                                                                                                               resources at a board.

                                                                                                                                                                                  In spite of these constraints, elementary teach-                       Labour Relations Disruptions and
                                                                                                                                                                               ers at 14 boards went on strike. In addition, there                       the A cademic P erformance of
                                                                                                                                                                               was one lockout over the six years studied (which I                       Elementary Students in Ontario:
                                                                                                                                                                               include in the analysis and label as a strike).1 Table                    Preliminaries
                                                                                                                                                                               1 presents the basic facts on the 15 strikes and 20
                                                                                                                                                                               work-to-rule campaigns in 1998/99, 2000/01, and                           The Measurement of Academic Performance
                                                                                                                                                                               2002/03. Two boards experienced both a strike and                         The provincewide data on student learning out-
                                                                                                                                                                               a work-to-rule campaign. In the first year of the data                    comes, by school for all schools with 15 or more
                                                                                                                                                                               almost 500 schools (about 15 percent) had a strike.                       students in the relevant grade, are drawn from the
                                                                                                                                                                               There were fewer strikes in 2000/01 and even fewer                        Grade 3 and Grade 6 Education Quality Account-
                                                                                                                                                                               in 2002/03. Strikes ranged in length from 1 day to 16                     ability Office (EQAO) assessments.2 Assessments
                                                                                                                                                                               days of school. Work-to-rule campaigns were much                          were first written in 1996/97, and results for most
                                                                                                                                                                               longer in duration. In 2002/03, almost 40 percent of                      schools in the province are available beginning in
                                                                                                                                                                               schools experienced a work-to-rule campaign. In the                       1998/99. The usual measure of school success is

                                                                                                                                                                               Table 1
                                                                                                                                                                               The Incidence of Labour Relations Disruptions in Ontario Elementary Schools: 1998–2003

                                                                                                                                                                               Academic Year and Type       Number        Number       Numbers of Instructional       Percentage        Average    Standard Deviation
                                                                                                                                                                               of Labour Relations         of Boards     of Schools   Days Affected by Disruptions     of Schools      Length of      of Disruption
                                                                                                                                                                               Disturbance                 with This      Affected       (shortest to longest)        in Province   Disruption over Length over All
                                                                                                                                                                                                           Disruption                                                   Affected      All Schools        Schools

                                                                                                                                                                               1998/99 Strikes                 8            484       1, 1, 5, 8, 9, 11, 11, 15          14.9            1.1              3.2
                                                                                                                                                                               2000/01 Strikes                 4            205       3, 10, 14, 16                       6.8            0.8              3.3
                                                                                                                                                                               2000/01 Work-to-rule            6            828       6, 18, 22, 23, 25, 27              23.5            5.4            10.1
                                                                                                                                                                               2002/03 Strikes                 3            247       5, 11, 12                           8.5            0.7              2.8
                                                                                                                                                                               2002/03 Work-to-rule           14          1,316       7, 9, 11, 13, 18, 19, 21, 24,      38.4            9.7            14.6
                                                                                                                                                                                                                                      28, 29, 29, 30, 31, 41

                                                                                                                                                                               Notes: There were no labour relations disruptions in academic years 1999/00, 2001/02, and 2003/04. Contracts were two years in length,
                                                                                                                                                                               and negotiations and thus disturbances took place every other academic year. There were no work-to-rule campaigns associated with
                                                                                                                                                                               negotiations during the 1998/99 academic year.
                                                                                                                                                                               Source: Author’s compilation from data provided by the Ontario Education Relations Commission.

                                                                                                                                                                               Canadian Public Policy – Analyse de politiques,                    vol . xxxvii , no . 4 2011

                                                                                                                                                                    CPPDec2011Inside.indb 482                                                                                                                       29/11/11 4:07 PM
Do Strikes and Work-to-Rule Campaigns Change Elementary School Assessment Results?                  483

                                                                                                                                                                               the percentage of all students who obtain a Level 3            Table 3 (and later Table 4) make it clear that
                                                                                                                                                                               (meets expectations) or a Level 4 (exceeds expecta-         while it is important to take into account the varia-
                                                                                                                                                                               tions) result. For convenience this is called the pass      tion by year and type of test in trying to explain the
https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26

                                                                                                                                                                               rate. There is a reading assessment, a writing assess-      association between labour relations disruptions
                                                                                                                                                                               ment, and a mathematics assessment. At a school             and achievement, it is not enough. The top panel
                                                                                                                                                                               with both Grade 3 and Grade 6 classes in all six            of Table 3 considers groups of schools with and
                                                                                                                                                                               years of the study period, there are 36 assessment          without either type of labour relations disruption
                                                                                                                                                                               results, 18 in each grade.                                  in the year of those assessments, the middle panel
                                                                                                                                                                                                                                           compares schools with and without strikes, and the
                                                                                                                                                                                  Table 2 presents distributional statistics for pass      lower panel compares schools with and without
                                                                                                                                                                               rates, by year and by type of assessment, in Grade          work-to-rule campaigns. The table presents uncon-
                                                                                                                                                                               3 and Grade 6. The key aspect is that the various           ditional differences in average pass rates for groups
                                                                                                                                                                               measures of distributional spread (more specific-           of schools on those assessments in a specific year.
                                                                                                                                                                               ally, the standard deviation and interquartile range)
                                                                                                                                                                               remain very similar within a test across years while            This table reinforces ideas already presented.
                                                                                                                                                                               the average pass rate is clearly different in different     Large differences within a year in average pass
                                                                                                                                                                               years on the same assessment. Because the assess-           rates on different assessments are clear. Large dif-
                                                                                                                                                                               ments are designed to be comparable across years,           ferences between average pass rates on the same
                                                                                                                                                                               four (at least) non-mutually exclusive interpretations      assessment in different years are clear. The most
                                                                                                                                                                               of the changes over time in the average pass rate on        interesting aspect of Table 3 is that if we simply
                                                                                                                                                                               a specific assessment are consistent with the data in       compared the pass rate within a year and assess-
                                                                                                                                                                               Table 2. Ontario elementary students could be get-          ment type at schools with and without different
                                                                                                                                                                               ting better results over time with this gain attributed     labour relations disruptions using a test of equality
                                                                                                                                                                               to improved teaching and/or a better curriculum.            of the mean pass rate for the two groups, we might
                                                                                                                                                                               Ontario teachers could be learning, over time, to           conclude disruptions raise results significantly (as
                                                                                                                                                                               deliver the curriculum that is being assessed rather        they do in 27 cases), lower results significantly (as
                                                                                                                                                                               than some other curriculum previously taught. The           they do in 15 cases), or leave results unchanged (as
                                                                                                                                                                               third possibility is that the assessments could vary        they do in 6 cases). How can this be? The answer
                                                                                                                                                                               in difficulty across years. Finally, it is possible that,   is straightforward. Labour relations disruptions af-
                                                                                                                                                                               even if the assessments are equal in difficulty, the        fect all schools within a board. Boards and schools
                                                                                                                                                                               same answers on an assessment of equal difficulty           within boards have different social and economic
                                                                                                                                                                               are graded differently. Either of the last two reasons      characteristics. These differences must be taken
                                                                                                                                                                               could explain the declines in average provincial            into account to properly measure the association of
                                                                                                                                                                               results between pairs of adjacent years on the same         labour relations disruptions and pass rates.
                                                                                                                                                                               assessment. I find such large changes between adja-
                                                                                                                                                                               cent years in the quality of either all the teachers or     The Social and Economic Background of
                                                                                                                                                                               all the students in the cohorts assessed implausible        Elementary Students
                                                                                                                                                                               and conclude it seems very likely that the assessment       Table 4 presents summary statistics of the social and
                                                                                                                                                                               instruments, in spite of all the effort put into compar-    economic characteristics of students at all schools
                                                                                                                                                                               ability, are not always comparable. For the purposes        in Ontario, and then uses as groups the schools that
                                                                                                                                                                               of this paper, Table 2 shows that an accurate estimate      did and not experience labour relations disruptions
                                                                                                                                                                               of the effects of labour relations breakdowns must          at any point in the six years studied. A full explana-
                                                                                                                                                                               take into account the variation in average assessment       tion of the methods used to measure these social
                                                                                                                                                                               results by year and by type of assessment.                  and economic characteristics is found in Johnson

                                                                                                                                                                                                                    Canadian Public Policy – Analyse de politiques,       vol . xxxvii , no . 4 2011

                                                                                                                                                                    CPPDec2011Inside.indb 483                                                                                                     29/11/11 4:07 PM
484 David R. Johnson

                                                                                                                                                                               Table 2
                                                                                                                                                                               Descriptive Statistics for Pass Rates on EQAO Assessments 1998/99 to 2002/03
https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26

                                                                                                                                                                               Year                    Mean         Standard Deviation   Median        25th Percentile   75th Percentile   Interquartile Range

                                                                                                                                                                               Grade 3 reading assessment
                                                                                                                                                                               1998/99                43.7                 17.0           43                 32                56                  24
                                                                                                                                                                               1999/00                48.0                 18.0           48                 35                61                  26
                                                                                                                                                                               2000/01                47.7                 18.0           48                 36                60                  24
                                                                                                                                                                               2001/02                49.3                 19.0           49                 36                62                  26
                                                                                                                                                                               2002/03                49.5                 17.8           50                 37                62                  25
                                                                                                                                                                               2003/04                53.3                 18.0           54                 41                66                  25
                                                                                                                                                                                Grade 3 writing assessment
                                                                                                                                                                                1998/99                50.8                16.2            52                40                62                  22
                                                                                                                                                                                1999/00                50.7                16.9            51                39                62                  23
                                                                                                                                                                                2000/01                50.2                17.3            51                39                62                  23
                                                                                                                                                                                2001/02                54.0                16.7            54                43                66                  23
                                                                                                                                                                                2002/03                54.6                15.6            55                44                66                  22
                                                                                                                                                                                2003/04                57.4                15.7            58                47                69                  22
                                                                                                                                                                                Grade 3 mathematics assessment
                                                                                                                                                                                1998/99               54.2                 20.2            55                40                69                  29
                                                                                                                                                                                1999/00               55.8                 21.3            56                40                72                  32
                                                                                                                                                                                2000/01               58.4                 21.4            60                44                75                  31
                                                                                                                                                                                2001/02               56.8                 21.4            58                42                72                  30
                                                                                                                                                                                2002/03               55.8                 21.0            56                41                72                  31
                                                                                                                                                                                2003/04               63.3                 19.6            65                50                78                  28
                                                                                                                                                                                Grade 6 reading assessment
                                                                                                                                                                                1998/99                48.5                17.1            48                37                60                  23
                                                                                                                                                                                1999/00                50.6                17.9            50                38                63                  25
                                                                                                                                                                                2000/01                53.7                17.7            54                42                66                  24
                                                                                                                                                                                2001/02                54.3                17.6            55                43                67                  24
                                                                                                                                                                                2002/03                55.4                16.9            56                44                67                  23
                                                                                                                                                                                2003/04                57.7                17.0            58                46                70                  24
                                                                                                                                                                                Grade 6 writing assessment
                                                                                                                                                                                1998/99                47.7                16.5            48                37                59                  22
                                                                                                                                                                                1999/00                48.0                17.2            48                36                59                  23
                                                                                                                                                                                2000/01                51.8                16.9            52                41                63                  22
                                                                                                                                                                                2001/02                52.5                16.4            53                42                64                  22
                                                                                                                                                                                2002/03                53.1                15.8            53                43                65                  22
                                                                                                                                                                                2003/04                53.9                16.1            54                43                65                  22
                                                                                                                                                                                Grade 6 mathematics assessment
                                                                                                                                                                                1998/99               46.4                 19.6            46                32                59                  22
                                                                                                                                                                                1999/00               51.5                 19.9            52                37                59                  23
                                                                                                                                                                                2000/01               53.9                 20.0            54                40                63                  22
                                                                                                                                                                                2001/02               53.2                 20.1            54                40                64                  22
                                                                                                                                                                                2002/03               53.7                 19.9            54                40                65                  22
                                                                                                                                                                                2003-04               57.9                 19.8            59                44                65                  22

                                                                                                                                                                                Notes: EQAO = Education Quality Accountability Office.
                                                                                                                                                                                Source: Author’s calculations from EQAO results.

                                                                                                                                                                               Canadian Public Policy – Analyse de politiques,              vol . xxxvii , no . 4 2011

                                                                                                                                                                    CPPDec2011Inside.indb 484                                                                                                                29/11/11 4:07 PM
Do Strikes and Work-to-Rule Campaigns Change Elementary School Assessment Results?                                     485

                                                                                                                                                                               Table 3
                                                                                                                                                                               Comparing Average Pass Rates (percentage of students at Level 3 or Level 4) at Schools with and without Labour Relations
https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26

                                                                                                                                                                               Disruptions within a Year of an Assessment

                                                                                                                                                                               Test               Year                           Grade 3                                                 Grade 6

                                                                                                                                                                                                             No Disruption       Disruption         Diff.a           No Disruption        Disruption          Diff.a

                                                                                                                                                                               A labour relations disruption of any type
                                                                                                                                                                               Reading         1998            44.3 (.33)         47.3 (.72)       3.3***              49.0 (.33)         50.5 (.76)         1.5**
                                                                                                                                                                               Writing         1998            51.5 (.31)         54.4 (.70)       3.3***              48.1 (.32)         49.7 (.77)         1.6**
                                                                                                                                                                               Math            1998            55.2 (.38)         58.8 (.83)       3.5***              46.8 (.39)         48.3 (.90)         1.5*
                                                                                                                                                                               Reading         2000            48.0 (.36)         49.0 (.65)       1.0*                53.7 (.37)         56.7 (.67)         3.0***
                                                                                                                                                                               Writing         2000            50.4 (.35)         52.8 (.62)       2.4***              51.9 (.35)         54.2 (.65)         2.3***
                                                                                                                                                                               Math            2000            58.1 (.44)         62.4 (.72)       4.3***              53.4 (.41)         57.7 (.79)         4.3***
                                                                                                                                                                               Reading         2002            50.4 (.41)         48.7 (.54)      –1.7***              57.0 (.40)         53.5 (.53)        –3.5***
                                                                                                                                                                               Writing         2002            54.8 (.36)         54.9 (.46)       0.1                 54.0 (.37)         52.0 (.48)        –2.0***
                                                                                                                                                                               Math            2002            56.2 (.49)         56.0 (.61)      –0.2                 55.0 (.47)         51.9 (.61)        –3.1**

                                                                                                                                                                               A strike or lockout
                                                                                                                                                                               Reading         1998             44.2 (.33)        47.3 (.73)       3.1***              49.0 (.33)         50.5 (.76)         1.5**
                                                                                                                                                                               Writing         1998             51.5 (.31)        54.3 (.71)       2.8***              48.1 (.32)         49.7 (.77)         1.6**
                                                                                                                                                                               Math            1998             55.2 (.38)        58.7 (.83)       3.5***              46.8 (.39)         48.3 (.90)         1.5**
                                                                                                                                                                               Reading         2000             48.6 (.32)        43.6 (1.4)      –5.0***              54.5 (.33)         49.7 (1.7)        –4.8***
                                                                                                                                                                               Writing         2000             51.3 (.31)        45.7 (1.5)      –5.4***              52.7 (.31)         47.7 (1.8)        –5.0***
                                                                                                                                                                               Math            2000             59.6 (.38)        54.5 (1.9)      –5.1***              54.9 (.37)         45.2 (2.0)        –9.7***
                                                                                                                                                                               Reading         2002             49.7 (.34)        50.5 (1.2)       0.8                 55.6 (.33)         57.2 (1.1)         1.6*
                                                                                                                                                                               Writing         2002             54.7 (.30)        58.0 (1.0)       3.2***              53.1 (.33)         56.7 (1.0)         3.6***
                                                                                                                                                                               Math            2002             56.1 (.40)        55.9 (1.4)      –0.6                 53.9 (.39)         51.9 (1.4)        –2.0*

                                                                                                                                                                               A work-to-rule campaign
                                                                                                                                                                               Reading        2000              48.0 (.36)        49.4 (.69)       1.4**               53.6 (.36)         57.2 (.70)         3.6***
                                                                                                                                                                               Writing        2000              50.4 (.34)        53.1 (.67)       2.7***              51.9 (.34)         54.7 (.69)         2.8***
                                                                                                                                                                               Math           2000              58.0 (.43)        63.2 (.76)       5.2***              53.2 (.41)         58.8 (.81)         5.6***
                                                                                                                                                                               Reading        2002              50.4 (.41)        48.6 (.54)      –1.8***              57.1 (.39)         53.4 (.53)        –3.3***
                                                                                                                                                                               Writing        2002              55.0 (.36)        54.8 (.47)      –0.2                 54.3 (.37)         51.9 (.49)        –2.4***
                                                                                                                                                                               Math           2002              56.3 (.49)        55.9 (.62)      –0.4                 55.1 (.47)         51.7 (.61)        –3.4***

                                                                                                                                                                               Notes: These schools have results for all 18 Grade 3 or Grade 6 assessments over the six-year period studied and belong to school boards
                                                                                                                                                                               with at least 15 schools. This is the sample used in the remaining tables.
                                                                                                                                                                               aDiff. is the difference between mean pass rates at disrupted schools and non-disrupted schools. A positive Diff. indicates mean pass

                                                                                                                                                                               rates are higher at disrupted schools and a negative Diff indicates mean pass rates are lower at disrupted schools.
                                                                                                                                                                               *Indicates that the null hypothesis of equal assessment results across schools with and without the labour market disruption would be
                                                                                                                                                                               rejected at a 10 percent level of significance (one sided); **at 5 percent; and ***at 1 percent.
                                                                                                                                                                               Source: Author’s calculations.

                                                                                                                                                                               (2005). To summarize, the Ministry of Education                        characteristics of a 20 percent sample of households
                                                                                                                                                                               provided three years (from the six years studied)                      in those areas. Using both the 1996 and 2001 cen-
                                                                                                                                                                               of postal codes of all students at all elementary                      suses, a weighted average of the characteristics of
                                                                                                                                                                               schools in Ontario. Postal codes place students in                     adult persons associated with these schools (living
                                                                                                                                                                               very small geographic areas with known average                         in the same geographic areas as the children who

                                                                                                                                                                                                                             Canadian Public Policy – Analyse de politiques,                vol . xxxvii , no . 4 2011

                                                                                                                                                                    CPPDec2011Inside.indb 485                                                                                                                          29/11/11 4:07 PM
486 David R. Johnson

                                                                                                                                                                               attended the school) is constructed. Social variables                      addresses of the students writing the assessment in
                                                                                                                                                                               include, for example, the percentage of lone parents                       each year. There is only one meaningful set of social
                                                                                                                                                                               or the percentage of adults with some university                           and economic characteristics for each school over
https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26

                                                                                                                                                                               education. There are also economic variables, for                          the entire six-year period.3 There are no data for the
                                                                                                                                                                               example, average income and the unemployment                               individual groups of students writing the assessment
                                                                                                                                                                               rate of the school’s population. It is quite import-                       at each school in each year. Thus the school-level
                                                                                                                                                                               ant that these data are from the postal codes of the                       data are used to represent, with error, the character-
                                                                                                                                                                               actual students who attend a school in order to                            istics of students writing the assessments. As long
                                                                                                                                                                               represent the characteristics of the adults from the                       as the measurement error is unrelated to the strike
                                                                                                                                                                               communities in which the students live. However,                           and work-to-rule occurrences, this does not raise
                                                                                                                                                                               the data are from the addresses of all students at-                        problems in analyzing the effects of labour relations
                                                                                                                                                                               tending the school in three different years, not the                       disturbances.

                                                                                                                                                                               Table 4
                                                                                                                                                                               Summary Information on Social and Economic Characteristics of Schools in Ontario and Labour Relations Disruptions

                                                                                                                                                                               Variable                                 Mean for         Schools with Labour        Schools without Labour        Difference in Means:
                                                                                                                                                                                                                       All Schools       Relations Disruptions       Relations Disruptions      Labour Disruption Minus
                                                                                                                                                                                                                           (SD)                Mean (SD)                   Mean (SD)             No Labour Disruptions

                                                                                                                                                                               Single parent (percent)                    22.2                   23.2                        20.8                        2.4***
                                                                                                                                                                                                                          (8.5)                  (8.9)                       (8.0)
                                                                                                                                                                               English or French as                       91.2                    88.7                       94.2                      –5.5***
                                                                                                                                                                               home language (percent)                    (9.6)                  (11.2)                      (6.0)
                                                                                                                                                                               Aboriginal status (percent)                 1.3                    1.2                         1.4                        –.2***
                                                                                                                                                                                                                          (1.8)                  (1.7)                       (1.9)
                                                                                                                                                                               Recent immigrants –                         4.6                     5.9                        3.0                        2.9***
                                                                                                                                                                               in Canada < 5 years                        (5.7)                   (6.7)                      (3.8)
                                                                                                                                                                               (percent)
                                                                                                                                                                               Percentage of dwellings                    64.7                    61.8                        68.4                     –6.6***
                                                                                                                                                                               that are detached                         (23.3)                  (25.1)                      (20.2)
                                                                                                                                                                               Average household                         63,519                  64,227                     62,659                     1708***
                                                                                                                                                                               income (2000 dollars)                    (18,886)                (21,238)                   (15,529)
                                                                                                                                                                               Unemployment rate of                        6.4                     6.9                        5.9                        1.0***
                                                                                                                                                                               persons with children                      (3.4)                   (3.7)                      (3.0)
                                                                                                                                                                               (percent)
                                                                                                                                                                               Adults with some                           22.7                    23.8                        21.5                       2.3***
                                                                                                                                                                               university education                      (11.3)                  (11.9)                      (10.4)
                                                                                                                                                                               (percent)
                                                                                                                                                                                Number of observations                   3,098                   1,698                       1,400                        NA

                                                                                                                                                                                Note: *Indicates that the null hypothesis of equal assessment results across schools with and without the labour relations disruption
                                                                                                                                                                                would be rejected at a 10 percent level of significance (one sided); **at 5 percent; and ***at 1 percent.
                                                                                                                                                                                Source: Author’s calculations.

                                                                                                                                                                               Canadian Public Policy – Analyse de politiques,                    vol . xxxvii , no . 4 2011

                                                                                                                                                                    CPPDec2011Inside.indb 486                                                                                                                           29/11/11 4:07 PM
Do Strikes and Work-to-Rule Campaigns Change Elementary School Assessment Results?                       487

                                                                                                                                                                                   The left-hand column of Table 4 presents the aver-           would be estimated. Pi,S,t is the pass rate at school
                                                                                                                                                                               age characteristics of all schools in the province used          S on assessment i in year t. There is, for each ob-
                                                                                                                                                                               in the analysis. Put simply, schools are very different          servation at a school in a year, a vector of dummy
https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26

                                                                                                                                                                               in their social and economic composition. This illus-            variables Dt where the value of Dt is unity in year t
                                                                                                                                                                               trates why comparing the unconditional difference                and zero otherwise. LS,t is a variable measuring if
                                                                                                                                                                               in average pass rates will generate a biased measure             there is a labour relations disruption and εi,S,t is a
                                                                                                                                                                               of the impact of labour relations disruptions. The               random error term. The coefficient vector δt captures
                                                                                                                                                                               middle columns present average social and economic               the variation in pass rates across schools between
                                                                                                                                                                               characteristics for schools with and without labour              years on the same assessment i, the variation ob-
                                                                                                                                                                               relations disruptions at any time in the six years               served in Tables 2 and 3. The term in equation (1),
                                                                                                                                                                               of data. Schools without labour disruptions have                 βi XG,S,t , would capture the association of the social
                                                                                                                                                                               fewer lone parents, more French or English speakers,             and economic characteristics X of the grade-cohort
                                                                                                                                                                               more Aboriginals, fewer immigrants, more detached                with pass rates on assessments of type i. The char-
                                                                                                                                                                               homes, lower household incomes, lower unemploy-                  acteristics of the grade-cohort writing the different
                                                                                                                                                                               ment rates, and a less educated adult population as              assessments are the same in each year at each school
                                                                                                                                                                               measured by the percentage of adults in the school               but the relationship between characteristic and result
                                                                                                                                                                               community with some university education. The                    could vary by type of assessment. The coefficient
                                                                                                                                                                               most important source of these differences is that the           vector βi would inform us that students who come
                                                                                                                                                                               two very large boards covering Metropolitan Toronto              from families with a higher level of parental edu-
                                                                                                                                                                               are the only two entirely urban boards in the province           cation are predicted to perform better on a specific
                                                                                                                                                                               and both Toronto boards experienced labour relations             assessment i, say the reading assessment.
                                                                                                                                                                               disruptions. When school pass rates, averaged over
                                                                                                                                                                               three years, are predicted for a cross-section using                The term in the equation denoted μi,S is a measure
                                                                                                                                                                               the set of social and economic variables in Table 4,             of the average quality of the school over the six
                                                                                                                                                                               about 40 percent of the variation in school pass rates           years of assessments. If at one school μi,S is posi-
                                                                                                                                                                               is associated with variation in the social and economic          tive and at another school μi,S is negative, then at
                                                                                                                                                                               background of students. When the differences between             the two schools with the same social and economic
                                                                                                                                                                               schools in social and economic background are ignored            characteristics, results are different. Johnson (2005)
                                                                                                                                                                               in the analysis of disruptions, as they are in Table 3,          labels the school with the higher value of μ i,S a
                                                                                                                                                                               then the analysis of the association between disrup-             higher-quality school. These school quality effects
                                                                                                                                                                               tions and student achievement is incorrect. A better             are not presented in this paper.4
                                                                                                                                                                               methodology is needed.
                                                                                                                                                                                                                                                   Finally, the association of the labour relations
                                                                                                                                                                                                                                                disruption at the school in a year on assessment of
                                                                                                                                                                               Labour Relations Disruptions and                                 type i is measured by the coefficient λi on a measure
                                                                                                                                                                               the A cademic P erformance of                                    of labour relations disruption at that school in that
                                                                                                                                                                               Elementary Students in Ontario:                                  year LS,t. The value of this coefficient varies by type
                                                                                                                                                                               Methodology                                                      of assessment; it may be that strikes do not reduce
                                                                                                                                                                                                                                                writing results but reduce mathematics results.
                                                                                                                                                                               If we had data on the social and economic character-
                                                                                                                                                                               istics of the members of each grade (3 or 6) cohort                As already stated, the school social and eco-
                                                                                                                                                                               writing an assessment in each year at each school,               nomic characteristics, now denoted XS, vary only
                                                                                                                                                                               the panel regression equation                                    by school, that is, the actual data have no time
                                                                                                                                                                                                                                                dimension. The data do not describe the specific
                                                                                                                                                                               Pi,S,t = α + δt Dt + βi XG,S,t + μi,S + λi LS,t + εi,S,t   (1)   characteristics of the students writing the assessment

                                                                                                                                                                                                                        Canadian Public Policy – Analyse de politiques,        vol . xxxvii , no . 4 2011

                                                                                                                                                                    CPPDec2011Inside.indb 487                                                                                                          29/11/11 4:07 PM
488 David R. Johnson

                                                                                                                                                                               at the school in a year, but only the average social            This specification tests for a non-linear association
                                                                                                                                                                               and economic characteristics of all students at-                between assessment results and the length of the
                                                                                                                                                                               tending a school over the six-year period. Thus in              disruption.
https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26

                                                                                                                                                                               the panel setting of equation (1), it is not possible to
                                                                                                                                                                               estimate equation (1) directly. An indirect approach               The fourth measure of disruptions is designed
                                                                                                                                                                               is taken to obtain the estimates of λi, the coefficients        to ask if the association between disruptions and
                                                                                                                                                                               of interest.                                                    achievement varies by social and economic status
                                                                                                                                                                                                                                               of the school. Here equation (2) is augmented as
                                                                                                                                                                                 The econometric model estimated that represents
                                                                                                                                                                               equation (1) is                                                 Pi,S,t = α + δt Dt + γi Si + λi1 LS,t
                                                                                                                                                                                                                                                      + λi2 LS,t ((EDS–mean(ED))/sd(ED))
                                                                                                                                                                                   Pi,S,t = α + δt Dt + γi Si + λi LS,t + εi,S,t.   (2)               + εi,S,t.                                     (3)

                                                                                                                                                                               Equation (2) is estimated separately for each assess-           The effect of a disruption now varies with the
                                                                                                                                                                               ment in Grade 3 and Grade 6 with standard errors                position of the school in the social and economic
                                                                                                                                                                               clustered at the school level. This is a regression             gradient. To be very specific, the social and eco-
                                                                                                                                                                               with school fixed effects (Si) and year fixed effects           nomic status of the school is measured by the
                                                                                                                                                                               (Dt). The coefficients on the school fixed effects              standardized logarithm of the percentage of adults
                                                                                                                                                                               capture both the variation across schools in the so-            in the school community who have some university
                                                                                                                                                                               cial and economic background of students and the                education, denoted ((EDS–mean(ED))/sd(ED)). This
                                                                                                                                                                               school quality effect; the two cannot be identified             variable is approximately normally distributed. For a
                                                                                                                                                                               separately with the available data. The fixed effects,          school at the mean value of this variable (zero), the as-
                                                                                                                                                                               both year and school, are allowed to vary by type of            sociation of disruption and performance on assessment
                                                                                                                                                                               assessment i. A school could, for example, have an              of type i is measured by λi1. For a school two standard
                                                                                                                                                                               exceptional Grade 6 mathematics teacher. When L S,t             deviations above (below), the association of disruption
                                                                                                                                                                               is non-zero, there is a labour relations disruption in          and performance takes the value λi1+ 2 λi2 (λi1– 2 λi2).
                                                                                                                                                                               that year at that school. Equation (2) estimates the            The predictions and standard errors are reported at the
                                                                                                                                                                               effects of labour relations disruptions by compar-              mean and one and two standard deviations above and
                                                                                                                                                                               ing results at disrupted schools with results at those          below the mean.
                                                                                                                                                                               same schools in years without a disruption. There are
                                                                                                                                                                               various measures of the labour relations disruptions.
                                                                                                                                                                                                                                               Labour Relations Disruptions and
                                                                                                                                                                                  The first measure of a labour relations disruption           the A cademic P erformance of
                                                                                                                                                                               is a simple indicator variable that takes a value of            Elementary Students in Ontario:
                                                                                                                                                                               one where there is a disruption in that year. The               Results
                                                                                                                                                                               second measure enters the number of days of the
                                                                                                                                                                               disruption in that year as the value of LS,t. The co-           Table 5 presents various measures of the association
                                                                                                                                                                               efficient on this variable represents the effects on            of labour relations disruptions and outcomes using
                                                                                                                                                                               the pass rate per day of the strike or the work-to-             the framework of equation (2). The first row presents
                                                                                                                                                                               rule campaign. If this coefficient were negative, the           coefficients for each type of assessment—reading,
                                                                                                                                                                               product of the coefficient and a longer strike would            writing, and mathematics—on an indicator variable
                                                                                                                                                                               be larger in absolute value than for a shorter strike.          that a school experienced either type of disruption, a
                                                                                                                                                                               A second model is estimated where the number of                 strike or a work-to-rule, in that year. The coefficients
                                                                                                                                                                               days of the disruption as well as the square of the             are all negative. Only the coefficient measuring
                                                                                                                                                                               number of days of the disruption enters the equation.           the association of Grade 6 mathematics results and

                                                                                                                                                                               Canadian Public Policy – Analyse de politiques,            vol . xxxvii , no . 4 2011

                                                                                                                                                                    CPPDec2011Inside.indb 488                                                                                                         29/11/11 4:07 PM
https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26

CPPDec2011Inside.indb 489
                                                                              Table 5
                                                                              The Association of Labour Relations Disruptions and Pass Rates in the Year of the Disruption by Grade, Type of Assessment, and Measure of Disruption

                                                                              Variable Measuring the Labour Relations                                         Grade 3                                                                      Grade 6
                                                                              Disruption
                                                                                                                                     Reading                  Writing                 Mathematics                 Reading                  Writing               Mathematics

                                                                                                                                                                All labour relations disruptions
                                                                              Indicator variable for disrupted school               –0.52 (.35)              0.09 (.33)                –0.40 (.41)               –.31 (.34)               –.27 (.34)              –.87 (.41)**

                                                                                                                                                                          Strikes or lockouts
                                                                              Indicator variable for strike                         –.09 (.55)               –.19 (.55)                 –.04 (.64)                .10 (.49)                .25 (.52)               –1.0 (.65)
                                                                              Number of days of strike                             –.034 (.048)             –.029 (.31)                –.030 (.05)               .013 (.04)               .047 (.04)              –.076 (.05)
                                                                              Number of days of strike                               .211 (.22)              .010 (.20)                –.136 (.26)               .015 (.19)              –.055 (.20)              –.107 (.26)
                                                                              Number of days of strike squared                      –.018 (.01)             –.002 (.01)                .007 (.02)               –.0001 (.01)             .007 (.01)               .002 (.01)

                                                                                                                                                                   Work-to-rule campaigns
                                                                              Indicator variable for work-to-rule                  –.89 (.42)**              –.13 (.38)                –.95 (.48)**              –.26 (.40)               –.21 (.39)              –.90 (.47)**
                                                                              Number of days of work-to-rule                      –.041 (.01)***             .011 (.01)               –.034 (.01)**             –.027 (.01)*             –.011 (.01)             –.051 (.01)***

                            Canadian Public Policy – Analyse de politiques,
                                                                              Number of days of work-to-rule                       –.024 (.05)              –.093 (.04)**              –.064 (.057)               .061 (.04)             –.073 (.04)              .095 (.05)*
                                                                              Number of days of work-to-rule squared              –.0004 (.001)            .003 (.001)***              .0009 (.001)             –.002 (.001)*            .001 (.001)            –.004 (.001)***

                                                                              Note: This table presents the coefficients on labour relations disruption variables using equation (2). School and year fixed effects are included in the estimating equation. The values in parenthe-
                                                                              ses are standard errors.
                                                                                                                                                                                                                                                                                       Do Strikes and Work-to-Rule Campaigns Change Elementary School Assessment Results?

                                                                              *Indicates that the null hypothesis of a zero coefficient is rejected at 10 percent; **at 5 percent; and ***at 1 percent.
                                                                              Source: Author’s calculations.

                            vol . xxxvii , no . 4 2011
                                                                                                                                                                                                                                                                                       489

29/11/11 4:07 PM
490 David R. Johnson

                                                                                                                                                                               labour market disturbances is significantly differ-            the reduction in the pass rate with an additional day
                                                                                                                                                                               ent from zero. It is quite a large coefficient, and it         of work-to-rule are quite precisely estimated for
                                                                                                                                                                               means that a school experiencing a labour relations            the reading and mathematics assessments. In those
https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26

                                                                                                                                                                               disruption, either strike or work-to-rule, has a pass          assessments, an additional day of work-to-rule re-
                                                                                                                                                                               rate 0.87 percentage points lower in the year of               duces assessment results by about 0.04 percentage
                                                                                                                                                                               the disruption relative to other years at the same             points per day. Thus a work-to-rule campaign that
                                                                                                                                                                               school. The other coefficients are not statistically           is ten days in length, a fairly ordinary work-to-rule
                                                                                                                                                                               significant.                                                   campaign (see Table 1), is predicted to reduce pass
                                                                                                                                                                                                                                              rates by 0.4 percentage points. This is considerable
                                                                                                                                                                                  The next row presents coefficients on an indica-            evidence that work-to-rule campaigns are associated
                                                                                                                                                                               tor variable related to the presence of a strike at            with lower assessment results, even for the aggre-
                                                                                                                                                                               the school in that year. None of the coefficients are          gate of schools. The last row of the table measures
                                                                                                                                                                               statistically significant. Is this a surprising result?        work-to-rule disruptions by the number of days and
                                                                                                                                                                               Recall from Table 1 that several of the strikes are            the number of days squared. Some of the coefficients
                                                                                                                                                                               very short and it might be difficult to observe any            on the quadratic terms are significant and others are
                                                                                                                                                                               impact of all strikes when strikes of such different           not. There may be more complex interactions of the
                                                                                                                                                                               length are treated as equivalent in impact.                    length of work-to-rule campaigns and assessment
                                                                                                                                                                                                                                              results to be further considered.
                                                                                                                                                                                  The next rows of results address the issue of
                                                                                                                                                                               strike length in two ways. The first strike measure is            The most interesting results of this study are
                                                                                                                                                                               the number of days of the strike and the coefficient           found in Table 6, which presents the results of the
                                                                                                                                                                               is the effect on the pass rate per day of a strike.            specification found in equation (3). Consider the co-
                                                                                                                                                                               Once again none of the coefficients are statistic-             efficient -0.191 in the upper left corner under Grade
                                                                                                                                                                               ally significant, but four of the six coefficients are         3 reading. This coefficient means that a school with
                                                                                                                                                                               negative. The coefficient on Grade 6 mathematics               the level of the normalized logarithm of parental
                                                                                                                                                                               is the largest coefficient. The next row presents a            education at two standard deviations below the mean
                                                                                                                                                                               pair of coefficients for each assessment, where in             will experience a reduction of the pass rate of 0.191
                                                                                                                                                                               equation (2) there are now two variables. The upper            percentage points per day of strike. For a strike of
                                                                                                                                                                               coefficient is on the number of days of the strike             ten days in length, this is 1.91 percent of the class
                                                                                                                                                                               and the lower coefficient is on the squared number             predicted to fail who would otherwise not fail. The
                                                                                                                                                                               of days of the strike. Again there are no significant          average percentage of adults in a school commun-
                                                                                                                                                                               coefficients. Non-linear effects in strike length do           ity with some university education is 22 percent in
                                                                                                                                                                               not appear to be important.                                    Table 4. Schools two standard deviations below the
                                                                                                                                                                                                                                              mean of the logarithm of the education variable have
                                                                                                                                                                                  The lower portion of Table 5 repeats the analy-             about 7 percent of adults in the school community
                                                                                                                                                                               sis for work-to-rule campaigns, the more common                with some university education, a school two stan-
                                                                                                                                                                               way for schools to experience labour relations                 dard deviations above the mean of the logarithmic
                                                                                                                                                                               disruptions. There is some evidence that work-                 variable in Table 6 has about 50 percent of the adult
                                                                                                                                                                               to-rule campaigns reduce assessment results. All               population with some university education. The as-
                                                                                                                                                                               coefficients on the indicator variables are negative.          sociation of strike days and reduction in results falls
                                                                                                                                                                               Three of the coefficients on the simple indicator              dramatically as schools move up the socioeconomic
                                                                                                                                                                               variable are statistically significant, including both         gradient. The negative effects of strike days in Grade
                                                                                                                                                                               coefficients on the mathematics results. When the              3 are strongest on the mathematics assessments,
                                                                                                                                                                               analysis is repeated using the number of days of               then the reading assessments. The relationship of
                                                                                                                                                                               work-to-rule campaigns, the coefficients measuring             strike days to writing results is not statistically

                                                                                                                                                                               Canadian Public Policy – Analyse de politiques,           vol . xxxvii , no . 4 2011

                                                                                                                                                                    CPPDec2011Inside.indb 490                                                                                                      29/11/11 4:07 PM
https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26

                                                                              Table 6
                                                                              The Change in the Predicted Pass Rate in the Year of the Disruption by Grade and Type of Assessment at Different Values of the Percentage of Adults Associated with the School
                                                                              with Some University Education

CPPDec2011Inside.indb 491
                                                                              Standard Deviations (approximate                                             Grade 3                                                                   Grade 6
                                                                              percentage with some university education)
                                                                                                                                    Reading                Writing              Mathematics                  Reading                  Writing              Mathematics

                                                                                                                              Effects per strike day evaluated at different points in distribution of logarithm of education
                                                                                         –2 (7%)                                  –.191 (.105)*          –.070 (.096)            –.285 (.124)**            –.097 (.109)            –.091 (.120)            –.410 ( .147)***
                                                                                         –1 (12%)                                 –.116 (.071)*          –.051 (.064)            –.164 (.083)**            –.047 (.071)            –.029 (.078)            –.261 (.097)***
                                                                                           0 (20%)                                –.041 (.049)           –.031 (.044)            –.043 (.056)                .001 (.044)            .032 (.047)            –.112 (.060)*
                                                                                         +1 (32%)                                  .032 (.056)           –.011 (.055)             .077 (.063)                .051 (.051)            .094 (.051)*            .037 (.067)
                                                                                         +2 (52%)                                  .107 (.086)           –.008 (.085)             .198 (.097)**              .101 (.083)            .156 (.086)*            .186 (.110)*

                                                                                                                           Effects per work–to–rule day evaluated at different points in distribution of logarithm of education

                                                                                         –2 (7%)                                  –.072 (.036)**         –.038 (.032)            –.097 (.041)**              .013 (.038)           –.003 (.038)            –.033 (.043)

                                                                                         –1 (12%)                                 –.059 (.024)**         –.017 (.022)            –.071 (.029)**            –.003 (.026)            –.006 (.025)            –.040 (.030)

                                                                                           0 (20%)                                –.046 (.016)***         .002 (.014)            –.045 (.019)**            –.020 (.017)            –.010 (.015)            –.048 (.019)*

                                                                                         +1 (32%)                                 –.033 (.016)**          .023 (.014)*           –.019 (.017)              –.037 (.017)**          –.013 (.017)            –.056 (.017)***

                            Canadian Public Policy – Analyse de politiques,
                                                                                         +2 (52%)                                 –.020 (.024)            .043 (.021)**           .006 (.025)              –.054 (.027)**          –.016 (.028)            –.064 (.026)**

                                                                              Note: The values in parentheses are standard errors. Education level at the school is measured by the normalized logarithm of the percentage of households with some university education for
                                                                              students attending that school. School and year fixed effects are included in the estimating equation.
                                                                                                                                                                                                                                                                              Do Strikes and Work-to-Rule Campaigns Change Elementary School Assessment Results?

                                                                              *Indicates that the null hypothesis of a zero coefficient is rejected at 10 percent; **at 5 percent; and ***at 1 percent.
                                                                              Source: Author’s calculations.

                            vol . xxxvii , no . 4 2011
                                                                                                                                                                                                                                                                              491

29/11/11 4:07 PM
492 David R. Johnson

                                                                                                                                                                               significant. In Grade 6 the relationship of strike            campaign. The reasons for this reversal of pattern
                                                                                                                                                                               days to mathematics pass rates is the only significant        in Grade 6 remain to be understood.
                                                                                                                                                                               coefficient. But the effect is very strong. A school
https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26

                                                                                                                                                                               two standard deviations below the mean of the
                                                                                                                                                                               normalized education variable is predicted to have            Conclusions
                                                                                                                                                                               a 4.1 percentage point reduction in the mathemat-
                                                                                                                                                                               ics pass rate after a ten-day strike. Coefficients two        The paper measures the differences in assessment
                                                                                                                                                                               standard deviations below the mean, one standard              results between schools in Ontario that have experi-
                                                                                                                                                                               deviation below the mean, and at the mean are all             enced labour relations disruptions (strikes, lockouts,
                                                                                                                                                                               statistically significant and large in absolute value.        and work-to-rule campaigns) and schools that have
                                                                                                                                                                               Strikes reduce Grade 6 mathematics results for the            not experienced these disruptions.
                                                                                                                                                                               majority of Grade 6 students.
                                                                                                                                                                                                                                                The results show that both strikes and work-
                                                                                                                                                                                   The lower part of Table 6 uses equation (3) to            to-rule campaigns are associated with reduced
                                                                                                                                                                               investigate the association of assessment results             assessment results in Grade 3 reading and math-
                                                                                                                                                                               and work-to-rule campaigns across the social and              ematics. The predicted reduction in assessment
                                                                                                                                                                               economic gradient. There are three noteworthy                 results is larger per day of strike than per day of
                                                                                                                                                                               results. Where both the coefficient on the number             work-to-rule. In Grade 3, the largest reductions in
                                                                                                                                                                                                                                             results are found at schools where children arrive
                                                                                                                                                                               of days of strikes and the coefficient on the num-
                                                                                                                                                                                                                                             with other disadvantages as measured by the level
                                                                                                                                                                               ber of days of work-to-rule are negative, then the
                                                                                                                                                                                                                                             of parental education.
                                                                                                                                                                               coefficient on the strike days is larger in absolute
                                                                                                                                                                               value. Strikes do have a larger impact than work-to-
                                                                                                                                                                                                                                                The picture in Grade 6 is more complicated.
                                                                                                                                                                               rule campaigns. There is evidence in Grade 3 that
                                                                                                                                                                                                                                             Additional strike days at disadvantaged schools
                                                                                                                                                                               work-to-rule campaigns are associated with lower
                                                                                                                                                                                                                                             are associated with reduced results in mathematics
                                                                                                                                                                               reading and lower mathematics results, and these
                                                                                                                                                                                                                                             in Grade 6; the effect is very large and extends to
                                                                                                                                                                               effects are quite large at schools where parental
                                                                                                                                                                                                                                             the midpoint of the social and economic gradient of
                                                                                                                                                                               education is low. Even at a school with the mean
                                                                                                                                                                                                                                             schools. Additional days of a work-to-rule campaign
                                                                                                                                                                               value of the gradient variable, an additional day
                                                                                                                                                                                                                                             in Grade 6 are associated with lower assessment
                                                                                                                                                                               of a work-to-rule campaign is associated with an
                                                                                                                                                                                                                                             results in reading and mathematics at the schools
                                                                                                                                                                               estimated 0.046 percentage point reduction in the
                                                                                                                                                                                                                                             where parental education is highest, not lowest.
                                                                                                                                                                               Grade 3 reading pass rate and a 0.045 percentage
                                                                                                                                                                                                                                             These effects are smaller.
                                                                                                                                                                               point reduction in the Grade 3 mathematics pass
                                                                                                                                                                               rate per day of work-to-rule. The coefficients are
                                                                                                                                                                                                                                                There is further work to be done. Using student-
                                                                                                                                                                               quite precisely estimated, perhaps because there
                                                                                                                                                                                                                                             level data would increase our understanding of what
                                                                                                                                                                               were numerous work-to-rule campaigns of widely                students are most hurt by strikes and work-to-rule
                                                                                                                                                                               varying lengths. The third observation from Table 6           campaigns. Student-level data would help us to
                                                                                                                                                                               is that the association of work-to-rule campaigns and         understand whether the reductions in academic
                                                                                                                                                                               results for Grade 6 is counterintuitive. There is no          performance are temporary or permanent if stu-
                                                                                                                                                                               association between results and the number of days            dents were tracked over time. In the period studied,
                                                                                                                                                                               of a work-to-rule campaign in Grade 6 at the schools          there are no individual student data. Finally, more
                                                                                                                                                                               in the lower part of the education gradient. However,         research needs to be done to understand the mech-
                                                                                                                                                                               at schools one standard deviation and two standard            anism through which strikes are associated with
                                                                                                                                                                               deviations above the mean of the education gradient,          the larger reduction in Grade 3 and 6 pass rates at
                                                                                                                                                                               assessment results fall when there is a work-to-rule          disadvantaged schools while in work-to-rule cam-

                                                                                                                                                                               Canadian Public Policy – Analyse de politiques,          vol . xxxvii , no . 4 2011

                                                                                                                                                                    CPPDec2011Inside.indb 492                                                                                                    29/11/11 4:07 PM
Do Strikes and Work-to-Rule Campaigns Change Elementary School Assessment Results?                             493
                                                                                                                                                                                                                                                    4 The
                                                                                                                                                                               paigns results fall more at disadvantaged schools in                        residual εS in Pi,S = α + βi XS + εS , a modified
                                                                                                                                                                               Grade 3 and more at advantaged schools in Grade 6.                version of equation (1), provides an estimate of school
                                                                                                                                                                                                                                                 quality. The equation is estimated over a cross-section of
https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26

                                                                                                                                                                                                                                                 schools with test results averaged over a number of years.
                                                                                                                                                                                                                                                 A positive residual is interpreted as a better quality school,
                                                                                                                                                                               Notes
                                                                                                                                                                                                                                                 that is, a school where for similar social and economic
                                                                                                                                                                               Useful comments on this paper were received at the Target         characteristics, assessment results are better.
                                                                                                                                                                               Conference at the University of British Columbia, from                5 The “school quality” terminology implicitly assumes
                                                                                                                                                                               the C.D. Howe Institute, from my colleagues at Laurier
                                                                                                                                                                                                                                                 that resources are similar across schools, a reasonable
                                                                                                                                                                               and from the referees of this journal.
                                                                                                                                                                                                                                                 assumption for schools in Ontario where teacher quali-
                                                                                                                                                                                   1 The period of this study followed a decade of con-          fications are provincewide and base funding to boards
                                                                                                                                                                               siderable change in the Ontario education system. Even            is also provincewide. Hence the same average resources
                                                                                                                                                                               before the period studied, there were strikes and work-           are available to each student at a board. There is no fur-
                                                                                                                                                                               to-rule campaigns. In the years from 1992/93 to 1997/98,          ther information about resource allocation to individual
                                                                                                                                                                               there were three stoppages and nine work-to-rule cam-             schools in Ontario in this period. There are provincially
                                                                                                                                                                               paigns at elementary schools as well as seven stoppages           imposed rules on the length of the instructional day, the
                                                                                                                                                                               and 29 work-to-rule campaigns at secondary schools.               number of instructional days and, for part of the period,
                                                                                                                                                                               Neither stoppages nor strikes were new to the Ontario             on class sizes. In the studies of Ohio and Pennsylvania
                                                                                                                                                                               education system. Campoleti, Hebdon, and Hyatt (2005)             cited above, districts did vary in overall resources avail-
                                                                                                                                                                               report some evidence on the incidence of teacher work             able in a meaningful way.
                                                                                                                                                                               stoppages relative to other unions in Ontario. The effect
                                                                                                                                                                               of labour relations disruptions on Ontario students cannot
                                                                                                                                                                               be measured prior to 1998/99 because there are no data            References
                                                                                                                                                                               on student results at either the individual or school level.      Baker, M. 2011. “Industrial Actions in Schools: Strikes
                                                                                                                                                                                   2 Thesedata are explored in detail in Johnson (2005).            and Student Achievement.” Working Paper 16846.
                                                                                                                                                                               They cover roughly 3,500 of 4,000 elementary schools in              National Bureau of Economic Research, New York.
                                                                                                                                                                               Ontario. During the period studied, Ontario students did          Campolieti, M., R. Hebdon, and D. Hyatt. 2005. “Strike
                                                                                                                                                                               not have tracking numbers that allow study of individual             Incidence and Strike Duration: Some New Evidence
                                                                                                                                                                               student data.                                                        from Ontario.” Industrial and Labor Relations Review
                                                                                                                                                                                                                                                    58(4):610-30.
                                                                                                                                                                                   3The postal codes do not identify the grade of the stu-       Carini, R.M. 2008. “New Directions for the Study of Col-
                                                                                                                                                                               dent at the school. The information is simply the number             lective Bargaining in Schools.” Journal of Collective
                                                                                                                                                                               of students at that school who live at a given postal code in        Negotiations 32(4):317-28.
                                                                                                                                                                               that academic year. The social and economic characteristics       Johnson, D.R. 2005. Signposts of Success: Interpreting
                                                                                                                                                                               are calculated for the average student at the school, not the        Ontario’s Elementary School Test Scores. Policy Study
                                                                                                                                                                               students actually participating in the assessment. When this         No. 40. Toronto, ON: C.D. Howe Institute.
                                                                                                                                                                               average is calculated for three of the six years studied, using   Rose, J.B. 2002. “The Assault on School Teacher Bar-
                                                                                                                                                                               the census closest to the year of enrolment data, the average        gaining in Ontario.” Relations Industrielles/Industrial
                                                                                                                                                                               characteristics of a school change by only very small amounts.       Relations 57(1):100-128.
                                                                                                                                                                               A very small amount of demographic data is collected directly     Thornicroft, K.W. 1994. “Teachers Strikes and Student
                                                                                                                                                                               from and about the students who write the assessment. These          Achievement: Evidence from Ohio.” Journal of Col-
                                                                                                                                                                               variables, available only since 2000/01, are three slightly          lective Negotiations 23(1):27-40.
                                                                                                                                                                               different language variables and the percentage of students       Wright, D. 2004. “Voice, Accountability and Dialogue:
                                                                                                                                                                               receiving special education support in that grade. Johnson           Recommendations for an Improved Collective Bar-
                                                                                                                                                                               (2005) shows that this set of variables is not very useful in        gaining System for Teacher Contracts in British
                                                                                                                                                                               equations like (1), and the data based on postal codes and           Columbia.” Commission to Review Teacher Collective
                                                                                                                                                                               the census are more useful.                                          Bargaining, Don Wright, Commissioner, December.

                                                                                                                                                                                                                        Canadian Public Policy – Analyse de politiques,             vol . xxxvii , no . 4 2011

                                                                                                                                                                    CPPDec2011Inside.indb 493                                                                                                                29/11/11 4:07 PM
494 David R. Johnson

                                                                                                                                                                               Zirkel, P. 1992. “The Academic Effects of Teacher             Zwerling, H.L. 2008. “Pennsylvania Teachers’ Strikes
                                                                                                                                                                                  Strikes.” Journal of Collective Negotiations in the          and Academic Performance.” Journal of Collective
                                                                                                                                                                                  Public Sector 21(2):123-38.                                  Negotiations 32(2):151-72.
https://www.utpjournals.press/doi/pdf/10.3138/cpp.37.4.479 - Friday, May 15, 2020 12:44:35 PM - Legislative Library of British Columbia IP Address:207.194.253.26

                                                                                                                                                                               Canadian Public Policy – Analyse de politiques,          vol . xxxvii , no . 4 2011

                                                                                                                                                                    CPPDec2011Inside.indb 494                                                                                                  29/11/11 4:07 PM
You can also read