A Systematic Review of the Effects of Learning Environments on Student Learning Outcomes - Terry Byers, Marian Mahat, Kirra Liu, Anne Knock and ...
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Technical Report 4/2018
A Systematic Review of the Effects
of Learning Environments on Student
Learning Outcomes
Terry Byers, Marian Mahat, Kirra Liu, Anne Knock and Wesley Imms
The University of Melbourne
2
A Systematic Review of the Effects of Learning Environments on Student Learning Outcomes
Byers, T., Mahat, M., Liu, K., Knock, A., & Imms, W. (2018). Systematic Review of the Effects of Learning Environments on Student Learning
Outcomes. Melbourne: University of Melbourne, LEaRN. Retrieved from: http://www.iletc.com.au/publications/reports
ISBN: 978-0-7340 5502-6
ARC Linkage project (2016-2019)
© Innovative Learning Environments & Teacher Change, LEaRN, The University of Melbourne, 2018.
This publication copyright is held by Innovative Learning Environments & Teacher Change, LEaRN, and the University of Melbourne.
Except as permitted under the Australian Copyright Act 1968 no part of this publication may be reproduced, stored in a retrieval system,
communicated or transmitted in any form or by any means without prior written permission.
This research is supported by Australian Research Council’s Linkage Projects funding scheme (project LP150100022). The
views expressed herein are those of the authors and are not necessarily those of the Australian Research Council
Acknowledgements
We would like to acknowledge the literature contributions of graduate researchers in the ILETC project and for their participation at various
stages of the systematic review process. We would also like to thank the ILETC Chief Investigators for their input at various stages of phase
1 of the project.
Design and layout: Lachlan Stewart.
Cover images: (front) Pegasus Primary School, Jasmax Architecture, Stephen Goodenough photography; (rear) Our Lady of Sion College,
Law Architects, Adam Thwaites photography.
Lead Chief Investigator
Associate Professor Wesley Imms
Chief Investigators Research Assistants
Professor David Clarke Kirra Liu
Dr Ben Cleveland Roz Mountain
Associate Professor Kenn Fisher Lachlan Stewart
Professor Lisa Grocott
Professor John Hattie Graduate Researchers
Professor Thomas Kvan Raechel French
Associate Professor Clare Newton Anne Knock
Victoria Leighton
Project Manager Daniel Murphy
Joann Cattlin Mark Osborne
Dion Tuckwell
Lead Research Fellow/Research Manager
Ethel Villafranca
Dr Marian Mahat
Pamela Yang
Fiona Young
Research Fellows
Chris Bradbeer
Dr Terry Byers
TECHNICAL REPORT A Systematic Review of the Effects of Learning Environments on Student Learning Outcomes
Assumption College, Y2 Architecture, Peter Clarke photography. 3
Overview
Aim Results
The systematic review identified evidence that Of the 5,521 articles retrieved, 20 were included
different learning environments (blended, innovative in this review. The studies ranged from single-site
learning environment (ILE), open-plan and traditional) comparative studies through to quasi-experimental
have an impact on student learning outcomes. There randomised designs at multiple sites. Samples
are significant methodological questions around the ranged from 17 to 22,679 students from primary
availability and viability of empirical evidence. This and secondary schools. The review revealed that
systematic review investigated how researchers assessment regimes that favoured the prevailing
measure changes in academic outcomes attributed view of academic progress in the domains of literacy
to the intervention of changes to the primary and and numeracy were most common. Importantly, the
review identified few robust and valid instruments
secondary schooling learning environments.
that assessed the impact of different spatial
Method layouts on student learning in the 21st Century
learning domains of creativity, critical thinking,
A search of twelve databases, which integrated
communication, collaboration and problem-solving.
fields of education or design, identified those
studies that addressed student learning outcomes
Interpretation
in a range of environments in both primary and
The review presented a small number of studies
secondary educational settings. Quantitative data
with adequate quality, sampling and statistical
was extracted using a customised form, with
process to isolate and then evaluate the impact of
the application of various processes to assess
different learning environment types. These studies
bias, reliability and validity to document changes
presented evidence of a positive correlation between
in discrete measure/s of academic or learning
learning environments, and improvements in student
outcomes.
academic achievement. At the same time, the
review highlighted the need for further longitudinal
evaluation of how different learning environments
impact a broader spectrum of student academic
outcomes.
4
5
Contents
Overview 4
Introduction 8
Research Questions 10
Methods 10
Systematic Review Results 14
Risk of Sampling Bias and Quality Assessment 28
Discussion 32
Conclusions 38
References 42
6Anglican Church Grammar School Centenary Library, Brand + Slater Architects. Christopher Frederick Jones photography. 7
Introduction
Rationale
School learning environments are a matter of global Development (OECD) countries, such as Australia
policy and systemic government investment (Dumont and New Zealand, to invest significant public funding
& Istance, 2010). The strategic reconsideration of in new school buildings. In Australia alone, more than
school learning spaces is a response to demographic, AU$16 billion was approved for investment in school
economic and technological changes that have building projects from 2009 (Wall, 2009).
altered the perceptions of what constitutes effective
Despite the current interest and systemic investment
teaching and learning (see MCEETYA, 2008; New
in school learning environments, there is a lack of
Zealand Ministry of Education, 2011; New Zealand
empirical data to adequately evaluate how existing
Ministry of Education, 2014; OECD, 2013). The
and alternative learning environments (blended, ILEs
narrative of ‘21st Century Learning’ (creativity, critical
and open) impact teaching and learning (Blackmore,
thinking, communication, collaboration and problem-
Bateman, O’Mara, & Loughlin, 2011; Brooks, 2011;
solving) has prompted some to question the efficacy
Gislason, 2010). Brooks is critical of the overt
of existing classroom models and to put forward
theorising around these new spaces, with a “dearth
blended, open and, more recently, innovative learning
of systematic, empirical research being conducted”
environments (ILEs) (See examples of Alterator &
on their impact on teaching and learning (p. 719). For
Deed, 2013; Benade, 2017; Dovey & Fisher, 2014;
Painter, Fournier, Grape, Grummon, Morelli, Whitmer,
Dumont & Istance; Imms, Cleveland, & Fisher, 2016).
& Cevetello (2013), this lack of evidence stems from
Debates around the form and function of what
the fact that there are very few methodologies and
constitutes an effective learning environment have led
metrics able to isolate and then assess how different
some Organisation for Economic Cooperation and
learning spaces affect both teachers and students.
8There remains little understanding if, or to what
extent, different school learning environments affect
student academic or learning outcomes (Blackmore
et al., 2012).
This systematic review draws on studies from
the integrated fields of education and design. It
focuses on identifying quantitative studies with valid
methodologies that isolate the variable of different
learning environment type/s (blended, ILEs, open-
plan and traditional) and analyse their impact on
reliable measures of student academic achievement.
The ensuing analysis tests the suggestion that there
is currently a lack of substantive, empirical data
around the claims that different learning environment
types correlate, whether positively or negatively, with
student academic or learning outcomes.
Objectives
In the context of learning environment type/s (blended,
ILEs, open-plan and traditional), the objectives of this
review are:
1. To identify studies that investigate the impact of
learning environment types on student learning
outcomes;
2. To identify measures of student learning outcomes
or academic achievement;
3. To determine the content of the identified student
learning outcome measures (those with published
evidence of reliability and validity).
The results of this study will provide researchers
and practitioners with a better understanding of
how currently available student learning outcome
measures are quantified in the context of learning
environments.
9Research Questions
1. What evidence exists that different learning environments have an impact on
student learning outcomes?
2. What student learning outcome measurement tools have been designed and used
for measuring student outcomes in different learning environment types?
3. What elements of student learning are quantified by the identified measurement
tools?
Methods
Review methodology
The review adopted the principles and techniques and the breadth of studies retrieved in the search
of systematic reviews, which involved sifting necessitated a full-team discussion to determine the
abstracts, scrutinising full papers and abstracting final list of included articles. The potential for selection
data. One researcher performed the initial search bias was addressed by the disciplined process
and subsequent data extraction. Two members of followed by reviewers, sometimes bringing into the
the team checked each title and abstract to decide discussion their or another team member’s specific
whether the full paper should be read. The lead team understanding of the construct.
member was consulted if a difference of opinion
arose. Similarly, each full paper was read by at least Search strategy
two members of the team and agreement sought The database search for this systematic review
from the lead team member for any variations of was performed in January 2017 using EBSCOhost
opinions. Two other members of the team checked databases (Academic Search Complete, Avery Index
10% of the abstraction records. The complexity to Architectural Periodicals, Education Research
of the underlying construct of different learning Complete, Educational Administration Abstracts,
environments and student learning outcomes
10ERIC), Proquest databases (Education Database, students in these schools. To ensure that relevant
Art and Humanities Database, Humanities Index, information was not missed, studies were included
PAIS), OVID, Informit, Scopus and Web of Science. that considered known student learning outcome
These databases integrate information from the measures, even if student learning outcomes were
fields of education and design and include articles not the primary study objective. In the current review,
addressing student learning outcomes in a range of the aim was to assess quantitative changes in
environments. A study protocol was not registered. student learning outcomes before and after assigned
Operational definitions were determined for each intervention(s); therefore, articles were included on the
aspect of our search, which related directly to the basis of discrete measure/s of academic or learning
research question and selection criteria (Table 1). outcomes.
The search terms were developed using related
literature and chosen by team consensus based on
their theoretical and practical significance. Search
terms addressed the concepts of student learning
outcomes and different learning environment type/s,
and the use of intervention-based study designs.
Where available, exploded search terms were used,
as well as associated terminology in the title, abstract,
and, where appropriate, the keywords of the articles.
Boolean operators helped narrow the search to
relevant research fields.
Selection criteria
An ILE is defined by the OECD (2013, p. 11) as “an
organic, holistic concept – an ecosystem that includes
the activity and the outcomes of the learning”. The
concept embraces the learning taking place as well
as the setting. The OECD describes ILEs as multi-
modal, technology-infused and flexible learning
spaces that are responsive to evolving educational
practices (OECD, 2015). The selection criteria for this
review (Table 2) aimed to document how researchers
perceived student learning outcomes and therefore
focused on the definitions they provided. To ensure
the inclusion of a comprehensive breadth of articles,
the application of the selection criteria did not use
operational definitions of student learning outcomes.
The population of interest in this review was both
primary and secondary school students. Therefore,
studies were limited to those which involved
11Table 1: Study search terminology
Topic Search terms Exploded search terms (abstract/title)
Population Elementary and Secondary School “Elementary school student*”
students “Elementary student*”
“Primary school student*”
“Secondary school student*”
“Secondary student*”
“High school student*”
“Junior high school student*”
“Junior high student*”
“Middle school student*” Pupil*
Schoolchild*
Schoolboy*
Schoolgirl*
Student Learning Student learning outcomes “Student learning outcome*”
Outcome
Academic achievement “Academic achievement*”
Academic outcomes “Academic outcome*”
Academic success “Academic success*”
Educational achievement “Educational achievement*”
Educational outcomes “Educational outcome*”
Grade point average “Grade point average*”
GPA*
Student outcomes “Student outcome*”
Innovative Innovative learning space “Learning space*”
Learning
Environments Modern learning space
Contemporary learning space
Physical learning space
21st century learning space
Innovative learning environment “Learning environment*”
Modern learning environment
Contemporary learning environment
Physical learning environment
21st century learning environment
Contemporary Innovative learning environment “Learning environment*”
learning space
Modern learning environment
Contemporary learning environment
Physical learning environment
21st century learning environment
Physical environment "Physical environment*"
Physical space "Physical space*"
School environment "School environment*"
School space "School space*"
Classroom environment "Classroom environment*"
12Table 2: Slection criteria
Search Include Exclude
Population Primary and secondary students Kindergarten and post-secondary students
Outcomes of Quantitative Qualitative
selected studies
Design Intervention-based studies including Other systematic reviews or literature reviews
randomised controlled trial, quasi-
randomised controlled trial, single-group
pre- and post-test studies, and single
case experimental designs.
Publication type Articles published as full texts in peer Articles or abstracts not published in peer
review journals review journals
Articles or abstracts published in languages
other than English
Grey literature (conference proceedings theses,
books and other grey literature)
Systematic reviews (although reference lists
were used to ensure all relevant publications
were located)
Generalised discussion papers of participation
measures that did not present new evidence
from a scientific study
Qualitative studies as the focus was on
quantitative outcome measures
Time 1960 - 2016 Prior to 1960
Data collection and assessment of quality and selection). While, assessments of internal
consistency and reliability were made on the relevant
Covidence was the primary screening and data
aspects of the COnsensus-based Standards for
extraction tool used for the systematic review.
the selection of health Measurement INstruments
The evaluation of sampling bias, reliability and (COSMIN) checklist (Terwee et al., 2012).
validity ascertained the quality of selected studies.
The Cochrane Collaboration tool assessed the risk Synthesis of studies
of selection, detection, attrition and reporting bias A two-part narrative approach analysed the results
(Higgins et al., 2011). Even though the tool was of the selected studies. First, the research questions
developed specifically for assessing randomised of this review framed the analysis of findings of the
control trials, it presented a viable means to evaluate selected studies. Second, a synthesis of the collective
a wider variety of methodological designs of the results addressed potential gaps and issues and
selected studies. A summary of the relevant measures established a frame for future meta-analysis.
utilised by the selected studies was summarised to
outline key characteristics, differences and similarities
across the final study selection. The work of Campbell
and Stanley (1963) informed the assessment of the
internal validity (history, instrumentation, maturation
13Systematic Review Results
The initial database search revealed 5,521 articles languages other than English, non-peer reviewed
after applying filters based on the selection criteria: journal articles and articles published before 1960.
primary and secondary students in quantitative Appraisal of titles and abstracts excluded 4,409
intervention-based and single-case experimental articles, with 72 articles undergoing full-text review.
studies, published as full texts in peer review journals Only 20 of these made the final analysis; 51 did not
between 1960 and 2016. Figure 1 displays the number have student academic or learning outcomes as a
of references yielded during the initial database search dependent variable or statistically analysed changes
and subsequent stages of the review. Following this in measures of achievement.
identification, the number of references was reduced
to 4,481 after the removal of duplicates, articles in
Records identified
through database
searching
(n = 5,521)
Identification
Records after
duplicates removed
(n = 1,040)
Records screened Records excluded
Screening
(n = 4,481) (n = 4,409)
Full-text articles Full-text articles
Eligibility assessed for eligibility excluded, with reasons
(n = 72) (n = 52)
Studies included in
qualitative synthesis
(n = 0)
Included
Studies included in
quantitative synthesis
(n = 20)
Figure 1. PRISMA flow diagram of the articles yielded during the systematic review process (including removal of duplicates and
references that did not align with selection criteria). The demographic descriptors of participants in each of the studies, the student
outcomes measured and study characteristics are shown in Table 3.
14Table 3: Demographic descriptors, student outcomes measured and study characteristics.
First Author, Year, Title Sample Characteristics Student Outcome
Measures
n Age (M, SD, Ra) Sex (Mb:Fc) Level
Primary (P)
Secondary (S)
Bottge, B., (2006). Total = 17 R = 15-18 10:7 Sd Maths achievement on
Situating math various tests
instruction in rich
problem-solving
contexts: Effects on
adolescents with
challenging behaviors.
Chandra, V., (2008). Total = 233 R = 15-16 NDf S Achievement on test
The methodological scores
nettle: ICT and student (7:7)
achievement.
Cicek, F .G., (2016). Total = 66 R = 15-16 S Academic achievement
Laboratory Control of students in Web
System’s effects on Experiment = 33 Design
student achievement
Control: 33
and attitudes.
Forman, S. G., Total = 129 R = 6-7 (66:63) Pg IQ, creativity,
(1978). Creativity achievement
and achievement of Open = 63 Open
second graders in (32:31)
Trad = 66
open and traditional
Trad.h
classrooms.
(34:32)
Fößl, T., (2016). A Total = 85 M = 10.6 (77:8) P Learning performance
field study of a video
supported seamless- Experiment = 24 SD = .31
learning-setting with
Control = 23
elementary learners.
FC1 = 25
FC2 = 13
McRobbie, C. J., Total = 1,594 R = 12-18 NDf S Student outcomes:
(1993). Associations student and class mean
between student Inquiry and LE scores
outcomes and =591 Attitude
psychosocial science and LE = 596
environment.
Ozerbas, M. A., (2016). Total = 58 R = 12-13 (31:27) S Academic success
The Effect of the
Digital Classroom on Experiment = 32
academic success and
Control = 26
online technologies
self-efficacy.
15Instrument Use Theory Focus of study LE Effects on SO Effect size
Multimedia aligned NDf To examine the effects Generative Students performed better FCC
tests: Fraction of of enhanced anchored maths LEf. on both curriculum aligned 0.75
the Cost Challenge instruction (EAI) on the tests: FCC & KCC which were
(FCC), Kim’s Komet maths achievement multimedia and hands-on, but KKC
Challenge (KKC) Vs. (particularly fraction showed no improvement in 0.78
Traditional: Fractions knowledge) of students the fractions computation test
computation test, of low achievement and standardised tests.
Standardized tests due to challenging
behaviours.
Test scores ND To assess the Trad.h LE Vs The blended LE improved NDf
achievement of two blended or student achievement in
groups of students e-learning comparison to the trad.
over two years: one LE. However, the comfort with
traditional, and one blended LE differed and was
blended, to determine independent of test-score.
whether ICT has an
impact on achievement.
Scale of Attitudes ND To determine the Laboratory Post-tests of academic ND
towards Learning academic achievement control achievement, retention and
and Teaching of students in Web system positive attitude after doing
Process and the Design in an LCS and (LCS) vs Web design were higher in
Achievement Tests Trad. LE. Trad. LE. the LCS LE. The Trad. LE also
(pre- and post-tests) improved achievement.
Primary Mental ND To compare the Open vs Students in traditional ND
Abilities Test (PMAT), level of creativity closed. classrooms scored higher in
Iowa Tests of Basic and achievement of fluency (in one school system),
Skills students in open LE vs vocabulary, and reading and
Trad. LE. mathematics achievement.
No significant differences for
uniqueness.
Learning ND To compare the Open- The students in the 0.31
performance learning performance of learning experimental/seamless LE
(pre- and post- students in a seamless approach setting performed better
test), Video views, real-world learning with video- on the post-test than the
Working progress, setting (using open supported students in a Trad. LE.
survey, student & learning and video) and seamless-
teacher interview, a Trad. LE. learning-
observations setting.
The Science ND To investigate the Science LE. Student outcomes were found ND
Laboratory correlation between to be affected by psychosocial
Environment the psychosocial environment-independent of
Inventory environment and their ability. Attitude and inquiry skill
effect on student outcomes were enhanced
outcomes. when laboratory activities
were integrated into non-
laboratory classes.
Pre and post-test ND To determine the effect Digital LE. The experiment group in ND
of the Academic of digital classrooms on the digital classroom has
success test and the academic success significantly higher academic
online technologies of students and success than the controls (no
self-efficacy scale technology self-efficacy. access to digital technology).
There was no effect on online
technology self-efficacy.
16First Author, Year, Title Sample Characteristics Student Outcome
Measures
n Age (M, SD, Ra) Sex (Mb:Fc) Level
Primary (P)
Secondary (S)
Reiss, S., (1975). Total = 182 (30 R = 7-8 (15:15) per P Persistence,
Persistence, per six schools school achievement
achievement, approx.)
and open-space
environments. Open = 85
Closed = 88
Shamaki, T. A., (2015). Total = 337 ND ND S Academic achievement
Influence of Learning
Environment on
Students’ Academic
Achievement in
Mathematics: A
Case Study of Some
Selected Secondary
Schools in Yobe State
– Nigeria.
Solomon, D., Total = 92 R = 9-10 (56:36) P Academic achievement,
(1976). Individual creativity, inquiry skills,
Characteristics and attitudes, behaviour
Children’s Performance
in “Open” and
“Traditional” Classroom
Settings.
Tanner, C. T., Total = 1,916 (24 R = 8-9 ND P Academic achievement:
(2008). Explaining schools) Iowa Test of Basic Skills
Relationships Among score (ITBS)
Student Outcomes
moreover, the School’s
Physical Environment.
Uline, C., (2008). Total = 1,134 (80 ND ND M Academic achievement
The walls speak the schools
interplay of quality
facilities, school
climate, and student
achievement.
Kazu, I. Y., (2014). Total = 54 R = 17-18 (37:17) S Academic performance
The effect of blended
learning environment Experiment = 27
model on high school
Control =37
students’ academic
achievement.
17Instrument Use Theory Focus of study LE Effects on SO Effect size
Behavioural post-test ND To examine the Open vs Open LE’s promoted higher ND
for persistence on difference in persistence Trad. LE. persistence on difficult tasks
a difficult task, the and achievement in than the traditional LE across
Stanford Preschool open-space LE’s vs all genders. Persistence and
Internal-External Trad. LE’s. achievement were more
(I-IC) Scale, and positively correlated in open
the California LE than Trad. LE for boys
Achievement Test only..
(CAT)
structural ND To investigate the Ideal vs dull Student performance was ND
questionnaire as well influence of the LE on LE. negatively correlated with dull
as an achievement mathematics students’ painting, over-crowdedness
test academic achievement. and poor lighting and
positively correlated with
adequate seating/room.
Adequate ventilation had no
effect.
Classroom ND To examine children’s Open vs Children in open classes ND
observations, characteristics and Trad. LE. scored significantly higher
questionnaires, Iowa performance in open in creativity, democratic/
Tests of Basic Skills and traditional settings. co-operative behaviour and
involvement in class activities
but significantly lower on
the achievement test,
undisciplined activity level and
social involvement.
Iowa Tests of Basic ND To investigate the Four As school design score MC
Skills effects of four sets different increased, so did achievement 0.069
of design patterns in design on the ITBS in each of 24
the physical school patterns: schools. LGMP
environment: movement ML, LGMP, 0.018
and circulation (MC), DLV, IN.
DLV
large group meeting
0.025
places (LGMP),
daylight and views IN
(DLV), and instructional 0.031
neighbourhoods (IN) on
achievement.
Teacher surveys, ND To examine the LE with School facility quality was ND
student SES and mediating role of school variable mediated by school climate
achievement data climate on the facility facilities variables: academic press,
quality level and student quality level. teacher professionalism and
achievement and community engagement,
the interplay of these and was found to affect
factors. student achievement.
Inadequate facilities meant
a focus on academics were
lowered & teachers were less
enthusiastic.
Pre and post- ND To compare students’ Blended LE Students learning in the ND
knowledge test, prior academic achievement vs Trad. LE. blended LE had better
achievement grades in a Blended LE and a academic outcomes than
Trad. LE. those in the Trad. LE.
18First Author, Year, Title Sample Characteristics Student Outcome
Measures
n Age (M, SD, Ra) Sex (Mb:Fc) Level
Primary (P)
Secondary (S)
Barrett, P., (2015). The Total = 3,766 R = 5-11 (1,883:1,883) P Reading, Writing and
impact of classroom Mathematics progress
design on pupils’ Year 1 = 447 points added to create
learning: Final results Year 2 = 606 an Overall Progress
of a holistic, multi-level Year 3 = 744 score
analysis.
Year 4 = 656
Year 5 = 708
Year 6 = 606
Barrett, P., (2017). Total = 3,766 R = 5-11 (1,883:1,883) P Reading, Writing and
The holistic impact (same as 2015 Mathematics progress
of classroom spaces study) points added to create
on learning in specific an Overall Progress
subjects. Blackpool = 715 score
Hampshire = 1,535
Ealing = 1,480
Byers, T., (2014). Total = 386 R = 11-14 (164:0) S English and
Making the case for Mathematics academic
space: The effect of n = 164 (6 achievement (A+ to E-)
learning spaces on classes in
teaching and learning. intervention) Verbal and Non-Verbal
Reasoning standardised
n = 222 (control data (as a proxy of
group who cognitive ability)
remained in
traditional with Student attitudinal
only learning
outcomes
compared
19Instrument Use Theory Focus of study LE Effects on SO Effect size
National Curriculum ND Assessments of the Existing Correlations of Overall ND
Key Stages 1 and 2 design parameters Classrooms Progress for each pupil
tests of Naturalness (Light, against environmental
Temperature, Air measures showed all ten
Quality), Individualisation parameters were positively
(Ownership, Flexibility) correlated with progress.
and Stimulation Multilevel (Two level) modelling
(Complexity and portioned between pupils
Colour)G were made (highest) and classroom
of 153 classrooms in levels. School level accounted
27 schools in order to for little (3%) of variance.
identify the impact of Naturalness parameters
the physical classroom (light, temperature and air
features on the quality) highest effect of 28%.
academic progress of Variation of the most effective
the 3766 pupils who classroom (Overall progress of
occupied each of those 16.05 NC points) minus least
specific spaces. effective (8.12 NC points),
results in 7.93 NC points for
variation. The impact of the
classroom environmental
factors, therefore, models at
7.93/50 (50 points is the total)
explains 16% of the variation
in pupils' academic progress
achieved.
National Curriculum ND Assessments of the Existing 1. See above. ND
Key Stages 1 and 2 design parameters Classrooms
tests of Naturalness (Light, 2. % Improvement due to
Temperature, Air classroom parameters
Quality), Individualisation was 9.3% (Reading),
( Ownership, Flexibility) 8.4% (Writing) and 11.7%
and Stimulation (Maths).
(Complexity and
3. For each of the different
Colour) were made
subject models, the
of 153 classrooms in
aspects of the classroom
27 schools in order to
environment taken
identify the impact of
together explained
the physical classroom
approximately 10% of
features on the
the variability of pupil
academic progress of
performance.
the 3766 pupils who
occupied each of those
specific spaces.
English and ND Examined the impact Existing A two-tailed, paired t-test Within d
Mathematics school- of learning spaces on Trad. LE with an alpha level of 0.05 =.40 in
based assessment teachers’ pedagogy, (control) and compared participating English
student engagement Retrofitted students’ assessments in
Academic and student learning NGLS- these subjects taken during Within d
Assessment Services outcomes in a model ILE the time they occupied =.41 in
Cognitive Ability Tests technology-rich school affordances. the traditional and the Maths
setting. Its quasi- NGLS classrooms. The null
Linking Pedagogy, Overall d
experimental design hypothesis was rejected
Technology and =.40
allowed examination with nine out of the twelve
Space repeated of differences in these results showing a statistically
measures survey variables between two significant improvement in
settings – ‘traditional’ student learning outcomes.
classrooms, and ‘new These statistically significant
generation learning improvements were justified
spaces.’ (NGLS). by Cohen’s d effect size,
ranging from the upper end of
the small to high effects.
20First Author, Year, Title Sample Characteristics Student Outcome
Measures
n Age (M, SD, Ra) Sex (Mb:Fc) Level
Primary (P)
Secondary (S)
Chang, C. Y., (2006). Total = 155 Mean of 16 years (74:81) S Earth Science Learning
Preferred - Actual Outcomes Inventory
learning environment -Students’ perceptions
“spaces” and earth on preferred/actual
science outcomes in learning environment
Taiwan. and the students’
learning achievement
and attitude
Chang, C. Y., Total = 155 Mean of 16 years ND S Earth Science Learning
(2011). Science Outcomes Inventory
Learning Outcomes -Students’ perceptions
in Alignment with on preferred/actual
Learning Environment learning environment
Preferences. and the students’
learning achievement
and attitude
Gilavand, A., Total = 210 R = 7-11 ND P Student Attitudinal Data
(2016). Investigating
the impact of
schools’ open
space on learning Observational Data
and educational
achievement of
elementary students.
Tanner, C. K. (2000). Total = 22, 679 R = 10 - 11 ND P Reading and
The influence of Mathematics scores
school architecture
on academic
achievement.
Note. a Age range; b Number of males; c Number of females; d Secondary years of schooling; e-Learning environment; f
Not disclosed; g Primary years of schooling; h Traditional
21Instrument Use Theory Focus of study LE Effects on SO Effect size
Earth science ND Determine whether traditional Results indicated that ND
classroom learning preferred–actual space (teacher- although preferred–actual
environment accounts for outcome centred) space is not related to
instrument (ESCLE) variance beyond that learning achievement, but is both
and Earth Science explained by actual environment statistically and practically
Learning Outcomes learning environment. (TLE) and associated with attitudes
Inventory (ESLOI) mixed (both toward the subject when
teacher- and actual learning environment
student- is controlled. regression
centred analysis revealed that the
learning pre-test scores were the
environment only significant predictors in
(MLE) explaining students’ learning
outcomes.
Earth science ND Aimed to investigate Student– Students preferred a ND
classroom learning students’ learning Teacher- classroom environment
environment environment Balanced where student-centred and
instrument (ESCLE) preferences and Instructional teacher-centred instructional
and Earth Science to compare the Model, approaches coexisted
Learning Outcomes relative effectiveness STBIM) and (STBIM) over a teacher-
Inventory (ESLOI) of instructional teacher- centred (TCIM) learning
approaches to centred environment. It was also
students’ learning group (the revealed that the STBIM
outcomes in 10th-grade Teacher- students’ achievement in and
earth science classes. Centred attitude toward earth science
Instructional were enhanced when the
Model, learning environment was
TCIM). congruent with their learning
environment preference.
Construction ND This study investigated Open Schools’ open space has a ND
Observational the impact of schools’ spaces – significant impact on learning
Checklist open space on learning exterior play and academic achievement
and educational spaces (ascertained through a
Academic achievement of questionnaire) of elementary
Achievement elementary students in school students in Ahvaz-
Motivation Ahvaz, Southwest of Iran.
Questionnaire Iran.
Iowa Test of Basic ND Focused on the Used SD A total of 7 design factors ND
Skills (ITBS) neglected aspect of the & PL scale were found to correlate to
physical environment of to assess student learning outcomes
the classroom impact. existing (compatibility with context,
Its purpose was to classroom clearly defined pathways,
University of determine how school spaces positive outdoor spaces,
Georgia’s School of architectural design computers for teachers,
Design and Planning factors might influence positive overall impression
Laboratory (SD & PL) student achievement had the greatest impact on
scale scores in elementary ITBS scores.
schools.
22Participants Byers et al., 2014; Chandra & Lloyd, 2008; Cicek &
Taspinar, 2016; Kazu & Demirkol, 2014; Shamaki,
The number of student participants in each of the
2015). The majority of studies utilised an amalgam of
studies ranged from 17 to 22,679 students (average
standardised test measures. Most prominently used
of 1,665). The age of students in the final studies
was the Iowa Test of Basic Skills (ITBS), followed by
ranged from 5-18yrs. Equitable distribution of boys
the California Achievement Test (CAT) and the English
and girls occurred in the respective samples, with the
National Curriculum Key Stages 1 and 2 tests. The
only exception being the Byers, Imms, and Hartnell-
external, standardised nature of these items presented
Young (2014) study (with an all-boys school as the
a reliable and valid means to facilitate generalisable
site). In many instances, authors either undertook a
analysis across multiple schools (see Barrett, Davies,
randomised selection process or ensured the sample
Zhang, & Barrett, 2015, 2017; McRobbie & Fraser,
reflected key student demographic characteristics
1993; Reiss & Dyhdalo, 1975; Tanner, 2000, 2008;
(i.e. ethnicity, socio-economic status and location)
Uline & Tschannen-Moran, 2008).
with equitable distribution between control and
intervention groups. A high proportion of the selected Even though this review focused on measures around
studies engaged in pre-testing of student academic student academic learning outcomes, many utilised
performance to show that they were statistically assessment of student attitudes to learning and
similar. Consequently, these sampling measures observations of students in various spatial layouts.
moderated the incidence of selection bias that would Some used repeated measure surveys to elicit
distort the statistical analysis of the between-group student attitudinal responses (see Byers et al., 2014;
comparisons. Chang, Hsiao, & Barufaldi, 2006; Chang, Hsiao, &
Chang, 2011; Cicek & Taspinar, 2016; Fößl, Ebner,
Outcomes and Measures
Schön, & Holzinger, 2016; Gilavand, Espidkar, &
The student outcomes identified in this review Gilavand, 2016). In others, comparative observations
were measured using computations of observed of traditional learning environments and ILEs (Fößl
participation behaviours, a variety of standardised et al., 2016; Gilavand et al., 2016; Solomon &
tests, general achievement tests, prior achievement Kendall, 1976) focused on discerning the differences
data and participant surveys. The range of measures in pedagogies and learning experiences. Finally, a
used to determine student outcomes is displayed in small number of studies (Barrett et al., 2015, 2017;
Table 4. Tanner, 2000, 2008) assessed the physical design
parameters (i.e. individualisation and stimulation)
The assessment of student learning outcomes
and environmental factors (i.e. light, temperature and
covered a range of assessment devices and types
air quality) to investigate the potential impact of the
across the 20 studies. Seven studies utilised school-
physical classroom features on student academic
based assessment to determine the impact of different
progress.
learning environment types on student academic
achievement. These school-based assessments
were in English, Mathematics and Science (see
23Table 4: Student outcomes, measures and constructs of final articles.
First author, year Measure Student outcome Construct
Bottge, B., (2006). Fraction of the Cost Maths achievement FCC = 36-point test with
Challenge (FCC) points allotted for correct
working and answer.
Kim’s Komet Challenge Maths achievement KCC = 37-point test with
(KKC) points allotted via degree
of difficulty.
Fractions computation Maths achievement Fractions computation
test test.
Standardized tests- the Academic achievement ITBS: Standardised
Iowa Tests of Basic Skills achievement in
(ITBS): Mathematics Mathematics test that
subtest. result in measures
such as National Grade
Equivalents (NGE).
Chandra, V., (2008). Test scores Achievement on test Tests developed within
scores the school were sat,
providing performance
data.
Cicek, F .G., (2016). Scale of Attitudes Attitude 38-item attitude scale.
towards Learning and
Teaching Process (SALTP)
Achievement test Academic achievement Achievement test-
consisting of 28 items.
Forman, S. G., (1978). Wallach-Kogan Test: Academic achievement Wallach-Kogan Test:
consist of three verbal: Provides a Fluency score-
Instances, Alternate Uses the number of appropriate
and Similarities subtest responses summed over
and two figural subtests: items, and uniqueness
Pattern Meanings and score-total number of
Line Meanings responses that appeared
only once in the study
sample for a given item.
ITBS: Mathematics Creativity ITBS: Standardised
subtest achievement in
Mathematics test that
result in measures
such as National Grade
Equivalents (NGE).
Primary Mental Abilities Achievement PMAT.
Test (PMAT)
Fößl, T., (2016). Learning performance Learning performance Learning performance
test test: 12 single choice
questions and 4 practical
exercises where students
had to draw something.
Students’ attitude Attitude SATWEV-17 item, five-
towards worked example point Likert-type scale
videos (SATWEV) survey survey.
24First author, year Measure Student outcome Construct
McRobbie, C. J., (1993). The Science Laboratory Student outcomes SLEI: Contains 35 items,
Environment Inventory with 7 assessing each of
(SLEI): student outcome the five scales (left) which
measures included four were scored on a 5-point
attitude measures and Likert scale (almost never,
two inquiry skill. seldom, sometimes,
often, very often).
Ozerbas, M. A., (2016). Academic Success Test Academic success AST: a multiple-choice
(AST) test of 24 items and a
maximum score of 100
(no points were taken
away for wrong answers).
Content includes 7th-
grade math lessons,
specific features of
circles.
Online Technologies Self- Competency with the OTSES: 33-item scale,
Efficacy Scale (OTSES) internet with four sub-scales of
Internet competencies,
Synchronous interaction,
Asynchronous interaction
I and Asynchronous
interaction II.
Reiss, S., (1975). Persistence test: A Persistence Persistent test: the
Behavioural post-test average time the child
measuring persistence worked on a puzzle
with puzzle-making. (consisting of problems
1 and 2 of the Wechsler
Intelligence Scale for
Children (WISC) Block
Design Test and Problems
9 and 10 of the Wechsler
Adult Intelligence Scale
(WAIS) Block Design
Test).
The Stanford Preschool Expectancies about locus SPIES: is scored in the
Internal-External Scale of control and behaviour internal direction, and
(SPIES) of children in theoretically represent expectancies
relevant situations for internal control of
positive events (I+) and
negative events (I-) and
the sum of these 2 (total I).
The California Academic achievement CAT: Standard
Achievement Test (CAT) administration in California
near the end of the school
year provides a percentile
ranking of each student.
Shamaki, T. A., (2015). General achievement test Academic achievement -
25First author, year Measure Student outcome Construct
Solomon, D., (1976). CAT Academic achievement CAT: Standardised test
administered in California
near the end of the school
year which provides a
percentile ranking of each
student.
Virginia Standards of Academic achievement SOL: Standardised
Learning (SOL) tests test that produces an
English (Reading, achievement score.
Research, and Literature)
and Math.
Virginia Test of Academic Academic achievement Standardised test that
Proficiency produces an achievement
score.
Tanner, C. T., (2008). ITBS: composite portion Academic achievement Standardised test that
produces an achievement
score or test score.
Uline, C., (2008). ITBS Academic achievement Standardised tests that
produce an achievement
score or test score.
Virginia Standards of Academic achievement Standardised test that
Learning (SOL) tests produces an achievement
English (Reading, score.
Research, and Literature)
and Math
Virginia Test of Academic Academic achievement Standardised test that
Proficiency produces an achievement
score.
Kazua, I. Y., (2014). Pre and post-knowledge Academic achievement Pre-test and post-test
achievement test consisting of 25 and 21
items respectively, with
Prior achievement grades Academic achievement items in compliance
with the objectives of
the Biology course and
Bloom’s taxonomy.
Barrett, P., (2015). National Curriculum Key Academic achievement Reading, Writing and
Stages 1 and 2 tests Mathematics progress
points added to create an
Overall Progress score.
Barrett, P., (2017). National Curriculum Key Academic achievement Reading, Writing and
Stages 1 and 2 tests Mathematics progress
points added to create an
overall progress score.
26First author, year Measure Student outcome Construct
Byers, T., (2014). English and Mathematics English and Mathematics Achievement: A+-E-
school-based academic achievement
assessment
Student attitudinal data Learning and A nine-item, five-point
instrument? engagement Likert scale survey
measuring the effect
of learning space on
students’ learning and
engagement.
Academic Assessment Academic achievement Standardised test.
Services Cognitive
Ability Tests: Verbal and
Non-Verbal Reasoning
standardised data (as a
proxy of cognitive ability)
Chang, C. Y., (2006). Earth Science Learning Students’ learning ESLOI: Divided into
Outcomes Inventory achievement and attitude. two sections with a
(ESLOI)-including the ATESI: attitude towards total of 60 items. The
attitudes toward the earth science and ESAT: first, is the ATESI which
earth science inventory achievement in earth consists of 30 items with
(ATESI) and Earth Science science bipolar disagree/agree
Achievement Test (ESAT) on statements on a 1–5
Likert scale, and second,
the ESAT, with another 30
MCQ items.
Chang, C. Y., (2011). Earth Science Learning Students’ learning ESLOI: Divided into
Outcomes Inventory achievement and attitude. two sections with a
(ESLOI)-including the ATESI: attitude towards total of 60 items. The
attitudes toward the earth science and ESAT: first, is the ATESI which
earth science inventory achievement in earth consists of 30 items with
(ATESI) and Earth Science science bipolar disagree/agree
Achievement Test (ESAT) on statements on a 1–5
Likert scale, and second,
the ESAT, with another 30
MCQ items.
Gilavand, A., (2016). Academic Achievement Academic achievement AAMQH: 29 items based
Motivation Questionnaire and motivation on ten (in this case 9)
of Hermance (AAMQH) characteristics that
distinguish those of high
and low achievement
motivation.
Tanner, C. K. (2000). ITBS: Mathematics and Academic achievement ITBS: Standardised
Reading subtests achievement in Reading
and Mathematics tests
that result in measures
such as National Grade
Equivalents (NGE).
27Risk of Sampling Bias and Quality Assessment
Sampling bias Reiss & Dyhdalo, 1975; Uline & Tschannen-Moran,
2008). Subsequent reviews of attrition, detection and
The Cochrane Collaboration tool assessed the risk of
reporting bias, indicated that the design, sampling
sampling bias. The tool assessed the source of bias
strategy and method of analysis of these studies
in the domains of selection, detection, attrition and
were more robust than those employing a single
reporting on the basis of “low”, “high”, and “unclear”
intervention group (Bottge, Rueda, & Skivington,
risk (Higgins et al., 2011). The studies in the final
2006; Fößl et al., 2016; McRobbie & Fraser, 1993).
selection of this review tended to moderate sampling
The review highlighted the deficiency of these studies,
bias through their design and sampling process (Table
due to their within-group design, to reliably detect
5). Random selection of moderate to large control-
significant differences at the standard of those with a
and intervention-groups from single or multi-sites
comparative group design.
(Barrett et al., 2015, 2017; Chandra & Lloyd, 2008;
Chang et al., 2006; Chang et al., 2011; Gilavand
Quality
et al., 2016; Kazu & Demirkol, 2014; Ozerbas &
Erdogan, 2016; Shamaki, 2015; Tanner, 2000, 2008), The assessment of internal validity and reliability
decreased the incidence of selection bias. There were established the quality of individual studies. Due to
instances where randomisation was not employed. the single intervention or site designs of many studies
However, sampling bias was moderated through pre- in the final selection, the internal validity guidelines
testing of IQ, socioeconomic status or achievement of Campbell and Stanley (1963) were applied to
scores to establish that the comparative samples assess the validity in terms of history, instrumentation,
were not statistically different (Byers et al., 2014; maturation and selection. The remaining studies had
Cicek & Taspinar, 2016; Forman & McKinney, 1978; designs that incorporated multi-sites and random
selection of participants to establish generalisable
28Table 5: Assessment of sampling bias of selected studies using the Cochrane Collaboration tool.
First author, year Selection Detection Attrition Reporting
Bottge, B., (2006) - - + +
Chandra, V., (2008) + + + +
Cicek, F .G., (2016) + + + +
Forman, S. G., (1978) + + ? +
Fößl, T., (2016) - - - +
McRobbie, C. J., (1993) - - ? +
Ozerbas, M. A., (2016) + + - +
Reiss, S., (1975) + + - +
Shamaki, T. A., (2015) + + + +
Solomon, D., (1976) - + + +
Tanner, C. T., (2008) + + + +
Uline, C., (2008) + + + +
Kazu, I. Y., (2014) + + + +
Barrett, P., (2015) + + + +
Barrett, P., (2017) + + + +
Byers, T., (2014) + + + +
Chang, C. Y., (2006) + + + +
Chang, C. Y., (2011) + + + +
Gilavand, A., (2016) + + + +
Tanner, C. K. (2000) + + + +
Note + - ?
Low risk High risk Unknown risk
evidence. Regarding checks for reliability, the articles assessment of the quality of methodologies and
were evaluated by reported measures of internal measures were beyond the scope of the intervention-
consistency and inter-rater reliability as dictated by the based design of the selected studies. The application
COSMIN checklist (Terwee et al., 2012). The rationale of the COSMIN checklists four-point criterion of
for not using the COSMIN criteria to assess the validity “excellent”, “good”, “fair” and “poor” for both
of this selection is due to its focus on assessing large reliability and validity provided an efficient means for
sample, randomised control trials. The checklists establishing the overall assessment quality (“strong”,
29“moderate”, “low” and “unknown”) of each study’s 2015; Barret et al., 2017; Byers et al., 2014; Forman
design and measures. & McKinney, 1978; Gilavand et al., 2016; Kazu &
Demirkol, 2014; Reiss & Dyhdalo, 1975; Shamaki,
The quality of the selected articles ranged from low
2015; Solomon & Kendall, 1976). These articles often
to strong, with the majority falling into the category of
had rigorous elements to the validity of their design,
moderate (Table 6). The studies identified as strong
methods and means of analysis or application of
in terms of quality (Chang et al., 2006; Chang et al.,
measures of internal consistency and reliability, but
2011; Cicek & Taspinar, 2016; Ozerbas & Erdogan,
not both. Some tended to be based on location-
2016; Tanner, 2000, 2008; Uline & Tschannen-Moran,
specific assessments of student learning outcomes,
2008) were best described as large, multi-site,
while, others did not utilise or report the statistical
randomised comparative studies that tended to utilise
processes and reliability measures that were evident
assessment through existing external, standardised
in studies of strong quality. The remaining articles
testing instruments. Not only were their designs
suffered significant methodological and statistical
rigorous, but they utilised, and reported in detail,
deficiencies that lowered the quality of their findings.
intra-rater reliability and internal consistency through
There were correlations between the quality of these
Cronbach’s alpha. A larger group of studies were
studies and the higher incidence of sampling bias.
assessed as having moderate quality (Barrett et al.,
Table 6: The overall score for the quality of selected studies using the COSMIN 4-point checklist.
First author, year History Instru- Maturation Selection Internal Reliability Overall
mentation consistency result
Bottge, B., (2006) Poor Good Poor Poor Excellent Excellent Low
Chandra, V., (2008) Good Good Good Fair Poor Poor Low
Cicek, F .G., (2016) Excellent Good Excellent Excellent Excellent Excellent Strong
Forman, S. G., (1978) Excellent Excellent Excellent Fair Fair Poor Moderate
Fößl, T., (2016) Poor Good Poor Poor Good Good Low
McRobbie, C. J., (1993) Poor Good Poor Poor Good Good Low
Ozerbas, M. A., (2016) Excellent Good Excellent Excellent Excellent Excellent Strong
Reiss, S., (1975) Poor Excellent Good Good Poor Poor Moderate
Shamaki, T. A., (2015) Good Good Good Excellent Poor Poor Moderate
Solomon, D., (1976) Good Excellent Excellent Fair Good Good Moderate
Tanner, C. T., (2008) Excellent Excellent Excellent Good Excellent Excellent Strong
Uline, C., (2008) Excellent Excellent Excellent Good Excellent Excellent Strong
Kazu, I. Y., (2014) Excellent Good Excellent Good Fair Poor Moderate
Barrett, P., (2015) Excellent Excellent Excellent Excellent Good Good Moderate
Barrett, P., (2017) Excellent Excellent Excellent Excellent Good Good Moderate
Byers, T., (2014) Excellent Good Excellent Good Excellent Excellent Moderate
Chang, C. Y., (2006) Excellent Good Excellent Excellent Excellent Excellent Strong
Chang, C. Y., (2011) Excellent Good Excellent Excellent Excellent Excellent Strong
Gilavand, A., (2016) Excellent Excellent Excellent Excellent Fair Excellent Moderate
Tanner, C. K. (2000) Excellent Excellent Excellent Good Excellent Excellent Strong
30Hiukkavaara School and Community Centre, Lukkaroinen Architects / Ecophon Saint Gobain, Juha Sarkkinen photography. 31
Discussion
The review sought to establish an evidence base for a negative impact on student academic achievement.
the connection between learning environment type/s The collective evidence presented by the relatively
(blended, ILEs, open-plan and traditional) and their small number of studies does suggest some
impact on student learning outcomes. A descriptive correlation between the design, function and nature
critique is presented based on an analysis of the final of the physical learning environment and learning
selection of 20 studies, centred on the review’s three outcomes.
research questions.
In this sample, there were only three studies that
What evidence exists that different learning reported effect sizes. The Fößl et al. (2016) and
environments have an impact on student Bottge et al. (2006) studies reported effect sizes that
learning outcomes? ranged from d = .31 to d = .78 respectively. However,
The systematic review identified a small number both had questionable validity and reliability due to
of studies that presented empirical evidence of the issues of high sampling bias and relatively low
the impact of different learning environments, in quality. Byers et al. (2014), with effect sizes of d =
particular ILEs, on student academic outcomes. .40 and .41 for English and Mathematics respectively,
These studies presented evidence that different presented a more valid and reliable assessment of
learning environments, in particular, those aligned the impact of ILEs in comparison to traditional layouts
with the premise of ILEs, can positively impact on student academic achievement. Unlike the Fößl
student academic achievement. However, studies et al. (2016) and Bottge et al. (2006) studies, Byers
in the sample that compared open-plan learning et al. (2014) utilised a between-group comparison
environments with traditional classroom spaces of classes randomly assigned to the different spatial
suggested that the open-plan setting correlated with layouts, while controlling for the influence of student
32You can also read
