Managing National Assessment Report 2014 Tawa College

Managing National
  Assessment
    Report

      2014

  Tawa College

Managing National Assessment Report

Tawa College

18 – 19 June 2014

Introduction
The purpose of the Managing National Assessment (MNA) external review is:
 •    to help schools achieve valid, fair, accurate and consistent internal assessment
      according to the requirements of the Assessment (including Examination) Rules
      for Schools with Consent to Assess 2014 (Assessment Rules); and
 •    in combination with the most recent Education Review Office report, to ensure
      that schools are meeting the requirements of the Consent to Assess Against
      Standards on the Directory of Assessment Standards Rules 2011 (CAAS) and
      its Guidelines (CAAS Guidelines) in order to maintain their consent to assess.
This process is a partnership between schools and the New Zealand Qualifications
Authority (NZQA) to ensure that assessment systems remain effective and internal
assessment decisions are valid.

The MNA review has two components:
  1. The annual external moderation of assessment materials for an NZQA-selected
     sample of internally assessed standards across the curriculum and teacher
     grade judgements on the student work identified by the school’s random-
     selection process.
  2. An external review of each school’s assessment systems at least once every
     four years.

This report summarises NZQA’s evaluation of how effectively:
 •    senior management has addressed issues identified during the previous MNA
      review on 29 March 2011
 •    the school manages assessment for national qualifications
 •    the school manages internal and external moderation
 •    the school manages and makes use of assessment-related data
  •   the school maintains the currency of assessment policy and procedures, and
      communicates them to staff, students and families.

In preparing this report, the School Relationship Manager (SRM) met with the
Principal’s Nominee (PN) and the Heads of Department (HODs) for Mathematics,
Physical Education, Te Reo Māori and Visual Arts, and the Assistant HOD of Social
Sciences. The school also provided pre-review information and the staff and student
assessment procedures guides.

There was a report-back session with the Principal and the Principal’s Nominee at
the end of the visit to highlight strengths and weaknesses, with suggested strategies
for good practice and to agree on recommended actions.

Tawa College MNA Report                    2
June 2014

2011 Managing National Assessment Report (CAAS Guidelines 3v)
Evidence was sought that action items in the previous MNA report have been
effectively addressed.

All items from the 2011 MNA report have been addressed.

Assessment Practice (CAAS Guidelines 2.5v-vii, 2.6, 3v)
How does the school’s assessment practice allow for the effective management of
assessment for national qualifications?

Meeting students’ assessment needs The school community and staff place an
emphasis on academic achievement, resulting in a higher than average percentage
of students who leave from Year 13 going on to university. Coupled with this, the
participation and retention rates for priority learners are above those of other similar
decile schools. The PN noted that this has meant that the school has had to develop
new courses to meet the learning and assessment needs of specific groups of
students. New Drama, English and Science courses have been introduced which
provide for student interest, while assessment against Performing Arts standards
allows students to be assessed using valid evidence collected while participating in
activities outside the classroom such as musical performances and Kapa Haka
festivals. Individualised programmes within courses have given students the ability to
select the standards they wish to enter based upon their readiness for assessment.

The school has not extended its Consent to Assess to enable other new courses, but
has formed arrangements with a number of outside providers to provide students
with more vocationally focussed assessment, including opportunities in Automotive,
Construction and Childcare. Supported learning students are offered assessment
against Self-Management or Work and Study Skills unit standards as part of the
Specific Preparation for Employment and Citizenship course.

In some departments, the majority of evidence of achievement is collected through
formal, written, time-bound assessments. In others, staff describe a range of
different, valid assessment methods that allow students to produce authentic work.
Evidence-centred assessment is briefly mentioned in the teacher guidelines, and
expanding upon the information for teachers to include examples from within the
school would ensure that this good practice in gathering evidence is shared with all
staff.

Further assessment opportunities are not offered for all standards. Those teachers
interviewed who did offer them as part of their courses were clear about the school’s
requirements. Where students had not taken up an assessment opportunity, after
taking part in the learning process, staff follow school requirements and award a Not
Achieved Grade. This is in line with the Assessment Rules.

School documentation for teachers and students provides guidance on a range of
scenarios and outcomes in the event that a student misses an assessment deadline.
Students are required to complete a Missed Assessment application form to provide
evidence to be used to make a decision on whether to grant an extension. The PN
noted that the process works well and is tracked on the pastoral section of the
school’s student management system (SMS).

Tawa College MNA Report                    3
June 2014

Authenticity and breaches of the rules Staff are provided with advice on a range
of effective strategies to manage the authenticity of student work and the use of
many of these was discussed by staff during interviews. The PN reported that, in
some departments, staff have identified issues with the way students reference
material in their assessments. As a consequence, referencing procedures are now
being taught to students in both junior and senior classes in an effort to embed the
skills. Several HODs report that there have been recent instances of students being
suspected of breaching the school’s authenticity rules and detailed the effective
processes that were used to manage these breaches. The PN is kept informed of
any suspected breaches, and records are kept in the pastoral section of the SMS.

Special assessment conditions The school has a large number of students who
have been confirmed as eligible for special assessment conditions (SAC). Effective
processes are used to initially identify these students and their learning needs. As
many of the educational psychologist reports prepared for students did not contain all
of the necessary information, applications for them this year have been through the
alternative evidence pathway. The school has now identified the need to provide a
full needs-analysis for each student, and NZQA has agreed to run a SAC seminar at
the school later in the year to clarify this and other processes, so that the right
students get the appropriate assessment conditions.

All of the teachers interviewed are aware of which, if any, of their students are eligible
for SAC, how to access the information on them and how to obtain support from the
Leaning Support department when required. This is indicative of effective SAC
communication processes within the school.

Derived grades A sample audit of the school’s procedures for making derived
grade applications was conducted in March 2013 for three candidates. Other than
one application where the evidence used to derive a grade for a standard was
gathered from an assessment for a different standard, the school’s administration of
applications and the assessment procedures for deriving valid grades from standard-
specific evidence were found to meet NZQA guidelines. The school has addressed
the identified issue and HODs interviewed during this review described appropriate
methods to ensure evidence for derived grades can be collected from quality assured
and standard-specific assessments.

Privacy Act Staff are advised in their handbook that they must seek student
permission to use their work as benchmark material. However, several of the staff
interviewed acknowledge that written permission is not always obtained.
Management has agreed to review school procedures to ensure that the use of
student work as exemplars is in line with the requirements of the Privacy Act (1993).

Identified action
NZQA and senior management agree on the following action to improve the
management of assessment for national qualifications. Senior management
undertakes to:
    •   review procedures to ensure that the use of student work as exemplars is in
        line with the requirements of the Privacy Act (1993).

Tawa College MNA Report                     4
June 2014

Moderation (CAAS Guidelines 2.6, 3v)
How effectively does school internal and external moderation assure assessment
quality?

Internal moderation processes Teachers are expected to document the internal
moderation procedures they use during each assessment on an Internal Moderation
Cover Sheet (IMCS). The staff interviewed show a good understanding of the
moderation process and were able to provide evidence of their completion for
selected standards. The critiquing of assessment tasks prior to use, the use of
clarifications documents to inform assessment practice and grade verification of
purposefully selected sample of student work were common examples of good
practice in the departments visited. HODs report verification is by another specialist
teacher from within the department where practicable, or by colleagues from outside
of the school.

Audit of internal moderation processes The school documentation requires
teachers to provide the PN with a copy of the IMCS for each standard assessed, and
the PN was able to show evidence of this happening. There is, however, no specific
date provided within the documentation by which this process must be completed,
and the PN only checks a random sample of those submitted. Management cannot
therefore be assured that all results reported to NZQA in the end-of-year data file
submission have been part of an internal moderation process. There is also no
monitoring system in place that enables management to check that teachers are
doing what is documented on the IMCS.

At the beginning of each year HODs are expected to verify for the PN, the quality
assurance processes they will use within their departments that year. At the time of
the review, several of the HODs interviewed had yet to complete this verification for
2014. This does not provide management with an assurance that these processes
have actually been used during the year to produce credible results.

Management must monitor the internal moderation processes in place within
departments, and used by teachers, to ensure they meet the requirements of the
CAAS.

External moderation The PN is responsible for the random selection of students
whose work will be submitted for external moderation. The selection system meets
the requirements of the Assessment Rules. When moderated work is returned to the
school, the PN requires HODs to complete a Response to Moderation Report to
comment on any identified issues and provide action plans to address these. These
are signed off by the PN when she is informed the action is completed, but evidence
is not required. A more active system of monitoring completion of action plans should
be developed.

Moderation issues for some subject areas were identified in 2012. Consequently,
greater numbers of standards were selected for external moderation in these
subjects for 2013. This moderation showed that the grades teachers awarded are
mostly consistent with the national standard. However, for one subject, further issues
were identified that the school had not identified through its own processes. A more
robust process for monitoring the completion of internal moderation would make it
easier for management to identify and resolve issues as they arise.

Tawa College MNA Report                    5
June 2014

The selection and collection of work for moderation by industry training organisations
(ITOs) is carried out by the HOD Transition rather than the PN. Teachers are not
required to complete response reports for this moderation and there is no monitoring
to ensure that identified issues are addressed. Management have agreed to ensure
that ITO and NZQA moderation are dealt with using the same processes.

Identified action
The following action is required improve the school’s internal and external
moderation to assure assessment quality. The school must:
  •   ensure that the systems for moderation are adequate and appropriate, and that
      senior management is assured these meet CAAS guideline 2.6.

NZQA and senior management agree on the following action to improve the school’s
internal and external moderation to assure assessment quality. Senior management
undertakes to:
  •   ensure that similar processes are used for both ITO and NZQA moderation and
      that the PN has oversight of these.

Data (CAAS Guidelines 2.6, 2.7)
How effectively does the school manage and make use of assessment-related data?

Accuracy of assessment-related data The school system for creating and
checking entries is effectively managed, with very few late entries made. However,
not all internal entries had a result recorded for them in the final data submission of
2013. The percentage of unreported results for the school has been high for the last
three years. Management has undertaken to ensure that all entries have a result
reported for them, and to put in place effective data monitoring processes to ensure
data is complete and up-to-date.

Assessment-related data is sent regularly to NZQA. However, the school must
ensure that checks are made on the batch file and the Key Indicator checklist to
identify any issues. In 2013, the school reported results for six standards for which it
did not hold consent to assess. As a consequence, these results were not available
to the students for the award of qualifications until the correct assessor provider code
was supplied in 2014.

Students are expected to sign off their results prior to teachers entering them in the
markbooks within the SMS. However, there is no requirement for staff to check the
results held in their markbooks against NZQA reports to ensure these results have
been transferred correctly. Instead, students are encouraged to make their own
checks on their entries and results through their Learner logins. The school has
become more proactive in getting students registered to use their logins, with Year
11 students taken to the school’s computer rooms in form groups to complete the
process. As a result, 89 per cent of learners registered in 2013. While student
checking is good, it is likely that some students will find this difficult to do. As the
professionals responsible, teachers should also check the accuracy of the data that
the school submits to NZQA.

The school has no policy on the storage of digital evidence of student achievement.
Several of the HODs interviewed have developed their own systems of storage, in
some instances with little back-up capability. Management has agreed to introduce
systems to securely store digital evidence.
Tawa College MNA Report                     6
June 2014

Use of assessment-related data The school uses data provided by NZQA and the
Ministry of Education to analyse its assessment data. The Principal makes this
analysis available to staff and the Board of Trustees. HODs provide an analysis of
their department results to the Principal using specifically defined criteria. They select
three of the standards assessed by the department and compare school results
against national data. Reasons for differences, improvements or deterioration from
previous years or other issues are discussed as part of the analysis. HODs also
explained how their own analysis of departmental data is used to inform future
assessment practice.

Management has also used the analysed data to compare the results for internally
versus externally assessed standards. The school has identified that in comparison
to national data its internal results are lower than its external, and considers this has
reduced the number of endorsements available to students. The Principal is holding
discussions with HODs in an effort to try and identify possible reasons for these
differences and to develop strategies to address them. Cross-referencing the
school’s own data with the results of external moderation would be useful to confirm
the internal results are not lower because of issues with assessment quality.

Identified action
The following action is required to improve the management and use of assessment-
related data. The school must:
  •   ensure processes are in place to check the accuracy of the data on student
      achievement transferred to NZQA (CAAS Guidelines 2.7iii).

NZQA and senior management agree on the following action to improve the
management and use of assessment-related data. Senior management undertakes
to:
  • ensure that all entries have a result reported
  • develop systems to securely store digital evidence.

Communication (CAAS Guidelines 2.4i(f),2.4v, 2.6, 3v)
How effectively does the school maintain the currency of assessment policy and
procedures, and communicate them to staff, students and families?

Communication of policies and procedures to staff, students and their families
The following documents were provided for this visit:
  • Procedures for Assessment Towards National Qualifications: Teacher
    Guidelines 2014
 •    Tawa College Procedures for Assessment Towards National Qualifications:
      Student Guidelines
 •    course assessment statements for some of the departments visited.
The teacher guidelines provide staff with most of the information they require on the
school’s assessment and moderation requirements. There is a lack of clarity in the
guidelines about how and when management will monitor departments’ internal
moderation processes. Management has agreed to review their expectations and
clarify these for staff.

Tawa College MNA Report                     7
June 2014

Staff are reminded of information during daily staff briefings, through the school News
Sheets, departmental meetings and emails from the PN. The PN reported that, while
she still sends paper copies of NZQA circulars to relevant staff, most other
information she receives from NZQA is disseminated to staff through the messaging
facility of the school’s SMS.

The student guidelines are handed out by form teachers who are expected to go over
them with their students. While presently only available in hard copy for students, the
PN indicated the expectation is that all assessment information, including the
guidelines, will eventually be held on the recently introduced learning management
system (LMS), and accessible to the whole school community.

Parents are kept informed through school newsletters and email. Hard copies of
information are posted to parents if no email address is available. The school holds
two NCEA information evenings, one of which is combined with a subject choice
presentation for students. To improve parental understanding of NCEA prior to
students making their subject choices, NZQA-provided NCEA pamphlets are sent out
with the mid-year Year 10 reports.

The school has conducted parent surveys that include questions on the effectiveness
of its communications of assessment information and also gathers informal feedback
from situations where staff interact with students and parents. The PN considers that
the school’s communication systems work well. The development of a portal into the
LMS for parents next year is expected to further enhance communication with this
group.

Course outlines and assessment statements As well as being handed out to
students by subject teachers at the beginning of each year, these are now available
on the LMS. While expected to provide a standardised set of required information on
the standards assessed in a course, teachers are given the freedom to personalise
these to their course. Several of the outlines sighted provided students with extra
information, such as whether standards help meet literacy or numeracy requirements
or if further assessment opportunities were available for specific standards. This is
good practice and could be considered for inclusion when management next reviews
its requirements for course outlines.

Code of Practice for the Pastoral Care of International Students Tawa College
is a signatory to the Code of Practice. The school attests that it annually reviews both
its performance and the accuracy and relevance of all information provided to
international students and their parents, caregivers and homestay. (CAAS Guidelines
2.4v).

Identified action
NZQA and senior management agree on the following action to maintain the
currency of assessment policy and procedures, and communicate them to staff,
students and families. Senior management undertakes to:
  • clarify for staff, how and when management will monitor internal moderation
    processes.

Summary
Tawa College provides students with a wide range of assessment opportunities to
meet individual interests, vocational needs and requirements for qualifications. Staff
share a common understanding of the need to manage authenticity and to ensure
Tawa College MNA Report                    8
June 2014

that work presented for assessment by students is their own. Effective systems are in
place to manage any breaches of the authenticity rules and record the consequences
for students.

The school has processes in place to identify students eligible for special
assessment conditions, and has been proactive in seeking support from NZQA to
clarify the necessary processes involved in obtaining these conditions for its
students.

While there is evidence that internal moderation was happening in all departments
visited, there is no systematic monitoring of this by management other than the
collection of internal moderation cover sheets. External moderation is followed up
effectively, and staff develop action plans in response to identified issues. However,
there is no mechanism for assuring management that these plans are completed
prior to the standard being assessed again. The school must ensure that both
internal and external moderation processes are monitored so that they meet the
CAAS requirements.

Student entries are closely managed to ensure they are accurate and reflect the
intended assessment programme in individual courses. However, not all entries have
a result reported for them and inaccurate information has led to some results being
unavailable to students. The school must have a system to ensure the accuracy of
the achievement information it reports.

The school’s communications with students and parents are being enhanced through
the new learning management system. This offers management and staff the ability
to ensure that consistent assessment information is delivered all members of the
school community.

The 2014 Managing National Assessment review identified significant issues that
prevent the school from meeting the requirements of the Consent to Assess Against
Standards on the Directory of Assessment Standards Rules 2011 and its Guidelines.

Action Items
In order to maintain its consent to assess, the school must
  • ensure that the systems for moderation are adequate and appropriate, and that
     senior management is assured these meet CAAS guideline 2.6.
 •    ensure processes are in place to check the accuracy of the data on student
      achievement transferred to NZQA (CAAS Guidelines 2.7iii).
Based on the outcome of this assessment systems check, it is anticipated that the
next Managing National Assessment review will be conducted within two years.

Ian Munro
Manager
School Quality Assurance and Liaison

29 August 2014

Tawa College MNA Report                    9
June 2014