Managing National Assessment Report 2014 Tongariro School

Managing National
  Assessment
    Report

      2014

Tongariro School

Managing National Assessment Report

Tongariro School

20 & 21 August 2014

Introduction
The purpose of the Managing National Assessment (MNA) external review is:
  •   to help schools achieve valid, fair, accurate and consistent internal assessment
      according to the requirements of the Assessment (including Examination) Rules
      for Schools with Consent to Assess 2014 (Assessment Rules); and
  •   in combination with the most recent Education Review Office report, to ensure
      that schools are meeting the requirements of the Consent to Assess Against
      Standards on the Directory of Assessment Standards Rules 2011 (CAAS) and
      its Guidelines (CAAS Guidelines) in order to maintain their consent to assess.
This process is a partnership between schools and the New Zealand Qualifications
Authority (NZQA) to ensure that assessment systems remain effective and internal
assessment decisions are valid.

The MNA review has two components:
  1. The annual external moderation of assessment materials for an NZQA-selected
     sample of internally assessed standards across the curriculum and teacher
     grade judgements on the student work identified by the school’s random-
     selection process.
  2. An external review of each school’s assessment systems at least once every
     four years.

This report summarises NZQA’s evaluation of how effectively:
  •   senior management has addressed issues identified during the previous MNA
      review on 31 May 2012
  •   the school manages assessment for national qualifications
  •   the school manages internal and external moderation
  •   the school manages and makes use of assessment-related data
  •   the school maintains the currency of assessment policy and procedures, and
      communicates them to staff, students and families.

In preparing this report, the School Relationship Manager (SRM) met with the
Principal’s Nominee (PN) and the Teachers in Charge (TICs) for English,
Mathematics, Trades and Visual Art. The school also provided pre-review information
and the staff and student assessment procedures guides.

There was a report-back session with the Principal and the Principal’s Nominee at
the end of the visit to highlight strengths and weaknesses, with suggested strategies
for good practice and to agree on recommended actions.

Tongariro School MNA Report               2
August 2014

2012 Managing National Assessment Report (CAAS Guidelines 3v)
Evidence was sought that action items in the previous MNA report have been
effectively addressed.

There has been some progress on the 2012 action items. This will be discussed
further in this report.

Identified action
In order to maintain its consent to assess the school must:
  •    follow up external review recommendations and findings (CAAS Guidelines 3).

Assessment Practice (CAAS Guidelines 2.5v-vii, 2.6, 3v)
How does the school’s assessment practice allow for the effective management of
assessment for national qualifications?

In 2013, Tongariro School reviewed the learning and assessment programmes
offered to senior students. As a result, in 2014 the whole timetable was built on
student voice and aims to offer a range of options for students to achieve, both in
traditional academic pathways as well as more vocationally oriented pathways. A
range of outside providers are used to supplement the courses offered by classroom
teachers.

Currently, the school’s programme includes three academies; a Māori Performing
Arts Academy, a Services Academy and a Trades Academy where students work
towards qualifications in Agriculture, Horticulture and Forestry. The school has 32
Gateway placements where students gain work experience in a variety of local
businesses as well as travelling to attend classes at the Bay of Plenty Primary
Industries Trades Academy run by Waiariki Institute of Technology.

Gathering evidence of achievement In some subject areas gathering of evidence
of achievement is carried out under examination conditions. However, in the
academies and in Technology and Visual Arts students are assessed on practical
skills and in a variety of ways. In English, the TIC discussed that she is endeavouring
to use local contexts for assessment activities where possible. To further engage
students, it is suggested that a greater variety of assessment activities are utilised.
Discussion of effective strategies such as gathering video evidence, interviewing
students, using blogs and presentations could assist teachers to explore valid and
credible assessment options.

Missed and late assessments Teachers have a plan for completion of work but
retain flexibility so that students can negotiate to complete work so they have the
best opportunity to succeed. The teachers spoken with confirm that if a student does
not meet a negotiated deadline without an acceptable reason then a Not Achieved
result is reported to NZQA. Acceptable reasons include illness or unforeseen
circumstances beyond the student’s control.

Derived grades The teachers of courses offering standards assessed by external
examination explained that in the event a student is absent for the examination for an
approved reason, the derived grade supplied is based on standard-specific evidence,
generally from the school’s practice examinations. They understand that they must
use tasks that are not publicly available for these.

Tongariro School MNA Report                3
August 2014

Authenticity and breaches of the rules In one department, there have been some
issues with students submitting work partially or completely copied from the internet.
These were identified during external moderation. The TIC explained that she is now
very alert to the issue and, during the visit, strategies for preventing and detecting
plagiarism were discussed. The teacher discussed that the following actions are
being embedded:

  •   agreeing on a topic at the start of an assessment and monitoring the students
      as they complete work for this agreed topic
  •   using a local context where possible
  •   collecting in draft work at milestones so that student progress can be tracked
      and drafts compared with the finished product
  •   ensuring that students are made explicitly aware of what constitutes authentic
      work
  •   making sure that students know how to appropriately cite sources
  •   looking carefully for the student voice in completed work
  •   if there are any suspicions that work may not be authentic, using Google or
      similar to see if the work is in the public domain
  •   questioning students to provide evidence of ownership and understanding of
      the ideas expressed.
The PN has been provided with the resources used at the NZQA seminar on
authenticity earlier in the year. It is recommended that these, along with the
strategies listed above should be used promote understanding among the wider staff.

Literacy and numeracy and tracking student achievement The school tracks
student progress towards meeting the requirements of NCEA and University
Entrance. Students who are at risk of not achieving as expected are identified and
staff take responsibility both collectively and individually for supporting this target
group.

Meeting students’ assessment needs The 2012 MNA report directed the school
to develop a process for identifying and monitoring students with entitlement to
special assessment conditions (SAC). As a result, one of the Deputy Principals has
been given the responsibility for managing the process of identifying students who
may be permitted to use SACs. Currently, there is one student who has been
approved as entitled to SAC and the school has identified that other students may be
eligible. It should be a priority for the school to develop a process for gathering
evidence to support applications.

External assessment/examination centre issues There were no issues reported
by the Examination Centre Manager after the 2013 examination round. The PN
provided the necessary help and support and facilities met requirements.

Identified action
NZQA and senior management agree on the following action to improve the
management of assessment for national qualifications. Senior management
undertakes to:
  •   encourage teachers to explore appropriate ways of collecting evidence of
      achievement.

Tongariro School MNA Report                  4
August 2014

The following action is required to improve the management of assessment for
national qualifications. The school must:
  •   ensure all teachers share the understanding of what constitutes authentic work
      (CAAS Guidelines 2.6ii)
  •   develop a process for identifying and monitoring students with entitlement to
      special assessment conditions (Assessment Rules 2014, 4.4)

Moderation (CAAS Guidelines 2.6, 3v)
How effectively does school internal and external moderation assure assessment
quality?

Internal moderation processes The school makes it very clear to teachers that all
results submitted to NZQA must be subject to a robust internal moderation process.
This includes critiquing the assessment tasks prior to use and ensuring that a second
suitably qualified and experienced teacher verifies a sample of grades. Use of the
Internal Moderation Cover Sheet (IMCS) is required to record who has been
responsible for the completion of each step of the process.

In three of the departments visited, evidence was available to show that teachers
understand and generally follow this requirement. In the fourth department visited,
the teacher is fairly new to the school and, at the time of the visit, had not undertaken
the full cycle of quality assurance for any standards. The PN will continue to work
with this teacher to ensure that he fully understands the quality assurance
requirements and has appropriate support to verify grades.

Audit of internal moderation processes Under the previous PN, teachers
submitted all student evidence, accompanied by the IMCS, to the PN at the end of
the year. This process was introduced as a result of some identified lapses in
teachers meeting the moderation requirements. The practice has been reviewed with
the outcome that the newly appointed PN will visit all departments to check the
evidence of quality assurance. In 2014, it is intended that these visits be carried out
in the second half of Term 3 and in Term 4. As part of these visits, it should be
checked that the work has been critiqued and that the verifier is suitable. There
should be brief notes recording the professional discussions between the marker and
verifier and the PN needs to check that the IMCS records the completed steps. To
ensure that teachers take responsibility for quality assurance, at the end of the year,
they will attest that they have completed the expectations for all results submitted to
NZQA.

Completion of the steps outlined above will mean that the school’s senior managers
can be confident that all results reported to NZQA have been quality assured. This is
a requirement of the school’s Consent to Assess Against Standards.

Random selection The PN has agreed, that in future, all teachers will use the
school’s student management system to generate a random list of students whose
work will be submitted for external moderation. This will be done after all marking and
verification has been completed. This will ensure that the selection is undertaken by
someone other than the marker and cannot be predicted. Information about this
process should be added to the staff handbook.

External moderation The most recent external moderation round was May 2014
and the requested work for all 20 standards was submitted. The majority of the

Tongariro School MNA Report                 5
August 2014

assessment tasks were at the national standard (19), however, for one standard the
task was invalid and this must not be used again. The moderators agreed with the
majority of assessor judgements for 14 standards, for the remaining six standards
they agreed with half or less than half of the judgements.

The PN requires a response and an action plan to follow up any issues raised by the
moderator. Samples of reflective responses were sighted during the visit.

Identified action
NZQA and senior management agree on the following action to improve the school’s
internal and external moderation to assure assessment quality. Senior management
undertakes to:
  •   carry out the audit visits to departments to check the evidence that internal
      moderation has been completed
  •   require teachers to attest that all results submitted to NZQA have been quality
      assured.

Data (CAAS Guidelines 2.6, 2.7)
How effectively does the school manage and make use of assessment-related data?

Managing assessment-related data In 2013, the school reported approximately
1800 results for internally assessed standards. The school made an additional 125
entries in externally assessed standards.

In 2013, approximately 60 per cent of students registered for Learner login. About
half of the registered students accessed entries and results at least once during the
year. Having students access the data held by NZQA allows them to check the
accuracy of the data held by NZQA.

Teachers can further encourage use of the Learner login by:
  •   directing students to access their login when they want an update of their
      entries or results, instead of providing the information directly to students
  •   taking their form class to a computer room so as to foster the habit of students
      accessing their login.
Entries and results data For 2013, NZQA indicators of the accuracy of the school’s
data show the following:
  •   The school requested a very small number of supplementary examination
      papers to cover entries in external examination standards that had not been
      made by the 1 September deadline.
  •   There was an examination session absence rate of 28 per cent. The school
      recognises this as a concern that needs to be addressed, and is trying to
      encourage students to attend the external examination sessions they have
      agreed to enter.
Analysis of results data for 2013 also shows the following:
  •   Approximately 30 per cent of results reported for internally assessed standards
      involved the school’s use of an external provider. The PN holds a current
      memorandum of understanding with each provider that it engages with and a
      sample of these were sighted during the visit.

Tongariro School MNA Report                 6
August 2014

•   In 2013, a number of students were ‘non participants’ at all levels. The changes
      in course design the school has made are expected to enable more students to
      gain sufficient credits for an award or qualification.

Privacy and password security The review served to clarify that the school should
ensure that teachers register for the Education Sector Authentication and
Authorisation process. Failure to register means teachers cannot login to the school’s
NZQA web pages and they are denied direct access to a range of key indicator and
reports. Accuracy of entry and results data is more likely when teachers check the
information held by NZQA prior to the key dates.

Identified action
NZQA and senior management agree on the following action to improve the
management and use of assessment-related data. Senior management undertakes
to:
  •   encourage students to register to log in to the NZQA website to check entries
      and results
  •   ensure that more teachers register for the Education Sector Authentication and
      Authorisation process.

Communication (CAAS Guidelines 2.4i(f),2.4v, 2.6, 3v)
How effectively does the school maintain the currency of assessment policy and
procedures, and communicate them to staff, students and families?

Communication of policies and procedures to staff, students and their families
The following documents were provided for this visit:
  •   Tongariro School Senior Course Booklet
  •   Tongariro School Managing National Assessment Document (a handbook for
      teachers)
  •   NCEA Assessment information for Students
The Senior Course Booklet gives useful information about how NCEAs and
Vocational Pathways are achieved. It is a very comprehensive document and
students negotiate their course from a very wide range of choices. The individual
course pages, which have information about literacy, numeracy and vocational
pathways, also double as an assessment statement. These are issued to students at
the start of a course. It is suggested that brief supplementary information is added to
these single page hand-outs to include the following:
  •   whether a course is endorsable
  •   approximate dates of assessments
  •   whether there is a further assessment opportunity
  •   a reminder to refer to the NCEA Assessment Information for Students
  •   any details of processes particular to the course not covered in the NCEA
      Assessment information for Students.

It was suggested in the 2012 MNA review report that the information in the teacher
handbook be reviewed to ensure that it aligns with the practice in the school. While
some progress has been made, much of the material is generic rather than relating to
the school’s own context. The PN agreed that it will be a priority to critique the
Tongariro School MNA Report                7
August 2014

material. In particular, the sections relating to internal moderation, monitoring internal
moderation and follow up from external moderation will be rewritten. During the visit
the PN was provided with a checklist to assist with evaluating the documents.

Identified action
NZQA and senior management agree on the following action to maintain the
currency of assessment policy and procedures, and communicate them to staff,
students and families. Senior management undertakes to:
  •   review the Tongariro School Managing National Assessment Document to
      ensure that it aligns with the school’s expectations and practice
  •   add supplementary information to the course hand-outs for students.

Tongariro School MNA Report                 8
August 2014

Summary
Tongariro School is developing processes to ensure the credibility of assessment for
national qualifications. The Principal’s Nominee, who had only been in a role for a
few weeks at the time of the visit, has a clear understanding of the expectations and
requirements and is leading the staff with the full support of the Principal.

Teachers have developed courses that reflect student’s aspirations and are working
to assess students in a variety of ways so that their best effort can be recognised.
Better access to special assessment conditions and use could improve outcomes for
individual students.

The school should review its processes to ensure student entries are accurate and
reflect student capability and the intended assessment programme in individual
courses.

The Principal’s Nominee will review the staff documentation to ensure that it aligns
with expectation and practice; it is expected that the revised documentation will assist
all staff to understand the requirements.

The 2014 Managing National Assessment review identified significant issues that
prevent the school from meeting the requirements of the Consent to Assess Against
Standards on the Directory of Assessment Standards Rules 2011 and its Guidelines.

Action Items
In order to maintain its consent to assess, the school must:
  •   ensure teachers share the understanding of what constitutes authentic work
      (CAAS Guidelines 2.6ii)
  •   develop a process for identifying and monitoring students with entitlement to
      special assessment conditions (Assessment Rules 2014, 4.4)
  •   follow up external review recommendations and findings (CAAS Guidelines 3).

Based on the outcome of this assessment systems check, it is anticipated that the
next Managing National Assessment review will be conducted within two years.

Ian Munro
Manager
School Quality Assurance and Liaison

31 October 2014

Tongariro School MNA Report                9
August 2014