Managing National Assessment Report 2013 Matamata College

Managing National
  Assessment
    Report

      2013

Matamata College

Managing National Assessment Report

Matamata College

13 & 14 May 2013

Introduction
The purpose of the Managing National Assessment (MNA) external review is:
  •   to help schools achieve valid, fair, accurate and consistent internal assessment
      according to the requirements of the Assessment and Examination Rules for
      Schools with Consent to Assess 2013 (Assessment Rules); and
  •   in combination with the most recent Education Review Office report, to ensure
      that schools are meeting the requirements of the Consent to Assess Against
      Standards on the Directory of Assessment Standards Rules 2011 and its
      Guidelines (CAAS Guidelines) in order to maintain their consent to assess.
This process is a partnership between schools and the New Zealand Qualifications
Authority (NZQA) to ensure that assessment systems remain effective and internal
assessment decisions are valid.

The MNA review has two components:
  1. The annual external moderation of assessment materials for an NZQA-selected
     sample of internally assessed standards across curriculum areas and of
     assessor judgements for samples of student work selected according to the
     school’s random selection process.
  2. An external review of each school’s assessment systems at least once every
     four years.

This report summarises NZQA’s evaluation of how effectively:
  •   senior management has addressed issues identified during the previous MNA
      review on 17 June 2010
  •   the school manages assessment for national qualifications
  •   the school manages internal and external moderation
  •   the school manages and makes use of assessment-related data
  •   the school maintains the currency of assessment policy and procedures, and
      communicates them to staff, students and families.

In preparing this report, the School Relationship Manager (SRM) met with the
Principal’s Nominee (PN), the Deputy Principal (DP) and the Heads of Department
(HODs) for Design and Visual Communications, Mathematics, Physical Education
and Science. The school also provided the SRM with comprehensive pre-review
information and the staff and student assessment procedures guides.

There was a report-back session with the Principal, the PN and Deputy Principal (DP)
at the end of the visit to highlight strengths and weaknesses, with suggested
strategies for good practice and to agree on recommended actions.

Matamata College MNA Report               2
May 2013

2010 Managing National Assessment Report (CAAS Guidelines 3v)
Evidence was sought that action items in the previous MNA report have been
effectively addressed.

Most of the items have been fully actioned. The two items not actioned are discussed
in this report.

Assessment Practice (CAAS Guidelines 2.5v-vii, 2.6, 3v)
How does the school’s assessment practice allow for the effective management of
assessment for national qualifications?

Gathering evidence of achievement Formal, discrete assessment events are
essentially the main tool used by teachers to gather evidence of achievement.
However, examples were described of authentic evidence being gained during the
normal teaching and learning process or from activities outside the classroom. These
include:
    •   annotating student work and using milestones to identify and authenticate
        valid evidence of achievement during the teaching and learning process
    •   coordinating the submission of valid evidence across their courses as needed
        for those students being assessed against the Work and Study Skills literacy
        units standards
    •   two classes established to build mathematical skills and confidence the focus
        on programme flexibility and readiness for assessment.

Meeting students’ assessment needs The school offers a range of courses so
that a variety of contexts are available for students to learn and demonstrate
achievement. Examples of how this is achieved include:
    •   extending the school’s consent to assess or engaging outside providers for
        specialised learning and assessment in areas such as agriculture and
        horticulture, first aid, hospitality, outdoor education, health and safety and the
        service sector
    •   providing vocationally based courses through its Trades Academy and
        Gateway programmes
    •   offering stage 1 Philosophy through online learning
    •   conducting a strong academic counselling programme so that students at risk
        of not achieving their potential are mentored and monitored through an
        appropriate pathway
    •   timetabling and structuring classes so all students in need of additional
        support are catered for.

The school has made Mathematics, Science and English compulsory at Year 11 as a
strategy intended to help raise student achievement. Classes are graduated in terms
of the mix of internal and external standards and students are placed in classes to
maximise their opportunities for success. Timetabling is blocked so that movement
between classes can occur as the need arises. Students are advised and mentored
about their entries into externally assessed standards. The DP reported that these
strategies have been successful in raising the expectations and performance for
students who may have previously lacked confidence in their ability to achieve in
these subjects.
Matamata College MNA Report                  3
May 2013

Literacy and numeracy Processes are in place to ensure that students are tracked
and mentored towards meeting these requirements. Students at risk of not gaining
literacy or numeracy through the achievement standards pathway are identified and
assessed against the Work and Study Skills unit standards within their programmes
or in designated classes. In the two Year 11 classes where these unit standards are
assessed, students are able to negotiate their own dates for assessment. With
oversight from a staff member appointed and trained as numeracy coordinator, the
school is confident that this programme is successful in raising the level of
achievement for these students.

Special assessment conditions The school has established a Special
Assessment Committee to ensure the effective identification and management of
students with these conditions. Information from previous schools, testing at Years 9
and 10, level C assessor reports, on-going analysis of needs, and referrals from staff
allow management to be confident that any student with entitlement is identified and
supported. The applications for eight candidates were approved in 2012 and these
were appropriately managed so that their conditions were available for both their
internal and external practice assessments. Reader/writers are trained in a timely
manner so that conditions are provided equitably.

Derived grades In conjunction with this visit, the SRM conducted an audit of the
applications for derived grades for three candidates for the 2012 external
examinations. These applications were appropriately administered and documented
and contained derived grades for eight standards from three subject areas. The
assessment procedures of these subject areas for deriving valid grades from
standard specific-evidence were found to meet NZQA guidelines.

Examination centre The PN works closely with the Examination Centre Manager,
who was positive about the quality of assistance he provides and that the delivery of
special assessment conditions is well organised.

Identified action
No action was identified as needed to improve the effective management of
assessment for national qualifications.

Moderation (CAAS Guidelines 2.6, 3v)
How effectively does school internal and external moderation assure assessment
quality?

Internal moderation processes The SRM found that sound internal moderation
practice is well-embedded. HODs are responsible for ensuring that internal
moderation is undertaken for each standard in their courses and that this practice is
documented using an Internal Moderation Cover Sheet. Appropriately completed
examples of these were sighted during interviews and HODs described sound
processes for reviewing assessment material and for verifying a sample of grades.
The PN ensures that there is faculty oversight for those who are sole teachers of
subject and that they are maintaining professional links with external colleagues.
HODs spoke of professional engagement in the form of involvement with subject
associations and cluster groups, attending professional development workshops and
their use of NZQA resources and reference to clarification documents.

Matamata College MNA Report               4
May 2013

Monitoring of internal moderation processes As evidence that results reported to
NZQA have been quality assured, HODs are required to submit a completed copies
of the cover sheets to the PN. These documents, along with information from
discussion items at faculty and department meetings allow management to identify
and respond to any issues of non-compliance in a timely manner.

External moderation procedures On return of materials from the moderator, the
PN copies and reviews the moderation reports. HODs are required to document their
action to implement the changes required by moderators on the school’s External
Moderation Response Form. This is submitted to and checked by the PN, ensuring
that any issues identified by moderation have been addressed and there is a cycle of
on-going improvement in assessment practice.

Random selection The school has been following the previously advised random
selection process and has adapted its procedures to meet the new requirements
(SecQual S2013/005) for the 2013 moderation round. When updated for 2014,
information for staff will reflect this change.

External moderation The most recent moderation round was November 2012 and
54 standards were selected. Work for all standards assessed was submitted as
required and the majority of the assessment tasks were at the national standard. The
rate of agreement between moderator and assessors’ decisions has become
established at a consistently high level, and this provides validation that the school’s
internal moderation practice is effective.

Identified action
No action was identified to improve the school’s internal and external moderation to
assure assessment quality.

Data (CAAS Guidelines 2.6, 2.7)
How effectively does the school manage and make use of assessment-related data?

Accuracy of assessment-related data Teachers are required check the accuracy
of results entered on the student management system. For oversight, the PN and the
school’s data manager regularly check subject mark books and course and entries
information. As examples, school data for the level of results not reported, Not
Achieved results and late entries for external examinations provide evidence that
procedures for managing data are effective. The school’s provider code was used in
error to report results for several of the standards assessed by outside providers and
consequently work was unavailable for these. The PN provided an assurance that
systems have already been reviewed to rectify this issue for 2013. This will ensure
that evidence of assessment will be available for through the assessing provider.

Use of assessment-related data The Principal receives a report annually from
each department which contains an analysis of results and a review of their
assessment programme, which are used to inform practice and set goals for the
following year.

Form teachers regularly meet with students to mentor them in their progress toward
qualifications and endorsements and students use a credit tracking document which
is a useful tool assist in this process.

Matamata College MNA Report                5
May 2013

Privacy and password security The school is accurately engaging with the
Education Sector Authentication and Authorisation process so that teachers have
their own secure access to the range of NZQA generated data and statistical reports
that are available through the school’s NZQA website login.

Learner’s login In 2012, 80 per cent of the school’s senior students registered for a
Learner login so that they could track their own progress via the NZQA website. This
also serves as an additional check for the school about the timeliness and accuracy
of results reported.

Identified action
No action was identified as needed to improve the management and use of
assessment-related data.

Communication (CAAS Guidelines 2.4i, 2.6, 3v)
How effectively does the school maintain the currency of assessment policy and
procedures, and communicate them to staff, students and families?

Communication of policies and procedures to staff, students and their families
The following documents were provided for this visit:
  •   Matamata College NZQF Policies and Procedures 2013 – staff document
  •   Matamata College Policy Assessment and Learning
  •   Course assessment statements for four subject areas.

The policies and procedures document contains most of the information fundamental
to assessment for national qualification. Assessment procedures are reviewed
annually, with staff feedback being gathered through the use of a questionnaire, and
through discussion at QMS Faculty meetings. When next reviewed, the PN has
undertaken to document for staff NZQA’s revised processes for random selection,
arrangements for working with industry training organisations and guidelines for staff
about the entering and checking of results. Also, students need to be informed about
their rights under the Privacy Act 1993, the guidelines for resubmission and the
availability of special assessment conditions.

Students receive documented information through their course outlines, assessment
statements, assemblies, notices and the school’s intranet. Version control is difficult
to maintain where school-wide procedural information is available in a more than one
setting. Management have agreed that information essential for students be
documented in their own designated handbook and to monitor that this information is
removed from course outlines. It was discussed that information can be published for
staff in the same way as for students, but with a supplement containing the
information which is not needed by students, such as the school’s moderation
procedures.

From discussions with staff and students during the visit, it is clear that the school’s
procedures for managing national assessment are understood and consistently
applied, providing an equitable assessment environment for teachers to gather valid
evidence of achievement. The PN is confident that a major reason for this is that the
regularly held Quality Management System Faculty Meetings have proved to be an
effective setting for communication and discussion for HODs about matters relating to
assessment practice. The PN sets the agenda items and distributes minutes of these
meetings to assist HODs to share and disseminate the outcomes at department
Matamata College MNA Report                6
May 2013

meetings. Minutes of recent meetings were available and viewed by the SRM during
the visit.

Communication with parents and the community is by way of the school website,
parents’ evenings and school newsletters. The availability of a parent portal on the
school’s intranet allows them to securely access the results and progress information
of their dependant.

Assessment statements There is a variety of formats used to present information
about standard be assessed. The school has agreed that the use of a common
template for this will assist management to monitor assessment programmes, provide
consistency of information across all subjects and assist students to accurately
monitor their progress and results. These statements should include all information
about standards to be assessed: the standard number, version, title, credit value,
internal or external, the mode and timing of assessment and the availability further
assessment. The school is aware of the need to monitor this. These statements can
also be used to indicate those standards tagged for numeracy, literacy and course
endorsement and as counting as university entrance approved subjects.

Identified actions
NZQA and senior management agree on the following actions to maintain the
currency of assessment policy and procedures, and communicate them to staff,
students and families. Senior management undertakes to:
  •   ensure that information which is essential to assessment for national
      qualifications is published as appropriate for staff and for students as stated on
      page 6 of this report
  •   develop a designated handbook for students
  •   monitor that school-wide assessment procedure information is removed from
      course outlines
  •   develop and monitor the use of a common template for course assessment
      statements.

Matamata College MNA Report                 7
May 2013

Summary
Matamata College is demonstrating on-going commitment to ensuring the credibility
of assessment for national qualifications.

Internal and external moderation is effectively managed and the school only reports
results for internal assessment that are subject to the school’s internal moderation
procedure. Heads of Faculty oversee internal moderation within their Faculty and
provide an assurance to the PN of its completion.

Teachers have a good understanding of the school’s assessment procedures and
practice is consistent across the school. They are collaborative and seek professional
engagement to develop their understanding of assessment and subject leaders
engage in reflective practice to inform any initiative or goal.

The Principal’s Nominee is experienced, effective and accessible in the role and has
the support and confidence of teachers.

The 2013 Managing National Assessment review identified no significant issues that
prevent the school from meeting the requirements of the Consent to Assess Against
Standards on the Directory of Assessment Standards Rules 2011 and its Guidelines.

Based on the outcome of this assessment systems check, it is anticipated that the
next Managing National Assessment review will be conducted within three years.

Matamata College MNA Report               8
May 2013