Physikos 4: J Wright and P Cliffe

1. Paul Cliffe and John Wright
john.wright@physics.org

2.  Taken from NZQA website
 “Only those students who meet the criteria for
Merit or Excellence Level get those grades.
Therefore proportions gaining Merit and
Excellence grades change a little from year to
year.”
 “This sets NCEA apart from qualifications
systems that use a predetermined distribution of
grades – where a limited number of high grades
is allocated to candidates who do better than the
rest, regardless of the standard of their work.”

3.  “Regardless of whether an achievement
standard is assessed internally or externally,
the criteria for Achievement with Merit and
Achievement with Excellence are set down in
the standard.”
 “Only those students who meet the criteria
for Merit or Excellence get those grades.
There is no predetermined distribution of
grades.”

4. Achievement Achievement with Merit Achievement with
Excellence
Identify or describe Give descriptions or Give concise
aspects of explanations in terms explanations that
phenomena, concepts of phenomena, show clear
or principles. concepts, principles understanding, in
and/or relationships. terms of phenomena,
concepts, principles
and/or relationships.
Solve straightforward Solve problems. Solve complex
problems. problems.

5.  Real standard
◦ Finished task assessment
 Virtual standard
◦ Developmental assessment
◦ No difference from a syllabus

6.  Enables recognition of a task
 Qualification is independently recognised
 Examples
◦ Driving licence
◦ Firearms licence
◦ Radio licence
◦ Athletic standards, cross country times
 All finished task assessment

7.  Attempts to show development in a process
 Has no direct employment value
 Examples
◦ Level 1 physics
◦ What job could you get with Level 1 Physics?
◦ Level 2 physics
◦ Level 3 physics
◦ Physics degree (if it were NCEA)
 What does achieving mean in physics?

8.  Standards
◦ Assess students by standards
◦ Graded against standards
 Process (percentage)
◦ Assess students on how much they get right
◦ Graded by norm referencing
◦ Cohort is the standard

9.  Standards assessment
◦ Arbitrary since the standards have no real meaning
◦ Standard does not set the difficulty, question does
 Process (percentage) assessment
◦ Gives information on how much a student knows
◦ Can bench mark using norm referencing

11.  Answer from NZQA
 “All national exam systems have tools to
manage variation, ensuring consistency.”
 “PEPs are a tool for managing variation.”
 “PEPs consist of historical information and
results within standards and subjects.”
 “Statistically, for cohorts of a reasonable size,
candidate performance does not vary
significantly from year to year.”
 So why choose standards based assessment?

12.  “Examiners cannot tell in advance how well a
cohort will do, on untrialled questions.” Elley
 “NZQA had no mechanism in place to prevent
year-to-year and subject-to-subject variation
and seemed not to believe that it was
necessary to have such a mechanism.” Nash
 “A norm-referenced reality always underpins
criterion referenced standards in the
educational system.” Nash

13.  “NCEA‟s supporters, including NZQA, the Ministry
of Education, and the PPTA, are reluctant to
concede that that re-marking to maintain expected
„profiles‟ introduces a form of norm-referencing
and, being concerned to defend the examination
against criticism that threatens to undermine
public confidence in it, have no interest in debating
the point.” Nash
 “Remarking is not scaling, but the technique is
used to generate a distribution of grades more or
less consistent between subjects and years, and is
thus simply an expensive way of accomplishing the
same outcome.” Nash

19.  Answer from NZQA
 “Markers must follow the assessment schedule (mark
scheme) that has been finalised by the panel leader.“
 “This occurs after a substantial sample of answer
booklets have been marked by the panel leader and a
senior marker, giving an idea of the spread of
performance to be expected (referred to as setting the
standard).”
 Why is this important in standards based assessment?
 “The published assessment schedules are the ones
finalised by the panel leaders and applied by markers.”
 “Teachers use these to identify the answers
allowable for that particular exam.”

21.  Has NCEA ever been assessment based?
◦ PEPs
◦ GSM
 Are internals the same difficulty as externals?
 How does NCEA compare with Cambridge and
IB?
 Why are schools implementing the above
programs?
 Have universities noticed an improvement in
pupils since the introduction of NCEA?

22.  No real standards exist for physics
 Standards are used as a syllabus
 NCEA Ptolemaic
 Standards lead to the “right” answer (solve problems, no
method marks)
 Right answer encourages spoon feeding
 Mark schemes procrustean
 NCEA via PEPs leads to arbitrary grades
 Arbitrary results demotivates able pupils
 Low level of achievement hides lack of skills and
knowledge at bottom (about 16% to pass in maths)
 NCEA gets pupils to university, does it keep them there?
 NCEA pupils disadvantaged in competitive courses
 NZ university entry very low compared to rest of world
(120 points; 4 AS levels at D grade)

23.  A Change of Direction for NCEA: On Re-marking, Scaling and
Norm-referencing Roy Nash
 Fixing the NCEA: Ongoing problems, current reforms and
proposed changes
http://www.maxim.org.nz/files/pdf/policy_paper_ncea_refor
ms.pdf