2. June 2013

3. Workshop overview
The quality of learning –
what do we assess ?
Assessment is ongoing –
when do we assess ?
Approaches to assessment –
how do we assess ?

4. What is Validation and Moderation
Validation is a quality review process of assessment.
• producing valid, reliable, sufficient, current and authentic evidence
• enabling reasonable judgements to be made
• reviewing and making recommendations for future improvements
Moderation is the process of bringing assessment judgements and
standards into alignment.
• ensuring the same standards are applied to judge the quality of
performance (i.e. the assessment decision making rules)
• including administration, recording and reporting requirements
• it is an active process
Validation versus moderation
• have been used interchangeably in the VET sector; and whilst each are
based on similar processes, there are a number of distinctive features.

5. Features
Validation
Moderation
Assessment Quality Quality Review
Management Type
Quality Control
Primary Purpose
Continuous improvement
Bring judgements and standards into
alignment.
Timing
On-going
Prior to the finalisation of candidate
results
Focus
Assessment Tools and
Candidate Evidence (including
assessor judgements)
(desirable only)
Assessment tools and Candidate
Evidence, including assessor
judgements (mandatory)
Type of
Approaches
Assessor Partnerships
Consensus Meetings
External (validators or panels)
Consensus Meetings
External (moderators or panels)
Statistical
Outcomes
Recommendations for future
improvements
Recommendations for future
Improvements and adjustments to
assessor judgements (if required)

6. What is Assessment?
• purposeful process of systematically gathering, interpreting,
recording and communicating to stakeholders, information on
student performance.
• assessment for learning occurs when teachers use inferences
about student progress to inform their teaching (formative)
• assessment as learning occurs when students reflect on and
monitor their progress to inform their future learning goals
(formative)
• assessment of learning occurs when teachers use evidence of
student learning to make judgements on student achievement
against goals and standards (summative)

7. Why participate in
validation and moderation processes?
• to ensure assessments align with current industry practice
and benchmarks remain within acceptable limits
• to ensure the quality and consistency of assessment
• confirms the consistency of assessor/s judgement
• to gain a comprehensive understanding of the depth of
students’ knowledge and skills
• VET Quality Framework compliance

9. Outcomes of validation
Recommendations for future improvements:
• Review of context and conditions for the assessment
• Task/s to be administered to the candidates
• Administration instructions
• Criteria used for judging the quality of performance (e.g. the
decision making rules, evidence requirements etc.)
• Guidelines for making reasonable adjustments to ensure that
the expected standard of performance specified within the
Unit(s) of Competency has not been altered
• Recording and reporting requirements

10. Outcomes of moderation
• recommendations for future improvement and adjustments
to assessor judgements (if required)
• recommendations for improvement to the assessment tools
• adjusting the results of a specific cohort of candidates prior to
the finalisation of results and
• requesting copies of final candidate assessment results in
accordance with recommended actions.

12. Approaches to Validate and
Moderate Assessments
Statistical
Requires some form of common assessment
Strength
strongest form of quality control
Weakness
lacks face validity, may have limited content validity
External
Site visits
Strength
‘expert’ interpretations of standards/ advice to RTO/
assessors on assessment approaches and procedures /
observe actual assessment processes in real time
Weakness
can be expensive and there is less control than a statistical
approach

13. Approaches to Validate and
Moderate Assessments
Partnerships
Sharing, discussing and/or reviewing
one another’s tools and/or judgements
Strength
Low costs, personally empowering, non-threatening , easily
organised
Weakness
Potential to reinforce misconceptions and mistakes
Consensus
Reviewing own & colleagues assessment tools
and judgements as a group
Strength
professional development, networking, promotes collegiality
and sharing
Weakness
less quality control than external and statistical approaches as
they can also be influenced by local values and expectations and
requires a culture of sharing

15. Systematic Validation
Indicators
is there a plan for assessment validation (including validation of RPL assessment) in place?
•
•
•
•
•
are units of competency to be validated over a set period of time
provide dates for proposed validation activities
include details about who will participate in assessment validation
include a strategy to ensure that all relevant staff are involved
identify what processes and materials for implementing and recording the outcomes of
assessment validation
does the RTO have validation materials (policy, procedure, forms) in place that cause
participants to engage effectively in validation?
does the RTO have a process for monitoring the action taken as a result of validation?
does the RTO have a process and plan in place for reviewing the effectiveness of assessment
validation?

16. Assessment Quality Management
Quality Assurance
(Input approach)
• Industry competency
standards as the
benchmarks of varying
levels of performances
• National assessment
principles
• Minimum qualifications for
assessors
• Standardisation of
reporting formats
• Assessment tool banks
• Common assessment tasks
• Exemplar assessment tools
• Professional development
programs/workshops for
assessors
Quality Control
(Outcome approach)
Quality Review
(Retrospective approach)
Examples include:
Examples Include:
Moderation in which
adjustments to assessor
judgements are made to
overcome differences in
the difficulty of the
assessment tool and/or
severity of the judgement.
• Monitoring and auditing
of organisation
• Review and validation of
assessment tools,
processes and outcomes
to identify future
improvements.
• Follow-up surveys with
key stakeholders (e.g.,
student destination
surveys, employer
feedback on how well
the assessment
outcomes predicted
workplace performance).

17. Key Stages in developing
assessment tools
•
identify and describe the purposes for the assessment
•
identify the assessment information that can be used as evidence
of competence/learning
•
identify a range of possible methods that might be used to collect
assessment information
•
define the contexts for interpreting assessment information in
ways that are meaningful for both assessor and candidate
•
determine the decision making rules
•
define procedures for coding and recording assessment
information
•
identify stakeholders in the assessment and define their reporting
needs.

18. Essential Characteristics –
Assessment Tool
An assessment tool includes the following components:
•
•
•
•
the context and conditions for the assessment
the tasks to be administered to the candidate
an outline of the evidence to be gathered from the candidate
the evidence criteria used to judge the quality of performance
(i.e., the assessment decision making rules); as well as the
• the administration, recording and reporting requirements.

19. Ideal Characteristics
•
•
•
•
•
•
•
•
•
•
•
•
•
the context
competency mapping
the information to be provided to the candidate
the evidence to be collected from the candidate
decision making rules
range and conditions
materials/resources required
assessor intervention
reasonable adjustments
validity evidence
reliability evidence
recording requirements
reporting requirements

20. Where do you start the process?

21. Training and Assessment Strategy
Outlines the overall validation and moderation
processes that will be used for the qualification.
Includes a description of the types of
documentation that will be kept as evidence.
Validation and Moderation Schedule
Provides a schedule and description of
processes for when the various validation and
moderation activities will occur.
• Industry consultation of the TAS
• Industry consultation of assessment tools
• Validation activities with other assessors
• Moderation activities with other assessors
• Internal moderation of judgements
• Internal review & Internal audit
Internal Audit Report
Includes review of the validation processes and
the application of these with documented
evidence and the Internal Review Report.
The Review & Audit process lead into continuous
improvement in relation to the TAS and other
documentation and processes. All documentation
generated from the Validation & Moderation
process leads into the Internal Audit Process.
Internal Review Report
Includes review of the validation
processes and the application of
these with the documented evidence
provided (described below).
Record of Industry Consultation
• Industry consultation of TAS
• Industry consultation of
assessment tools
Record of Trainer/Assessor
Validation & Moderation
• Validation and Moderation
activities with other assessors
Provides documented evidence of
validation activities with relevant
industry stakeholders. This needs
to be signed by the industry
stakeholder involved in the process.
Provides documented evidence of
validation and moderation activities
with assessors not involved in the
development of the assessment
tools.
Record of Internal Moderation
• Internal moderation of
judgements
Provides documented evidence of
internal moderation where more
than one assessor is involved in
making judgements for students
enrolled in the qualification.

22. Industry
Validation of TAS

23. Who are my students?
What does competent look like in the current workplace?
How will I know someone is competent?
What evidence do I need to collect to confidently judge
someone as competent?
What methods will I use to collect that evidence?
What assessment tools will I need to collect that evidence?
Will I have enough evidence?

24. Competency Mapping
Step 1:
Step 2:
Step 3:
Unpack the unit of competency to
identify its critical components.
For each assessment method, list the
tasks to be performed by the candidate.
For each assessment method, map the
critical components of the unit to each
assessment task.

25. Level of specificity in mapping
– Risk Assessment
Risk can be determined by consideration of:
• safety (e.g. potential danger to clients from an incorrect
judgement)
• purpose and use of the outcomes (e.g. selection purposes)
• human capacity (e.g. level of expertise and experience of the
assessors)
• contextual (e.g. changes in technology, workplace processes,
legislation, licensing requirements and/or training packages)

26. Decision Making Rules
The rules to be used to:
• check the quality of the evidence (i.e. the rules of
evidence)
• judge how well the candidate performed on the task
according to the standard expected
• interpret evidence from multiple sources to make an
overall judgement

27. Reasonable Adjustments
• this section of the assessment tool should describe the
guidelines for making reasonable adjustments to the way in
which evidence of performance is gathered without altering
the expected performance standards
• in some cases the assessment tasks will need to be adjusted
or alternative tasks developed. A simple example would be
responding to questions orally rather than written.
• however, remember that the adjustments must not alter the
expected standard of performance specified within the unit/s
of competency.

28. Simulated assessment
• for the purposes of assessment, a simulated workplace is one
in which all of the required skills are performed with respect
to the provision of paid services to an employer or the public
can be demonstrated as though the business was actually
operating.
• in order to be valid and reliable, the simulation must closely
resemble what occurs in a real work environment.
• the simulated workplace should involve a range of activities
that reflect real work experience. it should allow the
performance of all of the required skills and demonstration of
the required knowledge.

29. Activities
• In your groups discuss what input employers could provide to
develop valid assessment tools and processes to employ a
person with a Cert IV in Business .
• Note down 2/3 questions you could ask employers and how
the responses will inform the development or review of
assessment tools and/or processes.

30. Questions

31. References
•
•
•
•
•
•
MAXWELL, G S 2001, Moderation of Assessment in Vocational Education and Training, University of Queensland, Australia.
(Wheeler, 1993).
www.nqc.tvetaustralia.com.au
http://policy.ballarat.edu.au/tafe/validation/
http://www.nssc.natese.gov.au/nqc_archive/nqc_publications/publications/assessment
http://www.velgtraining.com/

32. Thank you
Cecilia Sorensen
Project Manager | Sarina Russo Institute
P: 07 3308 2200
F: 07 3221 5304
E: CSorensen@sri.edu.au
W: www.sri.edu.au