Why do evaluation and assessment matter? Highest rated education policy priority among OECD countries Analysing strengths and weaknesses of the education system is key to defining strategies for further development Need to provide information about education quality to parents and society at large Decentralisation and school autonomy are creating a greater need for evaluation and assessment To ensure that schools get the information and feedback they need to improve their work To allow the government to monitor if national goals for quality and equity in education are achieved

1. OECD REVIEW ON EVALUATION AND
ASSESSMENT FRAMEWORKS FOR
IMPROVING SCHOOL OUTCOMES
Deborah Nusche, Analyst
OECD Directorate for Education
Bundesministerium für Unterricht, Kunst und Kultur
Vienna, 4 December 2012

2. Key topics for this presentation
• Why do evaluation and assessment matter?
• The OECD project
• Key findings from the Review

3. Why do evaluation and assessment matter?
• Highest rated education policy priority among OECD countries
• Analysing strengths and weaknesses of the education system is key to
defining strategies for further development
• Need to provide information about education quality to parents and
society at large
• Decentralisation and school autonomy are creating a greater need for
evaluation and assessment
– To ensure that schools get the information and feedback they need to
improve their work
– To allow the government to monitor if national goals for quality and
equity in education are achieved

4. OECD Review on Evaluation and Assessment
Frameworks for Improving School Outcomes
• Purpose: To explore how systems of evaluation and assessment can be
used to improve the quality, equity and efficiency of school education.
• Focus: A Review of national approaches to evaluation and assessment in
school education (primary and secondary schools)
• Comprehensive approach: The Review looks at the various components of
assessment and evaluation such as:
– Student assessment;
– Teacher appraisal;
– School evaluation;
– Education system evaluation.

5. OECD Review on Evaluation and Assessment
Frameworks: Overview
Evaluation and assessment framework
Student assessment
Classroom
Teacher appraisal
Improvement of
teaching and
School
learning
outcomes
School evaluation
System
System evaluation

6. OECD Review on Evaluation and Assessment
Frameworks: Methodology
Analytical strand
– Reviewing the literature and evidence on the impact of evaluation and assessment
procedures
– Gathering data on countries’ policies and practices
Country Review strand
– Country Reviews provide specific advice to individual countries.
– OECD-led Review Team
– The scope and focus is determined by the country in consultation with the Secretariat
Synthesis report
– Comparative report to analyse policy options and highlight good practices across
countries.
Spotlight reports
– Short reports providing a customised summary of the synthesis report putting the
spotlight on a particular country

7. OECD Review on Evaluation and Assessment
Frameworks: Participation
• 28 systems are providing detailed data on their evaluation and
assessment practices
• 26 systems are preparing a comprehensive Country Background Report
• 14 systems (in green) opted for a Country Review
• 6 systems (in orange) opted for a Spotlight Report
Australia, Austria, Belgium (Flemish Community), Belgium (French
Community), Canada, Chile, Czech Republic, Denmark, Finland,
France, Hungary, Iceland, Ireland, Italy, Korea, Luxembourg,
Mexico, Netherlands, New Zealand, Norway, Poland, Portugal,
Slovak Republic, Slovenia, Sweden and United Kingdom (Northern
Ireland).

8. OECD Review on Evaluation and Assessment
Frameworks: Outputs
• Background papers on issues such as formative
assessment, use of test results, teacher
appraisal, school evaluation etc. (11)
• Country Background Reports (26)
• OECD Publication series: OECD Reviews of
Evaluation and Assessment in Education
(14 Country Review Reports)
• Final Synthesis Report (early 2013)
All outputs available at:
www.oecd.org/edu/evaluationpolicy

9. OECD Review on Evaluation and Assessment
Frameworks: Key topics
1. Governance: Striking the right balance between central efforts and local
initiative and for improvement
2. Procedures: Designing the right instruments to ensure evaluation and
assessment contribute to improvement of teaching and learning
3. Capacity: Developing competencies for evaluation and for using feedback
at all levels of the education system
4. Use of results: Organising evaluative information in such a way that it
facilitates effective use by stakeholders; avoiding ‘misuse’ of E&A results

10. OECD Review on Evaluation and Assessment
Frameworks: Governance
Achieving the right balance between central and local initiatives
• Decentralisation and school autonomy are widespread across OECD countries
• In about two-thirds of OECD countries, most schooling decisions in lower
secondary education are taken at the local or school level
• At the same time: Central authorities remain responsible for ensuring quality
and equity of education nationally
• Strong accountability mechanisms are often seen as a counterbalance to
school autonomy

11. OECD Review on Evaluation and Assessment
Frameworks: Governance
Chart D6.1 Percentage of decisions taken at each level of government in public lower secondary education
(2011)
School Local
100
90
80
70
60
50
40
30
20
10
0
Australia
Turkey
Scotland
Germany
Italy
Austria
England
Spain
Estonia
Iceland
Greece
Hungary
Portugal
Indonesia
Poland
Slovenia
Chile
Israel
France
Japan
Canada
Ireland
Belgium (Fl.)
Denmark
Korea
United States
Sweden
Switzerland
Luxembourg
Mexico
Netherlands
Belgium (Fr.)
Norway
Slovak Republic
OECD average
Czech Republic
Countries are ranked in descending order of the percentage of decisions taken at the school level.
Source: OECD, Education at a Glance 2012. Table D6.1 See Annex 3 for notes (www.oecd.org/edu/eag2012).

12. OECD Review on Evaluation and Assessment
Frameworks: Governance
Advantages of a decentralised approach:
• Evaluation and assessment will only lead to improvement if stakeholders are fully
engaged and use results / feedback to adjust their practices
• Ownership and responsibility for evaluation and assessment can generate
commitment, professionalism and dynamism.
• Diversity of approaches allows for innovation.
Challenges of a decentralised approach
• Not all sub-national authorities and schools are systematic in their evaluation activities
 Concerns about those regions & schools that have less capacity or commitment for
implementing quality management approaches.
• School practices in student assessment, teacher appraisal and school self-evaluation
are variable and often depend on the leadership of individual principals and teachers.
 Concerns about equity and equivalence of education across the country
• The potential for system learning is not fully exploited.

13. OECD Review on Evaluation and Assessment
Frameworks: Governance
Some suggestions for policy development
• Develop central reference points for common orientation
– Student assessment: Standards, learning progressions, assessment criteria and exemplars to
support fair and reliable assessment of students across schools
– Teacher appraisal: A professional profile or standards for the teaching profession as a basis
for initial teacher education, teacher appraisal and professional development
– School evaluation: Shared quality indicators to guide both school self-evaluation and
external evaluations
• Map out existing tools to support student assessment, teacher appraisal and
school evaluation
• Clarify responsibilities for different aspects of evaluation and assessment
• Follow up with competency descriptions
• Develop adequate professional development offers

14. OECD Review on Evaluation and Assessment
Frameworks: Procedures
Using the right procedures for evaluation and assessment
• Research shows that evaluation and assessment can have a positive or
negative impact on teaching and learning, depending on how approaches are
designed and implemented
• Key question: How to design evaluation and assessment in a way that
optimises potential positive effects and minimise potential negative effects?
Some challenges
• Aspects assessed: How to avoid signalling narrow expectations?
• Instruments used: How to find the right mix of instruments to give attention
to all valued aspects of performance ? How to optimise use of ICT?
• Adaptations to context: How to ensure equity and fairness in E&A?

15. OECD Review on Evaluation and Assessment
Frameworks: Procedures
Testing formats used in national assessments, 2012
Closed-format
Open-ended writing Performing Oral questions and
Multiple choice short answer Oral presentation
tasks/calculations a task answers
questions
Austria Yes Yes Yes No Yes Yes
Australia Yes Yes Yes No No No
Belgium (Fr.) Yes Yes Yes Yes No No
Belgium (Fl.) Yes Possible Possible Possible No No
Czech Republic Yes Yes No No No No
Denmark Yes Yes No No No No
Finland Yes Yes No No No No
France Yes Yes No No No No
Hungary Yes No No Vocational only Vocational only No
Iceland Yes No ISCED 2 only ISCED 1 only ISCED 2 only No
Ireland Yes Yes No No No No
Israel Yes Yes Yes No No No
Italy Yes Yes No No No No
Mexico (sample) Yes Yes Yes No No No
Mexico (full) Yes No No No No No
Netherlands Yes No Yes No No No
New Zealand Yes Yes Yes Yes Yes No
Norway Yes Yes No No No No
Poland Yes No Yes No No No
Slovenia Yes No Yes No No No
Sweden Yes Yes Yes ISCED 1 only Yes ISCED 3 only

16. OECD Review on Evaluation and Assessment
Frameworks: Procedures
Some suggestions for policy development
• Ensure that all important aspects of teaching and learning are given some
attention in evaluation and assessment (beyond basic knowledge and skills)
• Draw on a comprehensive set of evaluation and assessment methodologies
(observation, performance-tasks, dialogue, portfolios)
• Use multiple evaluators and instruments, both internal and external to the
school
• Explore the potential of ICT to facilitate innovative evaluation and
assessment methods
• Ensure that the context is considered in implementing evaluation and
assessment procedures

17. OECD Review on Evaluation and Assessment
Frameworks: Capacity
How to ensure that stakeholders have the right competencies to evaluate and
use evaluation results?
• Increasing focus on the role of the learner in assessment
– Need to build student competencies to become active participants in their own assessment
– Teachers themselves need to develop skills for learner-centred teaching and assessment
• Countries report gaps in teacher skills regarding assessment
– Attention to assessment varies across initial teacher education institutions
– Approaches to professional development in assessment are often fragmented
• School leaders may not be well prepared for leading evaluation and
assessment in their schools
– In many countries they have traditionally more administrative roles
– Lack of training regarding analysis of assessment data, teacher appraisal and school self-
evaluation

18. OECD Review on Evaluation and Assessment
Frameworks: Capacity
Percentage of teachers in lower secondary education who have a ‘high level of need’ of
professional development in student assessment practices
50
45
40
35
30
25
20
15
10
5
0
Mexico
Ireland
Denmark
Bulgaria
Italy
Belgium (Fl.)
Australia
TALIS Average
Poland
Norway
Hungary
Korea
Malaysia
Brazil
Turkey
Iceland
Austria
Malta
Spain
Slovenia
Lithuania
Slovak Republic
Portugal
Source: OECD TALIS Database Estonia

19. OECD Review on Evaluation and Assessment
Frameworks: Capacity
Some suggestions for policy development
• Build student capacity for self-assessment
– Students will only develop their assessment capacity if teachers themselves have such capacity and are
adequately supported
• Build teacher capacity for student assessment
– Priority areas may include: assessment of key competencies; reliable grading; effective formative
assessment, inclusive and fair assessment of diverse students; use of national test results
– Adopt a strategic approach to teacher learning in assessment throughout initial teacher education and
professional development
• Build the capacity of school leaders for teacher appraisal and school self-evaluation
– In particular the capacity to evaluate, coach and guide staff; use of data for improvement.
• Build the capacity of sub-national authorities for school and local system evaluation
– In particular the capacity to understand and make decisions based on data; to provide constructive
feedback to schools
• Build central expertise for system evaluation

20. OECD Review on Evaluation and Assessment
Frameworks: Use of results
• Evaluation and assessment are typically used for both improvement and
accountability purposes
• But countries vary in the degree to which they emphasise one or the other
– Countries with a strong focus on improvement typically emphasise: formative, low-
stakes assessment of students; teacher appraisal that is linked to decisions regarding
teacher professional development and learning opportunities; school self-evaluation
and external support for organisational learning.
– Countries with a strong focus on accountability typically emphasise: high-stakes
standardised assessment of students; teacher appraisal that is linked to decisions
regarding career advancement, salary, promotion and dismissal; external reviews or
inspections of school quality; publication of school evaluation results and / or public
comparisons of school performance.
• Where the accountability function overshadows the improvement function, this
may lead to strategic responses (focussing only on the areas that are assessed)

21. 20
30
50
60
70
90
10
40
80
0
%
United States
United Kingdom
New Zealand
Netherlands
Slovak Republic
Sweden
Source: OECD PISA Database .
Norway
Canada
Poland
Turkey
Australia
Denmark
Luxembourg
OECD average
achievement data was posted publicly (2009)
Slovenia
Chile
Mexico
Frameworks: Use of results
Hungary
Korea
Estonia
Greece
Czech Republic
Italy
Portugal
Israel
Iceland
Ireland
OECD Review on Evaluation and Assessment
Germany
Percentage of 15-year-old students in schools where the principal reported that student
Spain
Austria
Japan
Switzerland
Finland
Belgium

22. OECD Review on Evaluation and Assessment
Frameworks: Use of results
Some suggestions for policy development
• Ensure the results of evaluation and assessment are used to make further
improvements to teaching and learning – this should be the key purpose of
all evaluation and assessment activities
• Link evaluation and assessment activities to professional learning and school
development
• Use multiple sources of evidence when using results for summative /
accountability purposes
• Consider the potential unintended effects when establishing high stakes
accountability systems
• Ensure that results are used in line with the original purpose of each
instruments. Avoid misuse of results.

23. VIELEN DANK!
MEHR INFORMATIONEN
WWW.OECD.ORG/EDU/EVALUATIONPOLICY