COMPASS is an online literacy and numeracy assessment tool designed by the Australian Council for Educational Research for ICAN www.ican.sa.edu.au. This presentation provides background information about COMPASS to community based teachers and support workers.
2. Compass
To design a literacy and numeracy tool that can
measure a student’s level of learning using
evidence based assessment data.
Compass is not an assessment of student
achievement in learning but rather an
assessment based on student’s ability to
reason.
3. Why Compass?
• The Compass project delivers online
assessment in literacy and numeracy for ICAN
students
• The assessment items are designed to
maximise student engagement
• The tasks are year level appropriate
• The stimuli are age appropriate
4. Understanding COMPASS
What capable
of achieving
Potential to
learning
ATTAINMENT
What learnt
What know
What can do
Schooling & Community
•Knowledge of Literacy & Numeracy
•Tested in variety of means
•Assess what has been learnt
Community
•Knowledge of Maths & Reading
•Personal experiences
•Indicator of general intelligence
Test student ability to perceive
relationships and then work out
the co-relationships without
acquiring knowledge of
language & mathematics.
Visuals & texts- culturally fair
Students general ability to
learn skills needed to attain.
Learning is developmental so
linked by scaffolds to
attainment
ACSG – Strategies that can be used to link to formalised learning.
5. Reasoning Test
Test students ability to perceive relationships and
then work out co-relationships without acquiring
knowledge of language & mathematics. Use
diagrams, symbols, shapes , words ,numbers &
experiential stories. Multiple choice questions
start from what is logical. The visual
questions/statements are independent of
language & mathematical ability and are
therefore an accurate indicator of a students
general intellectual ability. They are also culturally
fair.
6. Multiple Choice
• Capture a range of most likely answers
• May use a statement or question
• Can be quickly marked coded
• Could be testing specific features or most likely
using a range of multiple prompts
• Pilot testing ensured that students would
interpret answers in the same context
• Aim to capture different levels of student thinking
and knowledge.
7. Compass Features
• The assessment targets three levels of ability –
lower primary, middle primary and lower
secondary
• Compass data will support decisions about a
student’s ability level
• The programme will assist in identifying
suitable interventions for students
8. Characteristics of the Assessment
Accessible to all students
Culturally sensitive but links with learning or earning
Managed by the student
Accommodations don’t effect outcomes of assessment
User friendly
Addresses student needs
Is socially inclusive and outcomes based
Provides for personalised learning
9. COMPASS PROMPTS
–Designed to engage students through their topics of interest
–Considers the complexities of English language
–Promotes critical thinking through engaging questions that are designed to
enable students to analyse or review.
–Uses age appropriate reading and numeracy questions & visuals
–The strength of the assessment format is in the quality of each item being
assessed.
–Distracters in the multiple choice format can be very close to the key
answer
–Promotes self responsibility for learning through multiple log ins
10. Strengths of the Compass DESIGN
Pre-entry booklet to determine a starting point.
Culturally inclusive topics that link life experiences to cognitive outcomes
Visually engaging
Requires minimal computing skills
Online, multiple choice features.
Multiple log ins and no time barriers
Can be administered by non teachers
11. Strengths of the Compass - COGNITIVE
Referenced using young people’s learning preferences
Addresses access barriers to learning
Promotes interest for sustained learning
Uses culturally engaging language features
3 tiered entry points aimed at lower primary / middle-upper primary /
secondary
Student learning can be aligned to Australian Core Skills Framework &
measured developmentally
12. Strengths of Compass - REPORTING
Developed using strength based Framework
Discourse links to NAPLAN discourse
Connects student to mainstream learning
Helps a case worker to understand a student’s learning needs
Assessment outcomes link to learning & earning pathways
Provides for more appropriate intervention planning
13. Literacy & Numeracy
Three Numeracy content strands:
• Number and algebra
• Statistics and probability
• Measurement and geometry
Literacy
• Reading
14. 1. Can you tell me about the questions you
have just answered?
2. Did the pictures get you interested?
3. Take me through your answer and why you
thought of the answer you gave?
24. 24
COMPASS Trial Feedback
Too much language
Liked the visuals
FunEngaging
Sometimes difficult
Can we use alongside
other assessments
Great way to do
assessments
27. Complexities of Test Development
• Possibilities - diverse
• Distracters - specific
• Collaborative
• Process of development
• Genre specific
• 4Rs – Realistic, Relevant, Rational,
• Cognitive nature of students – student involvement was
paramount for developing ideas.
28. Sensitivity & Social Balance
• Inclusively Represented
• Contextually considered
29. Moving Forward
• Learning in Community
• Aboriginal Learning Framework
• ESL Methodology
• Australian Curriculum
• Outcomes Frameworks
• SACE Board – TAFE – University Pathways
Instructional Supports & Student Wellbeing
30. Aboriginal Learning Strategy
Framework of Transformative
Practice
Student driven
Cultural outcomes
Pedagogy
Standards
COMPASS Framework
Student Managed
Culturally designed
Lit/Num Interventions
Standards-student learning
pathways
How can COMPASS improve learning outcomes for aboriginal students?
Students can succeed and see themselves as
learners
34. For more information on COMPASS
http://www.acer.edu.au/tests/compass
www.ican.sa.edu.au
Notas do Editor
Important Characteristics of ICANTargets 3 levels but suitable for 12-19 year olds.Uses a bandwidth that is accessible for small groups or individual studentsUses assessment materials that are relevant for personalised and community based learnersCan be used to measure growth in learning – approx 9 months gap between assessmentsLinked to the Australian Core Skills Framework.
Probability: the feedback from the research showed that this particular item was quite popular. It deals with the relatively difficult notion of probability in an engaging way.
Baby: This is an example of an item where all the information required to answer the question is contained in the prompt. It is not assumed that the participants know how to convert from grams to kilos. It also reflects how Compass has been developed to suit the target audience, as we discovered in the cognitive labs that teenage mothers often form a part of the group of disengaged students.
Kinglake: this is an example of a literacy item where the participants are expected to engage with the character in the prompt, and they will need to ‘put themselves in the narrator’s shoes’ in order to answer the question. This proved relatively challenging for the participants.
Newspapers: this item is another example of all the information required answer the question is contained in the prompt. It is also an item where the notion of mathematical formulas are introduced gently.
Discuss 2010 strategies of above.ACER – May not be appropriate for students with very low literacy skills.
Processes that ACER went through: EXAMPLE BEST PRACTICEPossibilities - Brainstorming of possible itemsDistracters needed to address different levels of inferencingCollaborative - Work across ACER unitsDevelopment process: Draft / redraft / panel / repanel / refine / trial / match to data / identify errors / select /cog labs/ trial / analyse against dataGenre Specific; Focus on problem scenarios, explanations, instructions & background texts4 R’s –relalistic & appropriate for year level & target group, reading/maths meaningful,Cognitive nature of students – low reading load in Maths & targeted reading load in Reading.
Representations – diverse in visual images, names, family situations, locations. Names used should be short, easy to read and pronounce & equal across cultural groups. Context information should not detract or interfere with the maths being assessed or become a barrier to the students ability to demonstrate their knowledge.
STORY TELLING INTRODUCTION: Gavin WanganeenAt the Aboriginal Students in the SACE Conference Chris Wilson from Yunggorendi First Nations Centre for Higher Education at Flinders Universityspoke about ‘Engaging Aboriginal Communities in Curriculum Design & Research opportunities. He spoke about how a ‘Framework of Transformative Practice’ could be used to bring the knowledge systems of aboriginal students into the schooling system.The Framework includes Performance Standards, privileges student driven learning, a pedagogy that involves learning in or for community and learning outcomes that are embedded in the cultural context of the students.The Compass Framework can be used as an assessment tool to facilitate the transformative Practices as exampled by the research of Chris White for aboriginal students Compass is student driven –managed by the student through the assessment processThe design of the assessment prompts have been influenced by the students cultural understandings with many aboriginal students involved in the COMPASS trials. Lit/Num outcomes as expressed In the student report enables a pedagogy that can focus directly on the interventions that can improve the learning outcomes of the students.Standards of the assessment are not based on educational achievement standards as expressed through NAPLAN like assessments, but rather the ability of the student. COMPASS has the ability to mediate between achievement standards and where a student is on their learning journey because the reporting framework enables discussions that can improve a student’s ability to meet educational standards.
Compass may not be the literacy and numeracy assessment tool that you want for your purpose but it will assist in identifying what a student can do and what interventions and pedagogical practices you as teachers or providers can decide to use.NAPLAN, TORCH, PAT R, PAT MATHS etc are all assessments which measure where a student is on their learning journey and demonstrate what a student has learnt from that journey.COMPASS can be the link that brings students onto that learning pathway so you can facilitate the learning experiences that will provide them with their passport for the future.
