Handout-SOLO-1 (1).pdf

MARILYN UBIÑA-BALAGTAS, Ph.D.
University Professor, College of Graduate Studies and Teacher Education Research, Philippine Normal University
President, Philippine Educational Measurement and Evaluation Association. Inc. (PEMEA)
balagtas.mu@pnu.edu.ph
Presented this 6th day of December 2021
at DepEd Bulacan//Via Zoom
Use of the SOLO Framework
in Test Development in the
New Normal

Mood Setting
2
Use of the SOLO Framework in Test Development in the New Normal
Dr. Marilyn Ubiña-Balagtas
December 2021

SESSION TARGET
3
December 2021
Topics to Discuss
1. The SOLO Framework and Use in Item Writing, Scoring and
Communication of Test Results
2. The SOLO Framework Combined with the Revised Bloom’s
Taxonomy for Test Development
3. Preparation of the Table of Specifications for Test
Development
The participants are expected to use the SOLO Framework in test
development in the new normal.

Pre-Test
4
December 2021
https://forms.office.com/r/5yPqL5GdtM

What is SOLO?
vSOLO to mean Structure of Observed Learning
Outcomes was popularized by Kevin F. Collis and
John B. Biggs in 1982 in their book “Evaluating the
Quality of Learning: The SOLO Taxonomy.
5
v It is a framework for understanding with five levels,
where each level embraces the previous level but adds
something more.
December 2021

6
Source of Illustration: Doug Belshaw - Own work, CC0, https://commons.wikimedia.org/w/index.php?curid=60807631
Understanding/Competence/Proficiency
What is SOLO?
December 2021

7
Source of Illustration: Doug Belshaw - Own work, CC0, https://commons.wikimedia.org/w/index.php?curid=60807631
Understanding/Competence/Proficiency
What is SOLO?
December 2021

What is SOLO?
8
December 2021

9
How is SOLO used in test construction?
v Use SOLO in structuring responses to questions in a
multiple-choice test format.
v Use SOLO in structuring rubrics in describing
responses to questions in an open-ended or
constructed-response type of test.
v Use SOLO to describe the degree of correctness of
responses given to questions in a test.
v Use SOLO in scoring, interpreting and
communicating test results.
December 2021

10
Level of
Understanding
SOLO
Code Meaning
Levels of
Thinking
Pre-structural (P) 0 I am not sure about this topic
Surface
Uni-structural (U) 1 I have one idea about this topic
Multi-Structural (M) 2 I have several ideas about this topic
Relational (R) 3 I can link the ideas together to see the big
picture
Deep
Extended Abstract
(EA)
4 I can look at this idea in a new and different
way.
December 2021

11
Levels of
Proficiency
Level of
Understanding
Code Meaning
Not Proficient Pre-structural 0 One does not know any idea about the lesson
assessed.
Somewhat
Proficient
Uni-structural 1 One knows an idea about the lesson
assessed.
Nearly Proficient Multi-Structural 2 One knows several ideas about the lesson
assessed but could not link them together.
Proficient Relational 3 One knows several ideas about the lesson
assessed and can link them together.
Highly Proficient Extended Abstract 4 One can link several ideas about the lesson
assessed and can represent it in a new and
different way/situation.
December 2021

12
Levels of
Proficiency
Level of
Understanding
Code Meaning
Level of
Understanding Test Format
Not
Proficient Pre-structural 0 One does not know any idea
about the lesson assessed.
Surface
Understanding
Multiple
Choice
Constructed-
Response
Somewhat
Proficient Uni-structural 1 One knows an idea about the
lesson assessed.
Nearly
Proficient
Multi-
Structural
2 One knows several ideas
about the lesson assessed
but could not link them
together.
Proficient Relational
and could link them
together.
Deep
Understanding
Highly
Proficient
Extended
Abstract
4 One could link several ideas
and could represent them in a
new or a different
way/situation.
December 2021

13
Levels of
Proficiency
Level of
Understanding
Code Meaning
Level of
Understanding Scoring Options
Not
Proficient Pre-structural 0 One does not know any idea
about the lesson assessed.
Surface
Understanding
0 0
Somewhat
Proficient Uni-structural 1 One knows an idea about the
lesson assessed. 1 1
Nearly
Proficient
Multi-
Structural
but could not link them
together.
2 3
Proficient Relational
and could link them
together.
Deep
Understanding
3 5
Highly
Proficient
Extended
Abstract
4 One could link several ideas
and could represent them in a
new or a different
way/situation.
4 6
December 2021

14
Let’s have an example
of questioning
in a test that is in
constructed-response
format.
December 2021

15
As of June 30, 2021
Study the table below. What percent of the population of the USA got infected of COVID-19 compared
with that of the Philippines?
December 2021

16
Student 1
Philippines
1, 408,058 = 0.01268%
111,025,935
USA
34, 527, 493= 0.1037%
332,929,050
Student 3
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
Student 2
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
10.37% - 1.27% = 9.1%
Who among the four students gave the correct answer?
Student 4
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
Answer: 10.37% of the population of USA got infected of
COVID-19 compared to the Philippines with only 1.27%.
December 2021

17
Student 1
Philippines
1, 408,058 = 0.01268%
111,025,935
USA
34, 527, 493= 0.1037%
332,929,050
Student 3
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
Student 2
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
10.37% - 1.27% = 9.1%
How do the students differ in their understanding of the problem based on their answers?
This student knows how to compute the rate but
does not know how to express this in percent.
This student knows how to compute the rate and how
to express it in percent but did not provide clearly the
correct answer.
This student knows how to compute the rate and how to express it in
percent but misunderstood the problem as it does not require subtraction.
Student 4
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
COVID-19 compared to the Philippines with only 1.27% .
This student knows how to compute the rate and how to
express it in percent to answer the problem.
December 2021

18
Student 1
Philippines
1, 408,058 = 0.01268%
111,025,935
USA
34, 527, 493= 0.1037%
332,929,050
Student 3
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
Student 2
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
10.37% - 1.27% = 9.1%
Which of the SOLO levels does each response represent?
This student knows how to compute the rate but does not
know how to express this in percent. (UNISTRUCTURAL)
percent but did not provide clearly the correct answer.
(MULTISTRUCTURAL)
percent but misunderstood the problem as it does not require
subtraction. (MULTISTRUCTURAL)
Student 4
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
express it in percent to answer the problem. (RELATIONAL)
December 2021

19
Student 1
Philippines
1, 408,058 = 0.01268%
111,025,935
USA
34, 527, 493= 0.1037%
332,929,050
Student 3
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
Student 2
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
How could Extended Abstract in the SOLO framework look like?
Student 4
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
Answer: 10.37% of the population of USA got infected of COVID-19 compared to the
Philippines with only 1.27% COVID cases. This explains why USA is no.1 in the world in
terms of COVID cases while the Philippines is just ranked 24th
.
percent to answer the problem. The student also gave more acceptable
information than what is needed. (EXTENDED ABSTRACT)
This student knows how to compute the rate but does not
know how to express this in percent. (UNISTRUCTURAL)
express it in percent but did not provide clearly the correct
answer. (MULTISTRUCTURAL)
express it in percent to answer the problem. (RELATIONAL)
Dr. Marilyn Ubiña-Balagtas//December 2021

20
Levels of Quality
Adjectival
Description
Indicator SOLO Level
5 Excellent
The answer given is correct and beyond what was
asked for.
Extended Abstract
4 Very Good
The answer given shows understanding of two or more
processes in solving the problem and it directly answers
the problem.
Relational
3 Good
The answer given shows understanding of two or more
processes in solving the problem but does not directly
answer the problem.
Multistructural
2 Fair
The answer given shows understanding of one process
that leads to answering the problem.
Unistructural
1 Poor
There is NO attempt to answer the problem or there is
an answer but meaningless.
Prestructural
How will the rubric be designed using SOLO?
December 2021

21
Levels of
Quality
Adjectival
Description
Indicator SOLO Level Code
Example 1
of Scoring
Example 2
of Scoring
5 Excellent The answer given is correct and beyond
what was asked for.
Extended
Abstract
4 4 Points 10 Points
4 Very Good The answer given shows understanding of
two or more processes in solving the
problem and directly answers the problem.
Relational
3 3 Points 8 Points
3 Good The answer given shows understanding of
two or more processes in solving the
problem but does not directly answer the
problem.
Multistructural
2 2 Points 6 Points
2 Fair The answer given shows understanding of
one process that leads to answering the
problem.
Unistructural
1 1 Point 3 Points
1 Poor There is NO attempt to answer the problem
or there is an answer but meaningless.
Prestructural 0 0 Point 0 Point
How are responses scored based on SOLO?
December 2021

22
Student 1
Philippines
1, 408,058 = 0.01268%
111,025,935
USA
34, 527, 493= 0.1037%
332,929,050
Student 3
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
Student 2
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
How are responses scored based on SOLO?
This student knows how to compute the rate but does not know how to
express this in percent. (UNISTRUCTURAL) – 3 Points
Student 4
Philippines
1, 408,058 = 1.27%
111,025,935
USA
34, 527, 493= 10.37%
332,929,050
COVID-19 compared to the Philippines with only 1.27%
COVID cases.
percent and how to answer the problem. (RELATIONAL) – 8 Points
Answer: 10.37% of the population of USA got infected of COVID-19 compared to
the Philippines with only 1.27% COVID cases. This explains why USA is no.1 in
the world in terms of COVID cases while the Philippines is just ranked 24th.
This student knows how to compute the rate and how to express it in percent
and how to answer the problem. The student also gave more information than
what is needed. (EXTENDED ABSTRACT) – 10 Points
percent but did not provide the clearly the correct answer.
(MULTISTRUCTURAL) – 6 Points

23
How do we transform the
item into a
multiple-choice format?
December 2021

24
As of June 30, 2021
A. 10.37% of the population of USA got
infected of COVID-19 compared to the
Philippines with only 1.27%.
B. 0.1037% of the population of USA got
Philippines with only 0.01268%.
C. 0.1037 of the population of USA got
Philippines with only 0.01268.
D. USA has 9.1% more COVID cases than
the Philippines.
December 2021
How do we transform the
item into a
multiple-choice format?

25
As of June 30, 2021 Options
SOLO
CODE
SOLO Levels
A
10.37% of the population of
USA got infected of COVID-19
compared to the Philippines
with only 1.27%.
3
Relational
B.
with only 0.01268%.
1
Unistructural
C.
0.1037 of the population of
with only 0.01268.
1
D.
USA has 9.1% more COVID
cases than the Philippines. 2
Multistructural
December 2021

26
As of June 30, 2021
Options SOLO
Explanation/
Feedback
A
with only 1.27%.
3
You know how to
compute the rate and
how to express it in
percent to answer the
problem.
B.
with only 0.01268%.
1
You know how to
compute the rate, but
you do not know how to
express this in percent.
C.
0.1037 of the population of
with only 0.01268.
1
D. USA has 9.1% more COVID
cases than the Philippines.
2 You know how to
compute the rate and
how to express it in
percent, but you
misunderstood the
problem as it does not
require subtraction.
Study the given graph. What percent of the population of the USA got infected of COVID-19 compared
How do we communicate test results using the SOLO Framework?
December 2021

SESSION TARGET
27
December 2021
Topics to Discuss
Development

28
Dimensions Remember Understand Apply Analyze Evaluate Create
Factual
least student-centered
& authentic 5 9 13 17 21
Conceptual 2 6 10 14 18 22
Procedural 3 7 11 15 19 23
Metacognitive 4 8 12 16 20
most student-
centered &
authentic
How do we combine SOLO with RBT to make
assessment multidimensional?
Revised Bloom’s Taxonomy
(Krathwohl & Anderson, 2001)
December 2021

29
Remember:
Retrieving, recognizing, and recalling relevant knowledge from
long-term memory.
Understand:
Constructing meaning from oral, written, and graphic messages
through interpreting, exemplifying, classifying, inferring,
comparing and explaining.
Apply:
Carrying out or using a procedure through executing, or
implementing.
Analyze:
Breaking material into constituent parts, determining how the
parts relate to one another and to an overall structure or
purpose through differentiating, organizing, and attributing.
Evaluate:
Making judgments based on criteria and standards through
checking and critiquing.
Create:
Putting elements together to form a coherent or functional
whole; reorganizing elements into a new pattern or structure
through generating, planning, or producing.
Cognitive Behaviors in the Revised Bloom’s Taxonomy
If you aim for
authentic
assessment,
which of these
levels should
you target?
December 2021

30
Knowledge Dimensions in the
Knowledge
Dimension
Description
Factual Terminologies, specific details and elements
Conceptual Knowledge of classifications, principles ,
generalization, theories, models, and structures
Procedural Knowledge of subject-specific skills and
algorithms/procedures
Metacognitive Self-knowledge, strategic knowledge, contextual
and conditional knowledge, knowledge of a
cognitive task
If you aim
for
authentic
assessme
nt, which
of these
levels
should you
target?
December 2021

31
Factual 1 5 9 13 17 21
Conceptual 2 6 10 14 18 22
Procedural 3 7 11 15 19 23
Metacognitive 4 8 12 16 20 24
Which is the most learner-centered and
authentic cell in this matrix?
December 2021

32
Factual
& authentic 5 9 13 17 21
Conceptual 2 6 10 14 18 22
Procedural 3 7 11 15 19 23
most student-
centered &
authentic
December 2021

33
Factual
& authentic 5 9 13 17 21
Conceptual 2 6 10 14 18 22
Procedural 3 7 11 15 19 23
most student-
centered &
authentic
1. What is 9 in 4 +
5 = 9?
December 2021

34
Factual
& authentic 5 9 13 17 21
Conceptual 2 6 10 14 18 22
Procedural 3 7 11 15 19 23
most student-
centered &
authentic
2. What is addition?
December 2021

35
Factual
& authentic 5 9 13 17 21
Conceptual 2 6 10 14 18 22
Procedural 3 7 11 15 19 23
most student-
centered &
authentic
3. What is the 1st
step in adding
numbers?
December 2021

36
Factual
& authentic 5 9 13 17 21
Conceptual 2 6 10 14 18 22
Procedural 3 7 11 15 19 23
most student-
centered &
authentic
4. Maria bought 4 ripe
and 5 unripe mangoes.
How many ripe
mangoes did she buy?
December 2021

37
Factual
& authentic 5 9 13 17 21
Conceptual 2 6 10 14 18 22
Procedural 3 7 11 15 19 23
most student-
centered &
authentic
23. Show
how to add
any number.
11. What is the sum of
4 and 5?
3. What is the first
step when adding
numbers?
15. How does the
sum of 4 and 5
compare with the
sum of 6 and 3?
19. Is the sum of 4 and
5 10?
7. How will you show
addition of 4 and 5?

38
Factual
& authentic 5 9 13 17 21
Conceptual 2 6 10 14 18 22
Procedural 3 7 11 15 19 23
most student-
centered &
authentic
24. You need to buy items
at the store. What are these
and their total?
4.Maria bought 4 ripe and 5
unripe mangoes. How many
ripe mangoes did she buy?
8.Maria bought 4
ripe and 5 unripe
mangoes. Which
is more, ripe or
unripe mangoes?
20.Maria bought 4 ripe and
5 unripe mangoes. She
bought 10 mangoes in all.
Do you agree? Explain.
unripe mangoes. How many
mangoes did she buy?
unripe mangoes today. She
bought 10 mangoes yesterday.
How many mangoes did she buy
in all?

39
Let’s see more
examples of how to
increase the
authenticity of items
guided by the RBT.
December 2021

40
Which is more authentic and targets higher thinking
skill? Explain your answer.
1. What is 15 + 5?
A. 10
B. 15
C. 20
2. During their morning break, Romalyn bought pancit canton worth Php15 and
a glass of coconut juice worth Php5. How much did Romalyn spend for her
snacks?
A. Php 10
B. Php15
C. Php 20
December 2021

41
1. What is 15 + 5?
A. 10
B. 15
C. 20
2. During their morning break, Romalyn bought pancit canton worth Php15 and
a glass of coconut juice worth Php5. How much did Romalyn spend for her
snacks?
A. Php 10
B. Php15
C. Php 20
Add – Apply - Procedural
Add/Solve
Apply/Analyze
Procedural/Metacognitive
December 2021

42
1. During their morning break, Romalyn bought pancit canton worth Php15 and a glass of
coconut juice worth Php5. How much did Romalyn spend for her snacks?
A. Php 10
B. Php15
C. Php 20
2. During your morning break, you bought a pancit canton worth Php15 and a glass of
coconut juice worth Php5. How much did you spend for your snacks?
A. Php 10
B. Php15
C. Php 20
December 2021

43
1. During their morning break, Romalyn bought pancit canton worth Php15 and a glass of
coconut juice worth Php5. How much did Romalyn spend for her snacks?
A.Php 10
B. Php15
C.Php 20
2. During your morning break, you bought a pancit canton worth Php15 and a glass of
coconut juice worth Php5. How much did you spend for your snacks?
A.Php 10
B. Php15
C. Php20
Add/Solve
Apply/Analyze
Add/Solve
Apply/Analyze
Personal/
authentic
December 2021

44
1. During your morning break, you bought a pancit canton worth Php15 and a
glass of coconut juice worth Php5. How much did you spend for your snacks?
A.Php 10
B. Php15
C. Php20
2. During your morning break, you bought ______ worth Php____ and
_______worth Php_____. How much did you spend for your snacks?
December 2021

45
1. During your morning break, you bought a pancit canton worth Php15 and a
glass of coconut juice worth Php5. How much did you spend for your snacks?
A.Php 10
B. Php15
C. Php20
Add/Solve
Apply/Analyze
Personal/
authentic
Add/Solve/Create
Apply/Analyze/Create
Personal/
authentic
December 2021

46
2. Construct your own word problem involving addition of whole numbers.
December 2021

47
1. During your morning break, you bought ______
worth Php____ and _______worth Php_____.
How much did you spend for your snacks?
2. Construct your own word problem involving
addition of whole numbers.
Add/Solve/Create
Personal/
Authentic/Restricted
Add/Solve/Create
Personal/
Authentic/Extended
December 2021

48
Why combine
the RBT and SOLO
in test development?
December 2021

49
Why combine the SOLO Framework
and the RBT?
v Combining the SOLO Framework and the RBT makes assessment
multidimensional.
v Multidimensional assessment of learning is an approach of looking at
different facets of learning to identify the strengths and weaknesses of the
learner to better inform instruction and better describe achievement.
v Assessment is multidimensional if it satisfies any of the following:
1. It examines two or more dimensions in one’s learning (e.g., Revised Bloom’s
Taxonomy, PISA).
2. It examines both cognitive and non-cognitive dimensions of learning (e.g.,
TIMSS)
December 2021

50
Why do we need to practice
multidimensional assessment?
v Multidimensional assessment of learning is needed in the K to
12 program as the curriculum intends to develop holistically
the learners and equip them with 21st century skills to make
them locally and globally competitive.
v The curriculum framework of the different areas in basic and
higher education shows multidimensionality, hence assessment
should be multidimensional as well.
December 2021

51
The curriculum framework shows multidimensionality; hence assessment should be multidimensional as well.
Example of the Multidimensional Curriculum Framework
December 2021

52
The curriculum framework shows multidimensionality;
hence assessment should be multidimensional as well.
Example of the Multidimensional Curriculum Framework
December 2021

53
What are the models of
Some models of multidimensional assessment of learning that
are practiced in the Philippines are as follows:
Classroom
1. Revised Bloom’s Taxonomy
2. Revised Bloom’s Taxonomy combined with the SOLO
Framework
3. Combination of Different Assessment Approaches that
Appropriately Assess Different Learning Outcomes
December 2021

54
Some models of multidimensional assessment of learning that are
practiced in the Philippines are as follows:
National
1. Assessment of Competencies and Excellence of Students
(ACES) of Rex Institute for Student Excellence, Inc.
2. Basic Education Exit Assessment (BEEA)Framework
December 2021

55
Assessment of Competencies and Excellence of Students (ACES)
from Rex Institute for Student Excellence, Inc. (RISE)
December 2021

56
December 2021

57
December 2021

v
SHS Exit Assessment
• Language and Communication
• Wika at Komunikasyon
• Humanities
• Media and Information Literacy
• Mathematics
• Science
• Social Science and Personal Development
• Philosophy
Content
Knowledge
Performance
Standards
Competencies
21st Century Skills
Basic Education Exit Assessment (BEEA) for SHS
Source: Balagtas et.al (2020). Analysis of the
Exit Assessment for Senior High School
Students: Input to the K to12 Reform

59
Some models of multidimensional assessment of learning that are
practiced in the Philippines are as follows:
International
1. Program for International Student Assessment (PISA)
2. Trends in International Mathematics and Science Survey (TIMSS)
3. Southeast Asia Primary Learning Metric (SEA-PLM)
December 2021

PISA 2015/2018 Scientific Literacy
Assessment Framework
for Cognitive Dimension
Knowledge
Competencies Depth of Knowledge
December 2021

PISA 2015/2018
Scientific
Literacy
Assessment
Item Framework
COMPETENCIES
• Explaining
phenomena
scientifically
• Evaluating and
designing scientific
inquiry
• Interpreting data
and evidence
scientifically
KNOWLEDGE
• Content
• Procedural
• Epistemic
COGNITIVE
DEMAND
• Low
• Medium
• High
CONTEXTS
• Personal
• Local/National
• Global
December 2021

62
Figure #: Mathematics Framework of the PISA 2021
PISA 2021 Framework
for Mathematics
Literacy
December 2021

63
December 2021

64
December 2021

65
December 2021

66
How do we combine
the RBT and SOLO
December 2021

67
How do we combine the RBT and SOLO
Objective: Identify the different cognitive processes in the Revised Bloom’s
Taxonomy
Item
1. What cognitive process in the RBT is involved when a student is required to
organize data into various categories?
A. analyze ?
B. apply ?
C. classify ?
D. understand ?
December 2021
• What is the answer to the
item?
• What cognitive process
and knowledge dimension
does the item represent?
• What levels of
understanding based on
the SOLO Framework do
the choices represent?

68
December 2021
Taxonomy
Item
A. analyze ?
B. apply ?
C. classify ?
D. understand ?
Factual - Remember
item?
• What levels of

69
December 2021
Taxonomy
Item
A. analyze ?
B. apply ?
C. classify ?
D. understand ?
Factual - Remember
Do the options in the MC item
have the same
degree of correctness?

70
December 2021
Taxonomy
Item
A. analyze ?
B. apply ?
C. classify ?
D. understand ?
Factual - Remember
How will you describe the
offered responses
in terms of SOLO?

71
December 2021
Taxonomy
Item
A. analyze Relational
B. apply Uni-structural
C. classify Uni-structural
D. understand Uni-structural
Factual - Remember

72
December 2021
Taxonomy
Item
B. apply Unistructural
C. classify Unistructural
D. understand Unistructural
Factual - Remember
How will you represent
pre-structural in this
set of responses?

73
December 2021
Taxonomy
Item
B. apply Unistructural
C. classify Unistructural
D. characterize Prestructural
Factual - Remember
How will you represent
pre-structural in this
set of responses?

74
Let’s have another
example of how to combine
RBT and SOLO
in test development.
December 2021

75
Example:
Which is the first step in constructing an Achievement Test ?
A. Examine the instructional objectives
B. Prepare a table of specifications
C. Construct test items
D. Teach the students
Objective: Identify the steps in developing summative tests
item?
• What levels of
December 2021

76
Example:
Which is the first step in constructing an Achievement Test ?
A. Examine the instructional objectives Relational
B. Prepare a table of specifications Multistructural
C. Construct test items Unistructural
D. Teach the students Prestructural
Objective: Identify the steps in developing summative tests
Procedural – Remember
December 2021

77
Situation:
After teaching the day’s lesson, you gave your students a test. The result shows that almost all of
the students got a low score. Which of the following should you do?
A. Give a parallel test.
B Teach the lesson again in the same way.
C. Count the frequency of errors to know what to explain again.
D. Determine the items students found difficult that would require
further explanation.
Objective: Use assessment results in making decisions about teaching and learning
item?
• What levels of
December 2021

78
Situation:
A. Give a parallel test. Uni-structural
B Teach the lesson again in the same way. Uni-structural
C. Count the frequency of errors to know what to explain again. Multi-structural
D. Determine the items students found difficult that would require
further explanation.
Relational
Metacognitive - Apply
December 2021

79
December 2021
Situation:
A. Review its pre-requisite lesson. Uni-structural 1 point Somewhat
Proficient
B. Teach the lesson again in the same way. Uni-structural 1 point Somewhat
Proficient
C Count the frequency of errors to know what
to explain again.
Multi-structural 2 points Nearly Proficient
D Determine the items students found difficult
that would require further explanation.
Relational 3 points Proficient
Metacognitive - Apply
How do we score test items using the SOLO Framework?

80
Let’s have another
example of how to combine
RBT and SOLO
in test development.
December 2021

81
Objective: Justify the appropriateness of an assessment method given the
objective of the lesson (Constructed-Response Format).
Situation:
You want to find out if your students can make a new material out of recyclable
materials. You gave an essay test requiring them to explain what to do out of old
telephone directories. Considering the congruency between the intent and the
assessment method used, did you do what is right? Justify your answer.
December 2021

82
objective of the lesson (Constructed-Response Format).
Situation:
You want to find out if your students can make a new
material out of recyclable materials. You gave an essay
test requiring them to explain what to do out of old
telephone directories. Considering the congruency
between the intent and the assessment method used,
did you do what is right? Justify your answer.
item?
• What levels of
the SOLO Framework
could the answers
represent?
December 2021

83
objective of the lesson (Constructed-Response Format). Evaluate-Metacognitive
Situation:
You want to find out if your students can make a new
material out of recyclable materials. You gave an essay
test requiring them to explain what to do out of old
telephone directories. Considering the congruency
between the intent and the assessment method used,
item?
• What levels of
the SOLO Framework
could the answers
represent?
December 2021

84
objective of the lesson (Constructed-Response Format). Evaluate-Metacognitive
Situation:
You want to find out if your students can
make a new material out of recyclable
materials. You gave an essay test requiring
them to explain what to do out of old
telephone directories. Considering the
congruency between the intent and the
assessment method used, did you do what
is right? Justify your answer.
Sample Answers Given
• Example 1:
• Example 2:
• Example 3:
• Example 4:
December 2021

85
Objective: Justify the appropriateness of an assessment method given the objective of the lesson
(Constructed-Response Format).
Situation: You want to find out if your students can make a new material out of recyclable materials. You gave an
essay test requiring them to explain what to do out of old telephone directories. Considering the congruency
between the intent and the assessment method used, did you do what is right? Justify your answer.
Scoring Rubric
1 Poor You gave no response or showed no effort to respond.
2 Fair You gave a response but with no justification.
3 Good You gave a response with a justification that is partially acceptable or not applicable in the given
situation.
4 Very Good You gave a response with a justification that is totally acceptable or applicable in the given situation.
5 Excellent You gave a response with a very convincing justification that is acceptable in most situations and not
only in the given situation.
December 2021

86
Objective: Justify the appropriateness of an assessment method given the objective of the lesson (Constructed-
Response Format).
Situation: You want to find out if your students can make a new material out of recyclable
materials. You gave an essay test requiring them to explain what to do out of old telephone
directories. Considering the congruency between the intent and the assessment method used,
1 Poor You gave no response or showed no effort to respond. Pre-structural
2 Fair You gave a response but with no justification. Uni-structural
3 Good You gave a response with a justification that is partially acceptable or not applicable
in the given situation.
Multi-structural
4 Very Good You gave a response with a justification that is totally acceptable or applicable in the
given situation.
Relational
5 Excellent You gave a response with a very convincing justification that is acceptable in most
situations and not only in the given situation.
Extended
Abstract
December 2021

87
Response Format).
Situation: You want to find out if your students can make a new material out of recyclable materials. You gave an essay test
requiring them to explain what to do out of old telephone directories. Considering the congruency between the intent and the
1 Poor You gave no response or showed no effort to respond. Pre-structural
2 Fair You gave a response but with no justification. Uni-structural
3 Good You gave a response with a justification that is partially acceptable or not applicable in the given situation. Multi-structural
4 Very Good You gave a response with a justification that is totally acceptable or applicable in the given situation. Relational
5 Excellent You gave a response with a very convincing justification that is acceptable in most situations and not only in
the given situation.
Extended Abstract
December 2021
Sample Answers Given
• Example 1:
• Example 2:
• Example 3:

88
December 2021
Situation: You want to find out if your students can make a new material out of recyclable materials. You gave an essay test
requiring them to explain what to do out of old telephone directories. Considering the congruency between the intent and the
1 Poor You gave no response or showed no effort to respond. Pre-structural 0 Point Not Proficient
2 Fair You gave a response but with no justification. Uni-structural 2 Points Somewhat Proficient
3 Good You gave a response with a justification that is
partially acceptable or not applicable in the given
situation.
Multi-structural 5 Points Nearly Proficient
4 Very Good You gave a response with a justification that is totally
acceptable or applicable in the given situation.
Relational 8 Points Proficient
5 Excellent You gave a response with a very convincing
justification that is acceptable in most situations and
not only in the given situation.
Extended
Abstract
10 Points Highly Proficient
88
Meta-cognitive - Evaluate
How do we score test items using the SOLO Framework?
Response Format).

89
Situation: You want to find out if your students can make a new material out of recyclable materials. So you gave
an essay test requiring students to explain what to do out of old telephone directories. Considering the
congruency between the intent and the assessment method used, did you do what is right? Justify your answer.
A. Yes, the use of essay test is just appropriate to what is being measured.
B. No, because it is difficult to check the students’ responses to the task using an essay test.
C. No, it is more appropriate to observe them while they are creating something new out of recyclable
materials
D. Yes, asking the students to explain what to do with the material is already enough to gauge what
they are able to do.
December 2021

90
Situation: You want to find out if your students can make a new material out of
recyclable materials. So you gave an essay test requiring students to explain what to do
out of old telephone directories. Considering the congruency between the intent and
the assessment method used, did you do what is right? Justify your answer.
B. No, because it is difficult to check the students’ responses to the task using
an essay test.
C. No, it is more appropriate to observe them while they are creating
something new out of recyclable materials
D. Yes, asking the students to explain what to do with the material is already
enough to gauge what they are able to do.
• What is the answer
to the item?
• What cognitive
process and
knowledge
dimension does the
item represent?
• What levels of
understanding based
on the SOLO
Framework could the
choices represent?
December 2021

91
Situation: You want to find out if your students can make a new material out of recyclable materials. So you gave
an essay test requiring students to explain what to do out of old telephone directories. Considering the
congruency between the intent and the assessment method used, did you do what is right? Justify your answer.
B. No, because it is difficult to check the students’ responses to the task using an essay test.
C. No, it is more appropriate to observe them while they are creating something new out of recyclable
materials
D. Yes, asking the students to explain what to do with the material is already enough to gauge what
they are able to do.
December 2021

92
Situation:
Situation: You want to find out if your students can make a new material out of recyclable materials. So you gave an
essay test requiring students to explain what to do out of old telephone directories. Considering the congruency
between the intent and the assessment method used, did you do what is right? Justify your answer.
A. Yes, the use of essay test is just appropriate to what is being measured. Uni-structural
(1 point)
B. No, because it is difficult to check the students’ responses to the task using an essay
test.
Multi-structural
(2 points)
C. No, it is more appropriate to observe them while they are creating something new
out of recyclable materials
Relational (3 Points)
D. Yes, asking the students to explain what to do with the material is already enough to
gauge what they are able to do.
Uni-structural
(1 Point)

93
Some Rules in Using SOLO for Test Development
December 2021
1. Construction of Choices: Provide responses that the students will
likely give if the format of the test is constructed-response.
2. Scoring of the Responses: Score based on the degree of correctness
of the responses and level of cognitive demand. Make sure that there
is only one correct and best answer in every multiple-choice item
unless the item format is constructed-response.
3. Interpretation and Feedback to Responses: Explain sufficiently what
makes the response correct/incorrect so that students will learn from
the feedback provided. Indicate the proficiency/competence level of
the learner from the kind of response given.

94
Let’s look at
examples of questioning
in PISA.
December 2021

95
Let’s look at a sample test in PISA Math.
Ans: D
Source: PISA Released
Items in Mathematics,
December 2006, OECD
December 2021

96
Let’s look at a sample test in PISA Math.
Ans: D
Source: PISA Released Items in Mathematics, December 2006, OECD
Choices
SOLO
Level
Score/
Code
A. 6 mg Unistructural 1
B. 12 mg Unistructural 1
C. 26 mg Unistructural 1
D. 32 mg Relational 3
How do we apply SOLO in explaining
students’ responses to the item?
December 2021

97
Ans: D
Source: PISA Released Items in Mathematics, December 2006, OECD
Choices
SOLO
Level
Score/
Code
Explanation/
Feedback
A. 6 mg Unistructural 1 You gave one correct
idea that is based on
the given information.
B. 12 mg Unistructural 1
C. 20 mg Unistructural 1
D. 32 mg Relational 3 You interpreted
correctly the graph
that led you to
giving the right
answer to the
problem.
December 2021

98
Let’s look
at a
sample
test item
in PISA
Reading.
Items in Reading,
December 2006, OECD
Ans:
A
December 2021

99
Let’s look at a sample test item in PISA Reading.
Source: PISA Released Items in Reading, December 2006, OECD
Ans:
A
Choices
SOLO
Level
Score/
Code
A. About two meters R 3
B. About 15 meters U 1
C. About 50 meters P 0
D. It has disappeared
completely.
U 1
E. The information is
not provided.
P 0
December 2021

100
Source: PISA Released Items in Reading, December 2006, OECD
Ans:
A
Choices
SOLO
Level
Score/
Code
Explanation/
Feedback
A. About two
meters
R 3 You interpreted
correctly the graph that
led you to giving the
right answer to the
problem.
B. About 15
meters
U 1 You gave one correct
the information given.
C. About 50
meters
P 0 You gave a response
that has no basis at all.
D. It has
disappeared
completely.
U 1 You gave one correct
the information given.
E. The information
is not provided.
P 0 You gave a response
that has no basis at all.
December 2021

101
Let’s look
at a
sample
test item
in PISA
Science.
Items in Science,
December 2006, OECD
Ans:
A
December 2021

102
Let’s look at a sample test item
in PISA Science.
Source: PISA Released Items in
Science, December 2006, OECD
Ans:
A
Choices
SOLO
Level
Score/
Code
A. The Earth rotates on
its axis.
R 3
B. The Sun rotates on
its axis. U 1
C. The Earth’s axis is
tilted. M 2
D. The Earth revolves
around the Sun M 2
December 2021

103
Source: PISA Released Items in
Science, December 2006, OECD
Ans:
A
Choices
SOLO
Level
Score/
Code
Explanation/
Feedback
A. The Earth
rotates on its
axis.
R 3 You were able to give the right
explanation why daylight and
darkness occur on Earth.
B. The Sun rotates
on its axis.
U 1 You know that the sun rotates
on its axis but this does not
explain day and night on Earth.
C. The Earth’s axis
is tilted.
M 2 You know that the Earth’s axis
is tilted which was explained in
the information provided but it
does not explain the
occurrence of day and night on
Earth.
D. The Earth
revolves around
the Sun
M 2 You know that the Earth
revolves around the Sun which
is the source of light but it
does not fully explain the
occurrence of day and night as
that is due to Earth’s rotation
on its axis.
December 2021

SESSION TARGET
104
December 2021
Topics to Discuss
Development

105
Content Competencies
Inst’l Time
(hrs) # of Items % of items
Level of Behavior, Format, No.,& Placement of Items and
the Knowledge Dimension
R U Ap An E C
1.Method of
Assessment
Examine the
good and bad
practices in
assessing
student’s
learning
3 10 20% I. 1
#1 (F)
I. 2
#6-7
(C)
I. 1
#15-16
(M)
1. 3
#17-19
(M)
I. 2
#36-38
(M)
II. 39-
40
(M)
Etc.
Total No. of Items 15 50 100% 5 10 20 10 3 2
Scoring 3 pts - maximum
for each item
4 pts - maximum
for each item
5 pts - maximum
for each item
SOLO Codes P-0; U-1; M-2; R-3 P-0; U-1; M-2; R-4 P-0; U-1; M-3; R-5
Total Number
of Points
190 points 45 Points 120 Points 25 Points
December 2021
Balagtas’ Model for Test Development with RBT and SOLO Combined

106
Content Inst’l Time
# of Items
% of items KD
Level of Behavior, Item Format, No.,& Placement
R U Ap An E C
1.Methods of
Assessment
3 10 20%
F I. 1
#1
C I .2
#6-7
I. 1
#16
P I. 3 #17
to 19
II. 2
#39-40
M I. 2
#36-37
2. etc.
Total 15 50 100% 5 10 20 10 3 2
Scoring 3 pts - maximum
for each item
4 pts - maximum
for each item
5 pts - maximum
for each item
SOLO Codes P-0; U-1; M-2; R-3 P-0; U-1; M-2; R-4 P-0; U-1; M-3; R-5
Total Number of Points 190 points 45 Points 120 Points 25 Points
December 2021

107
December 2021
Prepare a Grade Level Achievement Test for the Division
Things to Do
1. Prepare a Table of Specifications (ToS) for an
Achievement Test of a grade level. The number
of subject areas taught in the grade level will be
the same number of ToS to be prepared.
2. Use the suggested template in the preparation
of the ToS of an Achievement Test for a grade.
Dr. Marilyn Balagtas' Model TOS Template for Test Development with RBT and
SOLO Combined
https://tinyurl.com/pmz4hzyj
3. Exchange ToS for the validation guided by a
rubric.

108
Suggested Template for the Table of Specifications

109
December 2021
Post-Test
https://forms.office.com/r/bNmHPNGhm3

110
What is your take-away
from the session?
December 2021

“You cannot manage
what you cannot measure”
Peter Drucker
“If you cannot measure,
you can’t improve”
Gray Mac Kenzie
December 2021

No one is perfect and so is
an assessment tool.
Reduce its imperfection
through constant revision.
Marilyn U. Balagtas
December 2021
December 2021

113
About the Speaker
DR. MARILYN UBIÑA-BALAGTAS is a University Professor at the
Philippine Normal University teaching courses on educational assessment,
evaluation, and research from undergraduate to doctoral level in face-to-face and
online modality. She has been a Dean of PNU College of Flexible Learning and
ePNU. She finished her Doctor of Philosophy with specialization in Educational
Research and Evaluation at the University of the Philippines. She has been a regional
finalist in Metrobank Search for Outstanding Teacher in the Tertiary Level and has
been awarded a post-doctoral fellowship for Leadership in Research by the University
of New England, Australia for her projects as the Inaugural Director of the
Philippine National Research Center for Teacher Quality (RCTQ) that has led the
development of the Philippine Professional Standards for Teachers (PPST). She is
the founding President of the PATEF-United Professionals for the Development and
Advancement of Teacher Education (UPDATE) and currently the President of the
Philippine Educational Measurement and Evaluation Association (PEMEA), Inc.
She is a consultant in educational assessment of a testing company in the Philippines.
She is a researcher, trainor, and a writer in areas of assessment and teacher education.
December 2021

MARILYN UBIÑA-BALAGTAS, Ph.D.
University Professor
College of Graduate Studies and Teacher Education Research
Philippine Normal University
President, Philippine Educational Measurement and Evaluation
Association. Inc. (PEMEA)
balagtas.mu@pnu.edu.ph
Thank you.
When interested to use this material,
please cite as:
Balagtas, M. U. (2021). Use of the SOLO
Framework in Test Development in the
New Normal. Philippine Normal
University, Manila
December 2021

115
How to construct written tests?
(For more details, please refer to this book.)
December 2021

116
For details on performance-based and portfolio
assessments and grading, please refer to this book.
December 2021

117
For inquiries about Assessment in Learning 1 and 2 Textbooks,
please get in touch with given contact details below:
December 2021

118
For inquiries about Assessment in Learning 1 and 2 Textbooks,
please get in touch with given contact details below:
December 2021

Handout-SOLO-1 (1).pdf

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Handout-SOLO-1 (1).pdf

Semelhante a Handout-SOLO-1 (1).pdf (20)

Mais de ShaShean

Mais de ShaShean (6)

Último

Último (20)

Handout-SOLO-1 (1).pdf