20. Typical implementation
Minimum participation requirements for each
of two assessment exercises (PW1, PW2)
Write 1 Answer 5 Rate / comment 3
5% course credit
Physics 101, Energy & Waves
Winter Semester: 3 sections, ~800 students
21. Not so typical implementation
TOCCLT'.'
-
QestIsct
How 1o...
q/t4osTer
Submit ond onswer questions
on topics in lhe torget region,
just obove lhe physics
you hove olreody mostered.
Dislroclers
) l*"*"ò
rNrrltng
Ih s region corrk:ins llte
plrysics knowledgo ond colrco¡ls
you connol leorn yel
becouse the louncalions
crrc nol in plocc
PHYSICS TOPICS IN YOUR TARG T REGION
buo'5ot't1 'Den¡¡lu
B¿:ovvì 1 J
F:rcq - YOUR CHOSEN TOPIC
à..
auC'q^ x@
bôo'osro
pn ro, d logroo,
o 9tS
COMMON MISCONCEPÎIONS AND ERRORS
(Sec hHÞ:/jÞhy¡.udofo¡.cdu/CJP/trêconcoÞllo¡t.pdt
tor o llil ol common mhconcepllons)
¿K N(,qht %cce Açxs no exi¡þ
, c.rrrçrJ On oblec'-t 5 õÞYJrnr cìü?À
I in rtuiò. ü
lrh'brot4ont eprç,e .I¡1enÀs crq
den-s,{^ c>Ç c.,þ¡ec , cìo+ .*¿n:r
oÇ Çtu,8,
fr- btrrqon! $crCe c¿cs Àoc¡:'cr
fr¿lt{e- r *hcrn ú?
Trr¿e. $ììfifflsz hrrr: ¿lÕoolr-g lnnS
(**= tcpo þ/rnl
{otr-* qrÇ bq.:. 0" Ooz
Physics knowledge
ond conccpluol underslonding
you lrove olrcody
conslrucleC ¡n your heod
¡,<'t.lerJ rt)^,¿z! Þ.rrp r- rr't¡ ,l?
Chcck lhol yovr on3Íêr ¡t rêo!ànoblê ðnd potrlble
'f<,,cc
Photo by Seth Casteel
http://www.littlefriendsphoto.com
Permission to use agreed
Writing original questions is a
demanding activity
Extensive scaffolding
exercises
Revisited in subsequent
tutorials
22. Selected results & analysis
Engagement - how do students use the system?
Benefits - what is the impact on learning?
Question quality - how good is what students produce?
Relevant publications:
Scaffolding student engagement via online peer learning - European Journal of Physics 35 (4),
045002 (2014)
Student-Generated Content: Enhancing learning through sharing multiple-choice questions.
International Journal of Science Education, 1-15 (2014).
Assessing the quality of a student-generated question repository - Phys Rev ST PER (2014)
10, 020105
Student-generated assessment - Education in Chemistry (2013) 13 1
23. Engagement with PeerWise
Number Multiplier Number Multiplier
Questions 1105 [1.7] 998 [1.6]
Answers 11393 [17.2] 11807 [18.7]
Comments 4901 [7.4] 5509 [8.7]
PW 1 PW 2
29. Engagement with PeerWise
Generally, students did
• Participate beyond minimum requirements
• Engage in community learning, correcting errors
• Create problems, not exercises
• Provide positive feedback
30. Engagement with PeerWise
Generally, students did not
• Contribute trivial or irrelevant questions
• Obviously plagiarize
• Participate much beyond assessment periods
• Didn’t all leave it to the last minute
36. Question/Explanation Quality
Bloom’s Taxonomy of levels in the cognitive domain
Score Level Description
1 Remember Factual knowledge, trivial plugging in of numbers
2 Understand Basic understanding of content
3 Apply Implement, calculate / determine. Typically one-stage problem
4 Analyze Typical multi-step problem; requires identification of strategy
Evaluate Compare &assess various option possibilities; often conceptual
Synthesize
Ideas and topics from disparate course sections combined. Significantly
challenging problem.
Text
38. Question/Explanation Quality
Description of explanation quality
Score Level Description
0 Missing
No explanation provided or explanation incoherent/
irrelevant
1 Inadequate Wrong reasoning and/or answer; trivial or flippant
2 Minimal
Correct answer but with insufficient explanation/
justification/ Some aspects may be unclear/incorrect/
confused.
3 Good Clear and detailed exposition of correct method & ans
4 Excellent
Thorough description of relevant physics and solution
strategy. Plausibility of all answers considered. Beyond
normal expectation for a correct solution
40. Results (UoE 2010-11)
2 successive years of the same course (N=150, 350)
‘High quality’ questions: 78%, 79%
Over 90% (most likely) correct, and 3/5 of those wrong were
identified by students.
69% (2010) and 55% (2011) rated 3 or 4 for explanations
Only 2% (2010) and 4% (2011) rated 1/ 6 for taxonomic level.
41. Bottomley & Denny Biochem and Mol Biol Educ. 39(5)
352-361 (2011)
107 Year 2 biochem students
56 / 35 / 9 % of questions in lowest 3 levels.
Momsen et al CBE-Life Sci Educ 9, 436-440 (2010)
“9,713 assessment items submitted by 50 instructors in the
United States reported that 93% of the questions asked on
examinations in introductory biology courses were at the
lowest two levels of the revised Bloom’s taxonomy”
Comparison with literature