Clickers can make teaching more effective and fun, but how does a teacher best use clickers in the class? In this interactive workshop, we’ll explore research-based ideas for questioning to achieve student engagement and deep learning. We will focus on the use of “peer instruction” in which students discuss challenging questions. We’ll compare example questions, practice writing questions, discuss common challenges, and share tips on getting students to productively reason through them. No software needed.

1. Make Clickers Work for You
FACILITATION TIPS AND
TECHNIQUES
Dr. Stephanie V. Chasteen
Dr. Steven Pollock
Physics Department
&
Science Education Initiative
Univ. of Colorado at Boulder
http://colorado.edu/sei
Web and blog: http://sciencegeekgirl.com
Email: stephanie.chasteen@colorado.edu

2. Who are you?
Show of hands
A. Natural sciences
B. Social sciences
C. Humanities
D. Arts
E. Languages
F. Other

3. Have you used response systems (clickers) in
your teaching?
Take a clicker & turn it on
If the green light flashes, your
vote has been counted
A. Not at all, and I haven’t seen them used
B. Not at all, but I’ve observed their use somewhat
C. I’ve used them a little
D. I’ve used them a lot
E. I could be (should be?) giving this workshop

4. How familiar are you with “Peer Instruction”
Colored cards
A. Fairly familiar, and I like it
B. Fairly familiar, but I’m not sure that I like it
C. I’ve heard of it but only have a vague idea what it is
D. Not familiar at all
E. Not sure

5. Introducing Me
5
Science Education Initiative
http://colorado.edu/SEI
Applying scientific principles to improve science
education – What are students learning, and which
instructional approaches improve learning?
Physics Education Research Group
http://PER.colorado.edu
One of largest PER groups in nation, studying
technology, attitudes, classroom practice, & institutional
change.
Blogger
http://blog.sciencegeekgirl.com

6. Agenda
1. Thinking about questioning
2. About clickers and peer instruction
3. Writing great clicker questions
4. Overcoming common challenges
5. Action plan

7. Why question?
7
1. Why do we question our
students?
2. When might you use
questioning in your classes?
3. For what purposes might
clickers be an appropriate
questioning tool?
Discuss in groups of 2-3 for 5
minutes.
(May make notes in your
handout)
whiteboard

8. What is special about clicker questions?
8
 Similar goals as other types of
questioning techniques
 Multiple choice
 Anonymous (to peers)
 Every student has a voice – the
loud ones and the shy ones Clickers are a tool
 Forced wait time
 You can withhold the answer
until everyone has had time to
think (choose when to show the
histogram)

9. When can we ask questions?
9
BEFORE
Setting up instruction DURING
E.g.: Developing
knowledge
Motivate
Assess prior knowledge Application
… (handout!) Elicit misconception
…
AFTER Relate to big picture
Assessing Demonstrate success
learning
…
Credit: Rosie Piller and Ian Beatty.

10. Gallery Walk
 With a partner, look at the “example
questions” trios on the wall.
 What do you think an instructor would be
trying to accomplish with such questions?
 Jot down any ideas on the sheet
5 minutes
Aihofanz2010 on Wikimedia

11. Agenda
1. Thinking about questioning
2. About clickers and peer instruction
3. Writing great clicker questions
4. Overcoming common challenges
5. Action plan

12. Clickers are a tool for questioning
12
But not a magic bullet!
Don’t equate the pedagogy with the technology.
So what IS the pedagogy?

13. Why use peer instruction?
13

14. An outline of Peer Instruction.
14

15. Anatomy of Peer Instruction
15
Ask Question
…Lecture… (Maybe vote)
Class Discussion Peer Discussion
Vote
* See also: Peer Instruction, A User’s Manual. E. Mazur.

16. Burning questions?
16
Ask Question
…Lecture… (Maybe vote)
Class Discussion Peer Discussion
Vote
* See also: Peer Instruction, A User’s Manual. E. Mazur.

17. U. Colorado clicker resources…
17
Videos of effective use of clickers 2-5 mins long
http://STEMvideos.colorado.edu
Clicker resource page
http://STEMclickers.colorado.edu
• Instructor’s Guide
• Question banks
• Workshops
• Literature / Articles

19. Peer instruction helps students learn
19
Research shows that:
 Students can better answer a similar question after
talking to their peers
 Peer discussion + instructor explanation of
question works better than either one alone
 Students like peer instruction
 Peer instruction classes outperform traditional
lectures on a common test
See http://STEMclickers.colorado.edu for various references

20. Example question: Physics
20
Which superpower would you
rather have? The ability to…
A. Change the mass of things
B. Change the charge of things
C. Change the magnetization of things
D. Change the boiling point of things
20
Question: Ian Beatty, UNC Greensboro Image: Thibault fr on Wikimedia

21. Example question: Literature
If Homer wrote the Iliad today, Stanley Fish and Harold
Bloom would argue, respectively, whether the work
should be categorized as:
A. Existential vs. Romantic
B. Postmodern vs Classical
C. Modern vs Romantic
D. Postcolonial vs Modern
E. Preliterate vs Postliterate
The Technology Enhanced Learning and
Research center at Ohio State
Origin unknown

22. Example question: Math
Your sister in law calls to say that she’s having twins. Which of
the following is the most likely? (Assume she’s having
fraternal, not identical, twins)
A. Twin boys
B. Twin girls
C. One girl and one boy
D. All are equally likely
22
Derek Bruff, Vanderbilt

23. Example question: History or Ethics
If you were a judge, how would you assess the “responsibility” of
the U.S. Government, for what happened in the world between
1933 and 1945?
A. Not responsible
B. Minimally responsible
C. Responsible
D. Very responsible
Origin unknown 23

24. Let’s try it
I think the toughest thing about using clickers
and peer instruction in class is / will be:
A. Writing good questions
B. Getting students to really think about the questions
C. Getting students to share their reasoning with the
whole class
D. The same students always respond in whole class
discussion
E. It takes too long / I have a lot of content to cover

25. Honestly, I think that I’m most likely to modify this
technique of Peer Instruction to suit me and my
students. I know that there are at least ___ parts of
the technique that I’ll be changing:
A. None
B. One
C. Two-three
D. Four or more

26. Is there a problem with modifications?
I won’t tell you how to teach. You’re smart & you care
about instruction. But realize that modifications may
change the effectiveness of the technique.
Be strategic about modifications. Know the research.

27. Some research on modifications
 63.5% of faculty (in physics) say they are familiar
with Peer Instruction
 30% report that they use Peer Instruction
 50% of those use Peer Instruction in the way
described by developers
 Often dropped are:
 Student discussion Is this a problem?
Use of conceptual questions

Probably.
 Whole-class voting
Dancy & Henderson, Pedagogical practices and instructional change of faculty, Am. J. Phys., 78(10), Oct
2010.Web survey of 722 physics faculty at various institutions, initial sample of 2000.

28. Exercise #1: Core Philosophies
28
In groups: What are the underlying principles that make this work?
Ask Question
…Lecture… (Maybe vote)
Class Discussion Peer Discussion
Vote
* See also: Peer Instruction, A User’s Manual. E. Mazur.

29. Some core philosophies of mine
Clicker questions are an integral part of my lecture
Students learn by
 … teaching each other
 … articulating their ideas
It’s important for me to
 …. hear student ideas
 … know what my students understand
I value and respect student ideas
I want students to
 … know that I value student ideas
 … feel safe sharing their ideas

30. Agenda
1. Thinking about questioning
2. About clickers and peer instruction
3. Writing great clicker questions
4. Overcoming common challenges
5. Action plan

31. Question-writing tips
 Move away from simple quizzes
 Use questions that prompt discussion
 Use questions that emphasize reasoning or process
 Use clear wording
 Use tempting distracters
 Use questions for a variety of instructional goals
 Use questions at a mixture of cognitive depth
 Ask challenging questions – don’t just test
memorized facts
See handout

32. Some instructional goals for clickers
32
BEFORE
Setting up instruction DURING
E.g.: Developing
knowledge
Motivate
Assess prior knowledge Application
… (handout!) Elicit misconception
…
AFTER Relate to big picture
Assessing Demonstrate success
learning
…
Credit: Rosie Piller and Ian Beatty.

33. Effective multiple-choice questions have believable
“distracters.”
1)Talking with other
instructors that
have taught the
course in the
past.
2)Talking with your
students one-on-
one before
class, after
class, during
office hours.
3)Using student
responses to open-
ended questions D. Duncan, Univ. of Colorado
33

34. Exercise #2: Try writing a question
 Choose one of the question goals (slide #3 on page 2
in handouts)
 Write a draft question that aims to achieve this goal.
5 minutes

35. An example question
What causes the seasons?
A. The change in the earth’s distance from the sun
during the year
Bad question.
B. The tilt of the earths axis Students can
C. Changes in the sun’s brightness answer by
memorizing a
D. Changes in clouds word (“tilt”)
E. None of the above
Can we make a better question on the SAME topic?
Yes…

36. Better seasons example
What would happen to the seasons if the earth’s
orbit around the sun was made a perfect circle
(but nothing else changed) ?
A. There would be no seasons
B. The seasons would remain pretty much as they
are today
C. Winter to spring would differ much less than
now
D. Winter to spring would differ much more than
now Much better question. Requires reasoning!

37. Use questions at a variety of cognitive depth
37
Do the questions you use intellectually
challenge your students or simply assess
their factual knowledge?
Higher order
----------------
Lower order
handout

38. Exercise #3: Rate and swap
 Use the Bloom’s Taxonomy worksheet to rate the
Bloom’s level of your question
 Swap your question with a neighbor. Do you agree
on the Bloom’s level of your question?
 Use the verbs on the detailed Bloom’s handout to
“Bloomify up” the level of your question.
5 minutes

39. What was the Bloom’s level of your question?
A. Remembering
B. Understanding
C. Applying
D. Analyzing
E. Evaluating

40. Share out
 What did you learn in this process?
 What worked well, what was challenging?
 How might you go about writing questions in your
class?

41. Agenda
1. Thinking about questioning
2. About clickers and peer instruction
3. Writing great clicker questions
4. Overcoming common challenges
5. Action plan

42. Exercise #4 Challenges in the Classroom
42
 You ask students a question, and ask them to discuss.
 You then ask them to share their answers and
reasoning in a whole-class discussion
 What could possibly go wrong?  5 mins
With a partner, brainstorm some challenges
and write them on the board (in the
appropriate area).
What are some possible solutions that are in
line with your core philosophies?

43. 1. Asking Question. Philosophies? Challenges?
43
Philosophies
•Questions are integral to lecture
•Students can learn by considering a
question
Best practices
•Ask several times during lecture
•Ask challenging, meaningful questions
•Don’t post until ready & give time to read
43
Handout/worksheet / whiteboard

44. 2. Peer Discussion. Philosophies? Challenges?
44
Philosophies:
• Students learn through discussion
• Students need to know that you value
their ideas & that it’s safe to share
Solutions:
•Make it clear why you’re doing this
• Circulate and ask questions / model
•Use questions they want to discuss
•Allow enough time (2-5 mins)
•Focus on reasoning in wrap-up

45. Talking brings convergence
45
 Eric Mazur -
Harvard U.
Before discussion After discussion
Why do you think this happens?
A B C
A B
(A) Students are getting answers from the ‘smart’ kids
C
(B) They’re learning from their discussions
(C) They just needed more time to think about it
Mazur, 1997

46. The hypothesis: If students learn from peer
discussion, they should show better performance
on a similar question. Ask a second, similar
question without any instructor input: Q2
Undergrad introductory genetics course. 16 Q1/Q2
pairs.
Research by Michelle
Smith, Bill Wood, Wendy
Adams, Carl
Wieman, Jenny
Knight, Nancy Guild, Tin
Tin Su, MCDB. Smith et al., Science. 2009, 323(5910):122.

47. Are they learning from peers?
100
100
1) Students answer
90
8080
Q1 individually. 70
6060
Percent
Q
Percent
50 Q
Q
4040
Students talk to
2)
30
neighbors and 2020
answer Q1 again
10
(Q1AD = Q1“After 00
Q1 Q1
Q1AD Q1a Q2
Q2
Discussion”).
Individual After Individual
Discussion
3) Students answer Q2
individually . Q2 tests
same concept as Q1.
n= 350 students
Then explain answers to Q1 and Q2
Smith et al., Science. 2009, 323(5910):122.

48. Can students answer difficult questions correctly after
discussion?
100100 Q1
90 90 Very few students
Q1after discussion
80 80 knew correct
Q2 answer to Q1, but
70 70
Percent correct
60 60 after
Percent
50 50 discussion, many
40 40 more answer
30 30 correctly: students
20 20 are constructing
10 10 their own
0 0 knowledge
Easy Easy Medium
Medium Hard
Difficult
(5 questions) (7 questions) (4 questions)
Smith et al., Science. 2009, 323(5910):122.

49. Student buy-in is key!

50. 3. Wrap-Up Discussion. Challenges?
50
Philosophies:
•Student ideas are important
•Students need to feel safe
Solutions:
•Establish culture of respect
•Consider whether to show the histogram immediately
• Ask multiple students to defend their answers
• Emphasize reasoning: Why are wrong answers wrong
and why right answer is right

51. Giving the answer stops student thinking!
51

52. Other things we haven’t talked about?
52
 Other challenges / solutions / philosophies?

53. Action Plan
53
 Take a few minutes to write down your action plan to
implement ideas you heard about in the workshop

54. U. Colorado clicker resources…
54
Videos of effective use of clickers
http://STEMvideos.colorado.edu
2-5 mins long
Clicker resource page
http://STEMclickers.colorado.edu
• Instructor’s Guide
• Question banks PLUS past workshops
And all workshop materials
•Literature / Articles
I can help you with your institution’s workshops too

55. Thanks!
Resource Page: http://STEMclickers.colorado.edu
Web and blog: http://sciencegeekgirl.com
Email: stephanie.chasteen@colorado.edu
 Many materials in this workshop (particularly the questioning cycle and the participant exercises)
were adapted from Rosie Piller, Making Students Think: The Art of Questioning. Short papers
published in: Computer Training & Support Conference, 1995; ISPI International Conferences, 1991
and 1996; ASTD National Conference on Technical & Skills Training, 1990. Related workshop
description at http://www.educationexperts.net/nstworkshop.html. Other materials (particularly
sample clicker questions and goals of clicker questions) adapted from Ian Beatty’s Technology
Enhanced Formative Assessment (TEFA) program. http://ianbeatty.com/crs