The ppt. from the workshop on day 2 (TEAM 2014.

1. KAYSERI, 29-31 JANUARY 2014
Tests of receptive skills
developing tests for Reading and Listening

3. Aims of the session
! challenges in testing for comprehension
! what sub-skills are to be tested
! selecting texts
! writing items
! task types: pros and cons

4. What makes testing
comprehension challenging?

5. What makes testing
comprehension challenging?
Pros
Cons
! relatively easy to create if
texts are available
! no observable learner
behaviour
! input can control learner
output
! familiarity and interest
may affect success
! objective marking is
normally possible
! if a learner fails, difficult to
know why they failed
! are we only testing what
we intend to test?

6. receptive skills can only
be tested indirectly
language performance ! inferences to comprehension
abilities

7. The process of writing a test
set
aims
select
texts
write
tasks

8. Sub-skills for reading

9. Sub-skills for listening

10. Selecting texts
! a representative sample of text types learners are
likely to encounter
! appropriate extent
! texts with high information content
! clear structure
! neutral but interesting themes
! avoid relying on general knowledge
! avoid culturally loaded texts
! avoid texts learners have already read

11. Item writing checklist
" text and item grammatically correct
" English natural and appropriate
" item achieves test objective
" the intended sub-skills needed to answer item
" one correct key (or a limited number of responses)
" open answers within student abilities
" open answers with valid and reliable marking
" distractors all work
" item economical and practical
" key complete and correct

12. Common task types for testing
receptive skills

13. Common task types for testing
receptive skills
! multiple choice
! short answers
! gap fill
! information transfer
! multiple matching
! for both reading and listening
! note-taking, dictation or transcription
! for listening only

14. Common task types for testing
receptive skills
! short answers
! multiple choice
! gap fill
! multiple matching
! information transfer
! for both reading and listening
! note-taking, dictation or transcription
! for listening only
What are the pros and cons of each task
type?

15. Short answers
Pros
! each student can
complete an item
according to their
abilities
! no distractors
! links reception to
controlled production
Cons
! need to decide if answer
length is specified
! unpredictable alternative
answers
! difficult to phrase stem
to determine answer
! encourages verbatim
quoting –
comprehension?

16. Multiple choice
Pros
! very familiar format
! quick to complete
! easy to mark
! very objective
! high discrimination
index
Cons
! difficult to write good
distractors
! need to produce one
correct answer only
! temptation to use for
every purpose
! may encourage
guessing

17. Tips for writing good distractors
" Make sure there is only ONE correct answer.
" They should be the same form as the key.
" They should be related logically to the key.
" Key should not stand out – similar length and level.
" Avoid synonyms so they can’t be eliminated at once.
" Keep distractor extent and repetition to a minimum.
" Don’t pair up two distractors (these focus attention).
" Check for unintended clues (e.g. an … ! vowel).
" Avoid illogical or obviously impossible distractors.
" If nobody chooses a distractor, it isn’t a distractor!

18. Gap fill
Pros
! familiar format
! easy to create
! relatively easy to mark
Cons
! difficult to predict all
alternative answers
! some items can be
answered without
reading the whole item

19. Multiple matching
Pros
! clear task, easy to do
! easy to mark
Cons
! errors duplicate as
items aren’t
independent of one
another
! can be solved by
elimination (where
there are no distractor
items)
! encourages guessing

20. Information transfer
Pros
Cons
! integrated ! real-life
skill
! two or more skills
tested at once
! involving and
challenging task
! difficult to write tasks
that limit the number of
variables
! difficult to mark

21. Note-taking, dictation,
transcription
Pros
Cons

22. Open comprehension
questions
Pros
! requires detailed output
! better differentiation
between students?
! integrated: tests
comprehension AND
production
Cons
! answer extent is
unspecified

23. Thank you!
zoltan.rezmuves@digimedia.hu
consonantvoiced.blogspot.com