Education

1. VALIDITY,
RELIABLILITY,
WASHBACK
Carolina Salazar
17.02.2014

3. The test measures what it is
intended to measure.
VALIDITY

4. CONTENT VALIDITY

5. Content Validity
TEST
RELEVANT!!!
skills
structures
concepts

6. Judging Content Validity
SPECIFICATIONS
ACTUAL TEST

7. Importance of Content Validity
Content
General
Content Construct
WASHBACK
ValidityValidity Validity
Validity

8. CRITERION-RELATED
VALIDITY

9. Criterion-related Validity
Assessment
DO
Test results THEY AGREE?
results

10. Kinds of Criterion-related Validity
Concurrent
C-R V.
Test =
criterion
EXAMPLES?Test 
Predictive
Criterion

11. Examples of Criterion-related Validity
One of the objectives of a Spanish course is to measure speaking
(especially pronunciation of vowels) and writing (especially
comments on readings). Testing these abilities in detail could take up
to 2 hours per student (too impractical), but only 20 minutes can be
devoted to each student. Can a twenty-minute session give an
accurate estimate of the student’s ability in these functions?
After the students finish the 20-min test, the teacher decides to
choose a sample of the students taking the evaluation to subject
them to the full 2-hour one. Then he compares the results.

12. Examples of Criterion-related Validity
A student takes a placement test for English. He receives the results
and is placed in 3rd level. How well did the test predict the most
appropriate class for the student? Will he succeed and pass?

13. SCORING VALIDITY

14. Scoring Validity
READING-COMPREHENSION TEST

15. FACE VALIDITY

16. Face Validity

17. HOW TO MAKE TESTS
MORE VALID?

18. Tips for more valid tests
1. Set specifications
2. Use direct testing
3. Relate the scoring to what you are testing

19. The test is repeatable.
RELIABILITY

20. Reliability

21. Reliability Exemplified
STUDENT
SCORE (A)
SCORE (B)
T
80
52
U
61
78
V
98
59
W
43
68
X
52
66
Y
40
98
Z
28
50
• 100-item test
• SCORE A: Monday
• SCORE B: Tuesday
STUDENT
SCORE (A)
SCORE (B)
A
91
97
B
88
83
C
46
50
D
54
52
E
99
99
F
66
70
G
27
25

22. The Reliability Coefficient
0
.....
1

23. Role of Reliability
…THE MORE IMPORTANT THE DECISIONS, THE
GREATER RELIABILITY WE MUST DEMAND.

24. How do we arrive to the R.C.?
Testretest
R.C.
Alternate
forms
Split-half

25. SCORER RELIABILITY

26. Scorer Reliability

27. HOW TO MAKE TESTS
MORE RELIABLE?

28. Tips for more reliable tests
1. Use enough items
2. Do not give too much freedom
3. Write unambiguous items
4. Provide clear instructions
5. Ensure that tests are legible
6. Make candidates familiar with formats/techniques
7. Provide non-distracting and uniform conditions
8. Use items that allow objective scoring
9. Provide a detailed scoring key
10. Train scorers
11. Agree upon acceptable responses and scores before scoring
12. Identify candidates by number
13. Employ multiple, independent scoring

29. Reliability and Validity
A reliable test is not always valid and
viceversa.
WHY?
http://www.youtube.com/watch?v=5lG3R6-4T3k

31. Effects of the test on students
WASHBACK

32. Kinds of Washback
POSITIVE WASHBACK
NEGATIVE WASHBACK
Encourages students
Clarifies concepts
…
Makes students nervous, anxious
Makes teachers/ students ignore
«unimportant» topics
…
http://tapchi.vnu.edu.vn/nn_4_09/b.8.pdf

33. Tips for achieving beneficial washback
1. Test the activities whose development you
want to encourage
2. Sample widely and unpredictably
3. Use direct testing
4. Make testing criterion-referenced
5. Base achievement tests on objectives
6. Ensure the understanding of the test
7. Provide assistance if necessary

34. VALIDITY,
RELIABLILITY,
WASHBACK
Carolina Salazar
17.02.2014