1. Measurement, Reliability and Validity

2.  Fundamentals of Measurement
 The relationship between Reliability and Selection
 The relationship between Validity and Selection
 How Validity is applied in utilizing assessments

3. 7-3
Key Concepts
• Measurement
• The process of assigning numbers to objects to
represent quantities of an attribute of the objects
• Scores
• The amount of the attribute being assessed
• Correlation between scores
• A statistical measure of the relation between the two
sets of scores

4. The Role of Measurement
 Assists the selection
manager with understanding
an applicants attributes
 It involves numbers playing
an important role
 It is essential to the
implementation and
administration of selection
programs

5. 7-5
Importance and Use of Measures
• Summary of measurement process
• (a) Choose an attribute of interest
• (b) Develop operational definition of attribute
• (c) Construct a measure of attribute as operationally defined
• (d) Use measure to actually gauge attribute
• Results of measurement process
• Scores become indicators of attribute
• Initial attribute and its operational definition are transformed
into a numerical expression of attribute

6. The Role of Measurement
Criteria
 Measurement of an
employees success on
the job
 Dollar of sales or dollars
saved in context of money
or the targets achieved
Predictors
 Forms of assessment
used to measure criteria
 Interviews , Performance
Tests, Paper pencil Tests,
or Computer Tests.

7. The Role of Measurement
Predictors:
 Background
information
 Resumes
 Applications
 Interviews
 Tests:
 Aptitude or Ability
 Achievement
 Personality

8. Use of Measures in Staffing
7-8

9. 7-9
Measurement: Standardization
• Involves
• Controlling influence of extraneous factors
on scores generated by a measure and
• Ensuring scores obtained reflect the attribute
measured
• Properties of a standardized measure
• Content is identical for all candidates measured
• Administration of measure is identical for all
candidates
• Rules for assigning numbers are clearly specified
and agreed on in advance

10. 7-10
Measurement: Differences in
Objective and Subjective Measures
• Objective measures
• Rules used to assign numbers to attribute
are predetermined, communicated, and
applied through a system
• Subjective measures
• Scoring system is more elusive, often
involving a rater who assigns the numbers

11. The Role of Measurement
Evaluating Criteria:
 Will situation change
alter the criteria
 Is it realistic and
represent the job
 Does management
accept the criteria

12. The Role of Measurement
Evaluating Predictors:
 Does the predictor
appear appropriate
 Is the cost less than an
inaccurate decision
 Has it been
standardized
 Can it be administrated
to a group v/s an
individual

13. 7-13
Quality of Measures
• Reliability of measures
• Validity of measures
• Validity generalization

14. Concept of Reliability
Reliability Defined
The degree of dependability,
consistency, or stability of
scores on measures used in
selection

15. Concept of Reliability
Factors Influencing The
Reliability of a Assessment
 Method of Estimating
Reliability
 Individual Differences
Among Respondents
 Length of a Measure
• Test Question Difficulty
• Administration of a measure

16. 7-16
Quality of Measures: Reliability
• Definition: Consistency of
measurement of an attribute
• A measure is reliable to the extent it
provides a consistent set of scores to
represent an attribute
• Reliability of measurement is of
concern
• Both within a single time period and
between time periods
• For both objective and subjective measures

17. 7-17
Ex. 7.6: Summary of Types of
Reliability

18. 7-18
Quality of Measures: Reliability
• Measurement error
• Actual score = true score + error
• Deficiency error: Occurs when there is
failure
to measure some aspect of attribute
assessed
• Contamination error: Represents
occurrence of unwanted or
undesirable influence on the measure
and on individuals being measured

19. 7-19
Ex. 7.7 - Sources of Contamination Error
and Suggestions for Control

20. 7-20
Quality of Measures: Reliability
• Procedures to calculate reliability estimates
• Coefficient alpha
• Should be least .80 for a measure to have an acceptable
degree of reliability
• Inter-rater agreement
• Minimum level of inter-rater agreement - 75% or higher
• Test-Retest reliability
• Concerned with stability of measurement
• Level of r should range between r = .50 to r = .90
• Intra-rater agreement
• For short time intervals between measures, a fairly high
relationship is expected - r = .80 or 90%

21. 7-21
Quality of Measures: Reliability
• Implications of reliability
• Standard error of measurement
• Since only one score is obtained from an
applicant, the critical issue is how accurate the
score is as an indicator of an applicant’s true
level of knowledge
• Relationship to validity
• Reliability of a measure places an upper limit on
the possible validity of a measure
• A highly reliable measure is not necessarily valid
• Reliability does not guarantee validity - it only
makes it possible

22. 7-22
Quality of Measures: Validity
• Definition: Degree to which a measure truly
measures the attribute it is intended to
measure
• How well an assessments predicts employee
performance.
• Accuracy of measurement
• Exh. 7.9: Accuracy of Measurement
• Accuracy of prediction
• Exh. 7.10: Accuracy of Prediction

23. Concept of Validity
Types of Validity Strategies
 Criterion-related Validity
testing the relationship between the predictor and
criterion in relation to job performance
 Content Validity
Validates those job behaviors that are necessary for
successful job performance
 Construct Validity
Test validation method that determines whether a test
measures certain traits or qualities that are important in
performing the job

24. 7-24
Ex. 7.9: Accuracy of
Measurement

25. 7-25
Exh. 7.10: Accuracy of
Prediction

26. 7-26
Exh. 7.10: Accuracy of
Prediction

27. 7-27
Validity of Measures in Staffing
• Importance of validity to staffing process
• Predictors must be accurate representations of
characteristics to be measured
• Predictors must be accurate in predicting job
success
• Validity of predictors explored through
validation studies
• Two types of validation studies
• Criterion-related validation
• Content validation

28. 7-28
Ex. 7.12: Concurrent and Predictive
Validation Designs

29. 7-29
Ex. 7.12: Concurrent and Predictive
Validation Designs

30. 7-30
Content Validation
• Content validation involves
• Demonstrating the questions/problems
(predictor scores) are a representative sample
of the kinds of situations occurring on the job
• Criterion measures are not used
• A judgment is made about the probable
correlation between predictors and criterion
measures
• Used in two situations
• When there are too few people to form a
sample for criterion-related validation
• When criterion measures are not available
• Exh. 7.14: Content Validation

31. 7-31
Validity Generalization
• Degree to which validity can be
extended to other contexts
• Contexts include different situations,
samples of people and time periods
• Situation-specific validity vs. validity
generalization
• Exh. 7.16: The Logic of Validity
Generalization
• Distinction is important because
• Validity generalization allows greater
latitude than situation specificity
• More convenient and less costly not to have
to conduct a separate validation study for
every situation