The WJ IV provides two primary methods for comparing tests or cluster scores. One is based on a predictive model (the variation and comparison procedures) and the other allows comparisons of SEM confidence bands, which takes into account each measures reliability. A third method for comparing scores, one that takes into account the correlation between compared measures (ability cohesion model) is not provided, but is frequently used by assessment professionals. The three types of score comparison methods are described and new information, via a "rule of thumb" summary slide and nomograph, are provided to allow WJ IV users to evaluate scores via all three methods.
2. The content of this presentation represents the work and opinions of
Dr. Kevin McGrew and does not necessarily reflect the opinions of all
the WJ IV authors or the publisher of the WJ IV (HMH)
Also note that in the examples provided, interpretation uses the
standard score (SS) metric. The preferable metric for understanding
performance on the WJ IV measures is the Relative Performance
Index (RPI). However, since the question that is addressed is “how
unusual must two test/cluster scores be from each other before I
consider them to represent a meaningful and unusual difference?”,
the SS metric must be used as this is not possible when using the RPI
metric.
3. Three primary models for
evaluating score differences
(Payne & Jones, 1957)
www.iapsych.com/articles/payne1957.pdf
It’s a pleasure when you use
the correct measure
34. How to interpret the base rate rule-of-thumb figure on prior slide: GIA/Gf+Gc example
How big of a SS difference is
needed between a person’s
GIA and Gf+Gc cluster
scores before I can consider
the difference rare and
meaningful?
If 1.5 (13 % base rate) is
your rule, then the
GIA/Gf+Gc difference must
be approximately + 12
points or more.
If 1.65 (10 % base rate) is
your rule, then the GIA
Gf+Gc difference must be
approximately + 13 points
or more.
35. How to interpret the base rate rule-of-thumb figure on prior slide: Gf cluster example
How big of a SS difference is
needed between a person’s
Number Series and Concept
Formation scores (Gf cluster)
before I can consider the difference
rare and meaningful?
If 1.5 (13 % base rate) is your rule,
then the Number Series/Concept
Formation difference must be
approximately + 24 points or more.
If 1.65 (10 % base rate) is your rule,
then the Number Series/Concept
Formation difference must be
approximately + 27 points or more.
36. The required magnitude of SS differences required varies by
degree of correlation (cohesion) between the two measures
Note that the critical
base rate values for a
cluster with highly
correlated tests (Gc; r =
.71; 18/20) are much
smaller than for a cluster
with tests that are more
weakly correlated (Gf;
r=47; 22/27)