1. Measurement Reactivity
A Meta-Analysis of the Effects of Presenting
Theory of Planned Behaviour Constructs on
Follow-Up Behaviour
Evon Mankarious
Emily Kothe

2. Outline of presentation
• Introduction
-What is measurement reactivity and why does it occur?
• Methods
-Inclusion and Exclusion criteria
-Database search and search terms
-Data-extraction and meta-analytical process
• Results
-Literature search
-Number of studies included
-Number of studies measuring particular behaviours
-Moderator analyses and sub-group analyses
• Discussion
-What do the results suggest?
-Strengths and limitations
-Future direction
2

3. What is measurement reactivity and why does it
occur?
• Phenomenon occurs when the mere presentation of questions about
intention at baseline changes follow-up behaviour
• Likely to occur in any theory where intention is a central construct- Theory
of planned behaviour was used here.
3

4. Theory of planned behaviour
4

5. Measurement reactivity changes in behaviour at
follow-up
• Measurement reactivity has been reported to change follow-up behaviour
in many health behaviours:
- Cervical cancer screening (Sandberg and Conner, 2009)
- Blood donation (Godin et al., 2008)
- Physical activity (Godin et al., 2011)
- Illicit drug use (Williams et al., 2006)
5

6. Limitations of measurement reactivity literature
Behavioural
change
Measurement Participation
reactivity effects
6

7. Problems with isolating measurement reactivity
from participation effects
• Kypri et al. (2011) – What participants think about the nature of the study
may affect subsequent behaviour and potentially bias study findings
• doesn’t address the issue of measurement reactivity as measurement of
intention remains common across all groups.
• Solomon four-group designs can experimentally manipulate baseline
assessment but do not isolate measurement reactivity from participation
effects.
7

8. Significance of the current analysis and aims
• Best way- investigate changes in non-intervention studies which measure
intention at baseline and behaviour at both baseline and follow-up through
meta-analytical techniques.
• Using studies not designed to investigate measurement reactivity reduces
risk of potential publication bias in measurement reactivity literature.
• No research has attempted to systematically investigate the existence of
measurement reactivity within non-intervention studies.
8

9. Aims
to determine the extent to which behaviour
changed in non-intervention studies which
applied the theory of planned behaviour
questionnaire to measure intention at baseline.
9

10. Investigation of Moderator Variables
• Type of behaviour (Socially desirable vs. undesirable)
- inconsistencies in measurement reactivity studies measuring socially
undesirable behaviour.
- Included to determine if classification of behaviour had a moderating effect
• Length of follow-up
- Based on previous literature, was hypothesised that increasing length of follow-
up time would decrease magnitude of measurement reactivity.
10

11. Inclusion/Exclusion criteria
Inclusion Exclusion
Only studies looking at health Cross-sectional, qualitative, and
behaviours intervention studies
measured all theory of planned Articles published in languages other
behaviour constructs at baseline than English
measured behaviour at baseline and
follow-up
11

12. Database search and search terms
• Literature search and data extraction phases were performed in June
2012.
• PsychINFO (OVID), MEDILINE (OVID) and Web of Science (ISI web of
knowledge) databases
• Search strategy used here was modelled on a recent meta-analysis on the
theory of planned behaviour in prospective studies conducted by
McEachan and colleagues (2011).
12

13. Data extraction and met-analysis procedure
• Mean, standard deviation, and sample size at both baseline and follow-up
were extracted to allow for the calculation of effect sizes
• Cohen’s d was calculated to determine the standardised mean difference
for behaviour from baseline to the first follow-up
• Meta-analysis conducted using the Metafor package for R
• Follow-up length of time and behaviour type examined in a mixed-effects
model.
• Sub-group analyses were also conducted to determine the mean effect
size for behaviour investigated in more than one study
13

14. Results: Literature search
Literature Search n
Database search (after duplicates removed) 4034
Title 1630
Abstract 2221
Full-text 183
Total 23
14

15. Number of studies measuring particular
behaviours and Type of behaviour
15

16. Length of follow-up
Length of follow-up n
1 week 5
2 weeks 3
4 weeks 1
5 weeks 3
6 weeks 1
8 weeks 2
3 months 2
6 months 6
16

17. Moderator analysis, changes in behaviour across
all studies and sub-group analyses
• Non-significant effects were found for behaviour type (p = .35) and for follow-
up length (p = .83)
• Average change in behaviour across all studies was small (d = -.08, 95% CI [-
0.17, 0.01])
• Physical Activity
- Average change from baseline to follow-up was small (k = 13, d < .001, 95% CI
[-0.07, 0.07]
• Binge Drinking behaviour:
- Average change from baseline to follow-up was small and decreased (k = 2, d =
-0.14, 95% CI [-0.41, 0.13]
17

18. Discussion: Does measurement reactivity occur?
• Results did not support a measurement reactivity account of behavioural change
• Changes in behaviour in intervention studies may actually be the result of other
factors, for example participation effects and not measurement reactivity
18

19. Is length of follow-up time a moderator?
• No- non-significant effects were found
• Hypothesis that increasing follow-up length of time would reduce magnitude of
measurement reactivity was not supported.
• Researchers using this theory should use the theory to accurately predict
behaviour and it does not appear that they should be concerned about
measurement reactivity.
19

20. Does behaviour type matter?
• No
• Changes in behaviour in studies investigating socially desirable behaviours did not
differ from studies investigating socially undesirable behaviours.
• Inconsistencies observed between studies which have measured socially
undesirable behaviours may be the result of:
- Question framing
- socially undesirable behaviours may also affected by social norms
20

21. Sub-group analyses
• Binge Drinking:
- Small decrease in behaviour
- Caution should be used as only 2 studies were found
- Previous studies have shown that presenting questions about intention at
baseline can decrease binge drinking behaviour
- As only 2 studies were included, it is possible that changes in binge drinking
behaviour may be the result of measurement reactivity.
• Physical Activity:
- Results suggest that changes in behaviour are unlikely to be the result of
measurement reactivity and may reflect participation effects.
- Not consistent with previous research
21

22. Strengths and Limitations
Strengths Limitations
• No previous meta-analysis looking at • Sub-group analyses could not be
measurement reactivity conducted on many behaviours
• Use of large body of literature • Studies may have measured
reduced publication bias behaviour twice but reported it once
22

23. Future direction
• To increase number of studies included in future meta-analyses, other theories that
have intention as a core construct could be included
• To our knowledge, no research exists which allows for the separation of
behavioural change believed to be the result of measurement reactivity vs.
behavioural changes as a result of participation effects.
• New design would be an 8-arm 2x2x2 fully crossed factorial design.
23

24. Example of new study design
24

25. Thank-you
25

26. References
1. Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179-211.
2. Ajzen, I., & Fishbein, M. (1980). Understanding attitudes and predicting social behavior. Englewood Cliffs, N.J.: Prentice-Hall.
3. Chandon, P., Morwitz, V. G., & Reinartz, W. J. (2004). The short- and long-term effects of measuring intent to repurchase. Journal of Consumer Research, 31(3), 566-572.
4. Chapman, K. J. (2001). Measuring intent: There's nothing “mere” about mere measurement effects. Psychology and Marketing, 18(8), 811-841.
5. Chatzisarantis, N. L. D., Frederick, C., Biddle, S. J. H., Hagger, M. S., & Smith, B. (2007). Influences of volitional and forced intentions on physical activity and effort within the theory of planned behaviour. Journal of Sports Sciences,
25(6), 699-709.
6. Chatzisarantis, N. L. D., Hagger, M. S., & Smith, B. (2007). Influences of perceived autonomy support on physical activity within the theory of planned behavior. Special Issue: Familiarity Impacts Person Perception, 37(5), 934-954.
7. Cooke, R., Sniehotta, F., & Schüz, B. (2007). Predicting binge-drinking behaviour using an extended TPB: Examining the impact of anticipated regret and descriptive norms. Alcohol and Alcoholism, 42(2), 84-91.
8. Elliott, M. A., Armitage, C. J., & Baughan, C. J. (2003). Drivers' compliance with speed limits: An application of the theory of planned behavior. Journal of Applied Psychology, 88(5), 964-972.
9. Elliott, M. A., & Thomson, J. A. (2010). The social cognitive determinants of offending drivers' speeding behaviour. Accident Analysis and Prevention, 42, 1595-1605.
10. French, D. P., & Sutton, S. (2010). Reactivity of measurement in health psychology: How much of a problem is it? What can be done about it? British Journal of Health Psychology, 15(3), 453-468.
11. Fulham, E., & Mullan, B. (2011). Hygienic food handling behaviors: attempting to bridge the intention-behavior gap using aspects from temporal self-regulation theory. Journal of Food Protection, 74(6), 925-932.
12. Godin, G., Bélanger-Gravel, A., Amireault, S., Vohl, M. C., & Pérusse, L. (2011). The effect of mere-measurement of cognitions on physical activity behavior: A randomized controlled trial among overweight and obese individuals.
International Journal of Behavioral Nutrition and Physical Activity, 8(1), 2.
13. Godin, G., Sheeran, P., Conner, M., & Germain, M. (2008). Asking questions changes behavior: Mere measurement effects on frequency of blood donation. Health Psychology, 27(2), 179-184.
14. Hagger, M. S., Chatzisarantis, N., Biddle, S. J. H., & Orbell, S. (2001). Antecedents of children's physical activity intentions and behaviour: Predictive validity and longitudinal effects. Psychology & Health, 16(4), 391-407.
15. Jackson, C., Smith, R. A., & Conner, M. (2003). Applying an extended version of the theory of planned behaviour to physical activity. Journal of Sports Sciences, 21(2), 119-133.
16. Kwan, M. Y. W., Bray, S. R., & Ginis, K. A. M. (2009). Predicting Physical Activity of First-Year University Students: An Application of the Theory of Planned Behavior. Journal of American College Health, 58(1), 45-52.
17. Lowe, R., Eves, F., & Carroll, D. (2002). The influence of affective and instrumental beliefs on exercise intentions and behavior: A longitudinal analysis. Journal of Applied Social Psychology, 32, 1241-1252.
18. Martin, J. J., Oliver, K., & McCaughtry, N. (2007). The theory of planned behavior: Predicting physical activity in Mexican American children. Journal of Sport & Exercise Psychology, 29, 225-238.
19. McEachan, R. R. C., Conner, M., Taylor, N. J., & Lawton, R. J. (2011). Prospective prediction of health-related behaviours with the Theory of Planned Behaviour: A meta-analysis. Health Psychology Review, 5(2), 97-144.
20.Nejad, L. M., Wertheim, E. H., & Greenwood, K. M. (2004). Predicting dieting behavior by using, modifying, and extending the theory of planned behavior. Journal of Applied Social Psychology, 34(10), 2099-2131.
21.Norman, P., Armitage, C. J., & Quigley, C. (2007). The theory of planned behavior and binge drinking: Assessing the impact of binge drinker prototypes. Addictive Behaviors, 32(9), 1753-1768.
22.Norman, P., & Conner, M. (2005). The theory of planned behavior and exercise: Evidence for the mediating and moderating roles of planning on intention-behavior relationships. Journal of Sport & Exercise Psychology, 27(4), 488-504.
23.Norman, P., Conner, M., & Bell, R. (1999). The theory of planned behavior and smoking cessation. Health Psychology, 18(1), 89-94.
24.Norman, P., Conner, M., & Bell, R. (2000). The theory of planned behaviour and exercise: Evidence for the moderating role of past behaviour. British Journal of Health Psychology, 5(3), 249-261.
25.Norman, P., & Cooper, Y. (2011). The theory of planned behaviour and breast self-examination: Assessing the impact of past behaviour, context stability and habit strength. Psychology & Health, 26(9), 1156-1172.
26.Norman, P., & Smith, L. (1995). The theory of planned behaviour and exercise: An investigation into the role of prior behaviour, behavioural intentions and attitude variability. European Journal of Social Psychology, 12(4), 403-415.
26