Gender effects in a randomized trial of individual tutoring with children in care Children living in foster care are a particularly vulnerable group of children that are at high-risk for experiencing a multitude of difficulties, including poor academic performance and achievement. Although the academic underachievement of foster children has been well-documented for decades, very few attempts have been made to address the problem. This is the second study to come out of the RESPs for Kids in Care research project, which represents one of three known randomized controlled trials aimed at providing an academic intervention to foster children with a view to improving their basic skills, increasing their chances of graduating from high school and enrolling in post-secondary education. The unique contribution and intent of the current study was to evaluate the differential gender effects of a foster parent-delivered tutoring intervention on the foster children's academic skills and mental health. The focus of this investigation was original and exploratory because the potential role of gender in moderating the impact of a tutoring intervention had never been examined in tutoring research conducted in the general population (Ritter et al., 2009) nor in that carried out among young people in care (Courtney, 2008; Flynn et al., 2012; Harper & Schmidt, 2012; Harper, 2012). A mixed-method approach was used to explore these differential effects and the main hypothesis of the project, that the foster children in the experimental group would demonstrate greater gains in reading and math than the children in the control group, between pre-test and post-test, regardless of gender. The results were promising: the foster children in the experimental group demonstrated significant gains in their basic reading and math skills after receiving the foster-parent delivered one-on-one tutoring, with some evidence of differential gender effects across the academic results.

1. Gender effects in a randomized trial
of individual tutoring with children
in care
Robyn Marquis & Robert J. Flynn
School of Psychology
& Centre for Research on Educational and
Community Services (CRECS)
University of Ottawa (Canada)
EUSARF 2014, Copenhagen
September 3, 2014

2. Canada &
Ontario

3. Low educational achievement of
young people in care:
Research in Canada
• In Canada, results similar to those in USA & UK
• Flynn & Biro (1998): higher rates of suspension and
grade retention than for peers in general population
• Flynn et al. (2004): In samples of young people in
care:
– 10-15 years of age: 80% scored in same range
as lowest third of general Canadian population on
parental ratings of reading, spelling, and math
– 5-9 years of age: 78% scored in same range of
lowest third of Canadian population (same
criteria)

4. • Effect size = size of effect of intervention
• Cohen’s d or Hedges g (nearly identical)
• Criteria for effect sizes in education:
– What Works Clearinghouse (2011): 0.25
– Lipsey et al. (2012) (medians):
• Individual interventions: 0.29
• Small-group interventions: 0.22
• Classroom: 0.08
• Whole school: 0.14
• Overall: 0.18
A note on effect sizes in education

5. Tutoring: A useful intervention
• Systematic review & meta-analysis by
Ritter et al. (2006, 2009):
– Studies of children in general population
– 21 randomized studies, 28 cohorts
– Tutoring produced positive effects:
• Reading overall (d = 0.30)*
• Reading global (d = 0.26)*
• Reading oral fluency (d = 0.30)*
• Reading letters & words (d = 0.41)*
• Reading comprehension (d = 0.18)
• Writing (d = 0.45)*
• Mathematics (d = 0.27)

6. Direct-Instruction Tutoring &
Maloney’s Teach Your Children Well
• Direction-instruction teaching method:
– Well-organized, structured, effective method
of teaching reading & math skills
– For special & general education students
– See National Institute for Direct
Instruction web site (http://www.nifdi.org/)
• M. Maloney’s Teach Your Children Well:
– DI-based (http://www.maloneymethod.com/)
– Combined with behavior management
– Uses tutor manuals, learn-to-read series of
books, workbooks, math CD-ROM, training

7. Our randomized trial
(Flynn et al., 2012)
• Collaboration between:
– 9 Children’s Aid Societies in Ontario &
– University of Ottawa (CRECS)
• Two main questions:
1. Does individual direct-instruction tutoring
help children living in foster care to catch
up in reading & math?
2. Do girls and boys benefit equally from
direct-instruction tutoring?

8. Method
• Participants: 77 foster children
– Children in foster care (grades 2-7, ages 6-13)
and their foster parents (tutors)
– Randomly assigned to control or intervention
groups
• 2008-2009 school year
– Wait-list control group (n = 35)
– Intervention group (n = 42): Tutoring by foster
parents, using Maloney’s TYCW method, for 30
weeks, 3 hrs/week

9. • Outcome measure:
– Wide Range Achievement Test (WRAT4):
• Word Reading
• Sentence Comprehension
• Reading Composite
• Spelling
• Math Computation
– Mental health measures
Method

10. Method - Analysis Sample
• Foster children reassessed at post-test:
– Total N = 64
– 30 children who had actually received the tutoring
intervention
– 34 children in wait-list control condition
• Intervention and control conditions still
equivalent, despite attrition

11. • Question no. 1:
Does individual direct-instruction
tutoring help children living in
foster care to catch up in reading
& math?
Results

12. WRAT4 Word Reading:
Results at post-test (N = 64)
95
96
97
98
99
100
101
102
103
104
105
Pre-test Post-test
MeanStandardScore
Assessment Occasion
Tutoring (n = 30) Control (n = 34)
(g = .19, p = .19, 1-tailed, ns;
post-test scores adjusted for pre-test scores)

13. WRAT4 Reading Comprehension:
Results at post-test (N=64)
95
96
97
98
99
100
101
102
103
104
105
Pre-test Post-test
MeanStandardScore
Assessment Occasion
Tutoring (n = 30) Control (n = 34)
(g = .38, p = .035, 1-tailed;
post-test scores adjusted for pre-test scores

14. WRAT4 Reading Composite:
Results at post-test (N = 64)
95
96
97
98
99
100
101
102
103
104
105
Pre-test Post-test
MeanStandardScore
Assessment Occasion
Tutoring (n = 30) Control (n = 34)
(g = .29, p = .096, 1-tailed;
post-test scores adjusted for pre-test scores

15. WRAT4 Spelling:
Results at post-test (N = 64)
95
96
97
98
99
100
101
102
103
104
105
Pre-test Post-test
MeanStandardScore
Assessment Occasion
Tutoring (n = 30) Control (n = 34)
(g = -.08, p = .37, 2-tailed, ns;
post-test scores adjusted for pre-test scores)

16. WRAT4 Math Computation:
Results at post-test (N = 64)
83
84
85
86
87
88
89
90
91
92
93
94
95
Pre-test Post-test
MeanStandardScore
Assessment Occasion
Tutoring (n = 30) Control (n = 34)
(g = .46, p = .009, 1-tailed;
post-test scores adjusted for pre-test scores)

17. • Question no. 2:
Do girls and boys benefit equally
from direct-instruction tutoring?
Results

18. WRAT4 Word Reading:
Pre/post change, by gender & condition
90
92
94
96
98
100
102
104
106
108
110
PRE-TEST POST-TEST
TUTORING (n = 17)
CONTROL (n = 19)
* 90
92
94
96
98
100
102
104
106
108
110
PRE-TEST POST-TEST
TUTORING (n = 13)
CONTROL (n = 15)
*
GIRLS (d =.39) BOYS (d = .01)
(*p < .05, 2-tailed)

19. WRAT4 Sentence Comprehension:
Pre/post change, by gender & condition
90
92
94
96
98
100
102
104
106
108
110
PRE-TEST POST-TEST
TUTORING (n = 17)
CONTROL (n = 19)
*
90
92
94
96
98
100
102
104
106
108
110
PRE-TEST POST-TEST
TUTORING (n = 13)
CONTROL (n = 15)
*
GIRLS (d =.12) BOYS (d = .44)
(*p < .05, 2-tailed)

20. 90
92
94
96
98
100
102
104
106
108
110
PRE-TEST POST-TEST
TUTORING (n = 13)
CONTROL (n = 15)
*
GIRLS (d = .25)
90
92
94
96
98
100
102
104
106
108
110
PRE-TEST POST-TEST
TUTORING (n = 17)
CONTROL (n = 19)
*
BOYS (d = .19)
(*p < .05, 2-tailed)
WRAT4 Reading Composite:
Pre/post change, by gender & condition

21. WRAT4 Spelling:
Pre/post change, by gender & condition
GIRLS (d = .15)
90
92
94
96
98
100
102
104
106
108
110
PRE-TEST POST-TEST
TUTORING (n = 17)
CONTROL (n = 19)
*
BOYS (d = .19)
90
92
94
96
98
100
102
104
106
108
110
PRE-TEST POST-TEST
TUTORING (n = 13)
CONTROL (n = 15)
*
(*p < .10, 2-tailed)

22. WRAT4 Math Computation:
Pre/post change, by gender & condition
GIRLS (d = .41)
80
82
84
86
88
90
92
94
96
98
100
PRE-TEST POST-TEST
TUTORING (n = 17)
CONTROL (n = 19)
*
BOYS (d = .21)
80
82
84
86
88
90
92
94
96
98
100
PRE-TEST POST-TEST
TUTORING (n = 13)
CONTROL (n = 15)
*
(*p < .05, 2-tailed)

23. Results – Conclusions
regarding gender effects
Girls:
– Made statistically significant gains on 4 out of 5
WRAT4 outcome measures
– d > median of .29 on Word Reading and Math
Computation
Boys:
– Made statistically significant gains on 3 out of 5
WRAT4 outcome measures
– d > median of .29 on Sentence Comprehension

24. Overall conclusions
• Tutoring by foster parents helps foster
children to catch up in reading and math
• Girls and boys both benefit in reading and
math
• More well-controlled evaluations of
interventions are needed

25. Thank you for your attention
• References: For papers by Forsman &
Vinnerljung (2012), Flynn et al. (2012), and
Harper & Schmidt (2012), see special issue of
Children and Youth Services Review, 34
(6), June, 2012, on improving educational
outcomes of young people in care.
• Contact: Robert Flynn (rflynn@uottawa.ca).
Feel free to write to me by e-mail