AERA 2009 - Variability in U.S. Virtual School Policies Affecting Course Completion Rates
1. U.S. VIRTUAL SCHOOL TRIAL
PERIOD AND COURSE COMPLETION
POLICY STUDY
ABIGAIL GONZALES, BRIGHAM YOUNG UNIVERSITY
MICHAEL K. BARBOUR, WAYNE STATE UNIVERSITY
AERA 2009
2. State of Virtual Schools in U.S.
Growth in schools
1997: 2 states with K12 online learning opportunities
2008: 44 states (Watson, Gemin, & Ryan, 2008)
Majority virtual high school. Trend in K-8 arena.
Growth in student population
40,000 to 50,000 students enrolled in 2000 (Clark, 2001)
1,030,000 students enrolled in 2007-2008. (Picciano& Seaman,
2009)
Primarily supplements brick-and-mortar schooling
Increasingly diverse student body
3. Challenges of virtual schooling
Attrition is a significant problem (Carr, 2000; Lary, 2002;
Rice, 2005)
Researchers call for standardizing performance
measures (Smith et al., 2005; Pape et al., 2006; Watson et al., 2006)
Two studies examining policies (Pape et al., 2006; Watson &
Ryan, 2007)
Findings:
variety in policies adopted
Study weakness: Small sample size 32 and 3 VS
Trial periods can sift out weaker students
(Ballas&Belyk, 2000; Cavanuagh, Gillan, Bosnick, Hess, & Scott, 2005;
McLeod, Hughes, Brown, Choi, & Maeda, 2005)
Variance in policies make comparison of quality
indicator meaningless.
4. Policies Affecting VS Attrition
Rates
When do we count students?
TrialPeriod Policy: period of time where student
withdraws with no penalty.
Begin counting at date of enrollment? After 4
weeks? When submit first assignment?
How do we count students in retention stats?
Course Completion Policy: # students completing
course # students started
course
Who counts as “completing”. Passing grade?
Failing grade? Remains in course?
5. Research Questions
1. What is the variation in trial period policies in
US?
Variability across types schools & geographic regions
2. What is the variation in how US virtual
schools define course completions?
Variability across types schools & geographic regions
6. Methods
Sample
159US schools
Schools listed in
iNACOL‟sOnline Learning Clearinghouse List „07
State-led schools in Keeping Pace w/ K12 Online
Learning (Watson, 2007)
Survey Study
3-question email survey w/ introduction, purpose
Presence of trial period
Length of trial period in days
Definition of successful course completion
7. US Geographical Regions
Northeaste
rn States
Central States
Western
States
Southeaste
rn States
8. Representativeness by Region
Sample Populatio Populatio
Region Sample % n n%
Central States 23 26.1 41 25.5
Northeastern
8 9.1 18 11.2
States
Southeastern
20 22.7 33 20.5
States
Western States 37 42 67 41.6
Total 88 159
9. Responses by School Type
Sampl Sample Populatio Population %
School type
e % n
Cyber
16 18.2 34 21.1
Charter
For Profit 1 1.1 9 5.6
Multi-district 7 8.0 11 6.8
Private 13 14.8 21 13
Single-district 26 29.5 49 30.4
State – led 17 19.3 24 14.9
University –
8 9.1 11 6.8
led
Totals 88 159
13. Trial period length variations
by…
School type:
Sig. @ p=.05 df(5) f3.909
Differences: Private school vs. state-led, cyber
charters, and single-district
Private
schools had shorter trial periods
compared to other schools
Geographical region:
No significant difference
16. Course Completion Definitions where…
Grade is Irrelevant
Definitions US US %
16 18.6
Remain in course
Complete all/majority of
11 12.8
coursework
Totals 27 31.4%
17. Course Completion Definitions where…
Other
Definitions US US %
Mastery not defined by grade 1 1.2
Individual schools define completion 4 4.7
Totals 5 5.9%
19. Findings Summary
Trial Period Presence
Prevalent practice ~70%
Supports Watson & Ryan 2007 survey of 32
VS with ~66%
Trial Period Length
Average length ~ 20 days
Most common lengths: 2 and 4 weeks
Supports Watson & Ryan 2007 findings
Regional differences: Not sig.
School type: Sig. - private schools
20. Findings Summary
Course completion definitions
Wide variation between and within groups
Remain in course
Future Research
Student characteristics, experience, and
reason for dropping out during trial period
duration
Comparison study with Canadian trial period
and course completion policies
21. Implications
Need common metrics for calculating attrition
Best if same as bricks-and-mortar schools
Gather data for internal and external reporting
Internal= Institutional metrics
External = Standardized metrics
Determining metric easier since geography
and school type factor little
1997 only two schools had K12 online programs. 11 years later up to 44 states had significant online learning opportunities according to Watson,Gemin, and Ryan’s 2008 Keeping pace wit hK-12 online learning.Majority of schools are virtual high schools but increasingly more K8 schools are opening.3. Estimated 1,030,000 students enrolled in some type of online program during the 2007-2008 academic year. This is a 47% increase from just two years previously.4. For most students this supplements their face to face or home schooling program. However there is a growing trend towards more full-time programs.3. Student body is expanding and becoming increasingly diverse. This is due to more and more at-risk students using virtual schools for credit recovery and legislative mandates, such as Michigan and Alabama, which require as part of graduation an online learning component. Other states may soon follow.
Long trial periods can act as a sifting mechanism during which weaker students to drop out, masking attrition rates for lower performing students. FLVS 1999 -2000 evaluation report—reported a 71% retention rate. If included dropout students from trial period – 54%In turn, virtual schools with generous trial periods would be able to report high retention rates because students who were having trouble and would have likely struggled to complete the course would have dropped out by the time the virtual school began counting then as students.
159 US schools located:: NACOL Clearinghouse list, State-led schools from Keeping Pace with K-12 Education 2007Canadian schools selected based on Email survey : 3 questions; 2 open-ended,
Single district, Cyber charter, and State –led: 67%
Single district, Cyber charter, and State –led: 67%Fairly representative set of responses compared to the sample set.
US: Of the 88 schools surveyed, 27 schools had no trial period compared to 61 schools had a trial periodTrial periods were a common practice in the US.Several instances where a trial period was marked by an event such as submitting your first assignment, taking your first quiz, paying your tuition; in contrast to a time period that was common in the US.
US most common: 28-30 days about 4 weeks accounted for 28.3% of the sampleMost common (14 days) 14-15 days: 2 weeks accounted for 26.7% of the sample
Ran One way ANOVAs to see if there were any significant differences in trial length based on school type or region.For School types found that US significant with an f value of 3.909Did a Post HokTukey test to see which variables were significant and found that
Wide range from passing the course w/ 60% to a mastery level with 90% or betterUS had significant variation within this category.
Wide range in completion definitions.
Not define by grade: elementary school
This study gives us evidence beyond anecdote or guess, that variations are significant and there is a need to standardize trial period policies and course completion definitions.We need to count students in the same time and same manner.Ideally, best if we could align this with how brick and mortar schools are calculating attrition/retention to allow for comparisons