I report on learning outcomes reported by various schools and districts piloting a comprehensive, multimedia-based Algebra 1 program, distributed openly on the Internet, developed by the Monterey Institute for Technology and Education. We believe that the new remix approach supported by this course can better serve diverse learner needs.

1. 2011 Sloan-C International Conference on Online Learning
Architecture and Impact
of an
Open, Online, Remixable,
and Multimedia-Rich
Algebra 1 Course
Ahrash N Bissell, Ph.D.
Special Projects Manager
Monterey Institute for Technology and Education

2. Our motivations...
• Enable free, personalized learning for every
person.
• Produce resources that are cutting edge and
of demonstrable quality.
• Build a community of educators and
institutions who inform and are empowered by
our work.
• Sustain the enterprise in the face of changing
technical and financial circumstances.

3. Our motivations...

4. Mathematics: a national challenge
Source: Final report of the National Mathematics Advisory Panel

5. Free, personalized learning for every person Free, personalized learning for every person
are
materials, tools, and media
used for teaching and learning
that are
free from copyright restrictions
or publicly licensed
for anyone
to use, adapt, and redistribute.

6. Free, personalized learning for every person Why OER?
http://www.flickr.com/photos/mimax/303567279/ Mimax CC BY-NC-SA
Open educational resources...
●
Bridge formal and informal education
●
Facilitate lifelong and distributed learning
●
Support individualization and localization

7. Free, personalized learning for every person But OER can be hard to navigate...

8. Free, personalized learning for every person … and even harder to use effectively.
CC BY http://www.flickr.com/photos/nickrate/4504163196/

9. Free, personalized learning for every person Our conclusions...
• Resources need to work where teachers and
learners actually use them:
• Online → Hippocampus
• At school → NROC Network
• The resources should be modular → think of
Legos.
• The resources should support any form of
instruction → online, offline, and hybrid.

10. Our conclusions...
Most importantly
Free, personalized learning for every person
--
The resources should
encourage and enable
experimentation and innovation
without the concomitant breakdown of
quality assurances
and
curricular relevance.
Requires attention to both design and distribution.

11. We zeroed in on math...
Project Goal: Increase the number of students that pass Algebra 1 (and
developmental math) as a bridge to a college education
Cutting-edge, high-quality resources
Funding:
The Bill & Melinda Gates Foundation
The William & Flora Hewlett Foundation
Project Audience:
Algebra 1: Ages 13-16, first-time algebra
students
Dev Math: Ages 16-80, students have failed
math at least once

12. … and focused on these objectives.
●
Provide learning activities to support an
efficient path to credit-bearing courses
Cutting-edge, high-quality resources
●
Offer multiple modes of instruction
●
Support multiple curricular standards
●
Allow for flexible course configurations with a
collection of learning objects
●
Leverage the power of digital media

13. Development Approach
Development Approach
Research Focus Groups
Cutting-edge, high-quality resources
Technology Product
& Design
Engineering
Professional Curriculum
Development

14. Development Approach
Development Approach
Research Focus Groups
Cutting-edge, high-quality resources
Technology Product
& Design
Development
Engineering
Professional Curriculum
Development

15. Focus groups
Focus Groups
68 roundtables. >720 students, instructors, and
administrators. Representatives from 153 academic
Cutting-edge, high-quality resources
institutions in 28 states.
Administrators and Faculty
• determine shortcomings of existing teaching materials
• better understand administrator and faculty use cases
• feedback for products in development
• develop pilot sites and adoption interest
Students
• what digital learning materials are they using?
• better understand student use cases
• feedback for products in development

16. Focus Group Key Findings
Administrators and Faculty
Cutting-edge, high-quality resources
 Professional development is essential
for supporting teacher success
 Few instructors and institutions are
satisfied with existing methods or digital
options
 Existing digital products:
 too expensive
 inflexible curricula
 proprietary management system
 High value placed on flexibility,
affordability, multiple pathways for
learners

17. Focus Group Key Findings
Students
Cutting-edge, high-quality resources
 Keep it simple: struggling students
and English language learners value
simplicity
 Real-world examples and application
are key to engagement
 Puzzles, animations, simple
illustrations, and problem sets are
important
 Humor, esoteric example, and idioms
trip-up or confuse struggling students,
especially English language learners

18. Review the Details in our Report
Focus Group Report
Cutting-edge, high-quality resources
Download it under Project News at:

19. The details...
Algebra 1
Instructional elements
Semester 1, Semester 2
Cutting-edge, high-quality resources
At the topic level:
●
warm-up (text)
Designed for first-time algebra learners ●
presentation (video, audio,
(US 8th grade, 15 years old) animation, and graphics)
●
worked examples (audio and
Correlated to 48 state frameworks and the graphics)
Common Core ●
problems (interactive text)
LO architecture: 12 Units, 26 Lessons, 68
●
review (text)
Topics, covering 170 learning objectives ●
text tab (online textbook)
At the unit level:
Designed for online and hybrid classroom use
●
virtual tutor (interactive video,
graphics, text)
Single-session formative assessment,
multi-session CMS-based ●
project (text and graphics)
●
puzzles (game-based, interactive
Rich, interactive multimedia with multiple
instructional elements designed animation, graphics)
for 15 year-old audience

20. The details...
Instructional Elements
At the topic level:
●
warm-up (text)
●
presentation (video, audio, animation, and
graphics)
●
worked examples (audio and graphics)
●
problems (interactive text)
●
review (text)
●
text tab (online textbook)
At the unit level:
●
virtual tutor (interactive video, graphics,
text)
●
project (text and graphics)
●
puzzles (game-based, interactive
animation, graphics)

21. The details...
Instructional Elements
At the topic level:
●
warm-up (text)
●
presentation (video, audio, animation, and
graphics)
●
worked examples (audio and graphics)
●
problems (interactive text)
●
review (text)
●
text tab (online textbook)
At the unit level:
●
virtual tutor (interactive video, graphics,
text)
●
project (text and graphics)
●
puzzles (game-based, interactive
animation, graphics)

22. The details...
Instructional Elements
At the topic level:
●
warm-up (text)
●
presentation (video, audio, animation,
and graphics)
●
worked examples (audio and graphics)
●
problems (interactive text)
●
review (text)
●
text tab (online textbook)
At the unit level:
●
virtual tutor (interactive video, graphics,
text)
●
project (text and graphics)
●
puzzles (game-based, interactive
animation, graphics)

23. The details...
Instructional Elements
At the topic level:
●
warm-up (text)
●
presentation (video, audio, animation, and
graphics)
●
worked examples (audio and graphics)
●
problems (interactive text)
●
review (text)
●
text tab (online textbook)
At the unit level:
●
virtual tutor (interactive video, graphics,
text)
●
project (text and graphics)
●
puzzles (game-based, interactive
animation, graphics)

24. The details...
Instructional Elements
At the topic level:
●
warm-up (text)
●
presentation (video, audio, animation, and
graphics)
●
worked examples (audio and graphics)
●
problems (interactive text)
●
review (text)
●
text tab (online textbook)
At the unit level:
●
virtual tutor (interactive video, graphics,
text)
●
project (text and graphics)
●
puzzles (game-based, interactive
animation, graphics)

25. The details...
Instructional Elements
At the topic level:
●
warm-up (text)
●
presentation (video, audio, animation, and
graphics)
●
worked examples (audio and graphics)
●
problems (interactive text)
●
review (text)
●
text tab (online textbook)
At the unit level:
●
virtual tutor (interactive video, graphics,
text)
●
project (text and graphics)
●
puzzles (game-based, interactive
animation, graphics)

26. The details...
Instructional Elements
At the topic level:
●
warm-up (text)
●
presentation (video, audio, animation, and
graphics)
●
worked examples (audio and graphics)
●
problems (interactive text)
●
review (text)
●
text tab (online textbook)
At the unit level:
●
virtual tutor (interactive video, graphics,
text)
●
project (text and graphics)
●
puzzles (game-based, interactive
animation, graphics)

27. The details...
Instructional Elements
At the topic level:
●
warm-up (text)
●
presentation (video, audio, animation, and
graphics)
●
worked examples (audio and graphics)
●
problems (interactive text)
●
review (text)
●
text tab (online textbook)
At the unit level:
●
virtual tutor (interactive video, graphics,
text)
●
project (text and graphics)
●
puzzles (game-based, interactive
animation, graphics)

28. The details...
Instructional Elements
At the topic level:
●
warm-up (text)
●
presentation (video, audio, animation, and
graphics)
●
worked examples (audio and graphics)
●
problems (interactive text)
●
review (text)
●
text tab (online textbook)
At the unit level:
●
virtual tutor (interactive video, graphics,
text)
●
project (text and graphics)
●
puzzles (game-based, interactive
animation, graphics)

29. The details...
Instructional Elements
At the topic level:
●
warm-up (text)
●
presentation (video, audio, animation, and
graphics)
●
worked examples (audio and graphics)
●
problems (interactive text)
●
review (text)
●
text tab (online textbook)
At the unit level:
●
virtual tutor (interactive video, graphics,
text)
●
project (text and graphics)
●
puzzles (game-based, interactive
animation, graphics)

30. http://nroc.remote-learner.net/

31. A focus on distribution...
Institutions → NROCNetwork.org
Individuals → HippoCampus.org

32. … which builds community...
The NROC Network
A community of educators and institutions
... is comprised of educators, designers,
technologists, and administrators working
together to promote the continuous improvement
of online courses through collaborative
development of high-quality content and
instruction.

33. A community of educators and institutions … and is flexible...
Usable within
popular LMS’s
and repositories.
Customizable by a teacher, a district, or a state.

34. A community of educators and institutions … while maintaining curricular relevance...
Correlated with
state standards
and popular
textbooks.
Complete
teaching
materials.

35. … and sustaining our work.
Academy School District 20 Online (CO) Illinois Virtual School (DOE) Pan American International High School
Alabama ACCESS (DOE) Indian Prairie School District #204 (IL) Pierce College
Alameda County Office of Education Iowa Community Colleges Online Consortium Placer Union High School District
Albuquerque Public Schools (NM) Iowa AEA Consortium Prince Georges County Public School (MD)
A community of educators and institutions
All Saints Day School (CA) IvyTech Charter Schools Portland State University
Anaheim Union High School District (CA) JFY Networks Poway Unified School District
Arizona Department of Education Joliet Junior College RAI Online Charter School
Bambugare Network (Sub-Saharan Africa) Jupiter Education Services Riverside Unified School District (CA)
Bay Port High School Lee’s Summit R7 School District Salish Kootenai College (MT)
Bishop O’Dowd High School Los Angeles Unified School District San Diego County Office of Education
Bethel Covenant College (Uganda) Louisiana Virtual School San Jose Education Foundation (CA)
Berkeley Tutoring Coalition Lubbock Independent School District Online (TX) San Luis Obispo County Board of Education (CA)
Butler Technology and Career Development Maryland State Department of Education Santa Barbara County Education Office
California State University at Fullerton Mason City School District (OH) School District of the Chathams (NJ)
Catholic School K12 Virtual Michigan Virtual High School Schenectady City Schools
Chattanooga State Tech.& Comm. College (TN) Minnesota Learning Commons (statewide P-20) School & Main Institute
Chesterfield School District (VA) Minneapolis Public Schools Online SIATech
CORE (China) Mississippi Virtual School (DOE) Sierra Vista High School (CA)
Corona-Norco Unified School District Missouri Dept. of Elem. and Secondary Education Sinclair Community College
Colorado Community Colleges Online Montana Digital Academy South Carolina Virtual School (DOE)
Colorado Online Learning Montana State University at Billings Southlands Christian Schools
CUDI (Mexico) Montgomery County Public Schools (VA) Tulare County Office of Education (CA)
Culver City Unified School District MSD of Wayne Township Ulster County BOCES
eMINTS/Missouri Myron B. Thompson Academy (HI) University of Alaska at Fairbanks
Exeter Public Schools (CA) National Association of Beginning Teachers University of California, Irvine
Fairmont Private Schools Niles Township High School District (IL) University of Georgia System Board of Regents
Florida Distance Learning Consortium Niswonger Foundation Universidad de Guadalajara (MX)
Forest Charter School Nebraska: Partnerships for Innovation (statewide) Utah Electronic High School
Georgia Virtual School (DOE) New York City Department of Education University of Texas El Paso
GLBTQ Online High School (MN) Nobel Learning Communities University of Texas at Brownsville
Global Literacy Foundation (AZ) North Carolina Community College System University of Texas at Dallas
Grand Rapids Public Schools North Dakota Center for Distance Education Valley Christian School (CA)
Greenville County Virtual School (SC) North Salinas High School (CA) Village Christian Schools
Grossmont Union High School (CA) Northwest Area Education Agency (IA) Virtual Virginia (DOE)
Gwinnett County Online Campus (GA) Odyssey Charter Schools (NV) West Virginia DOE
Horry County Virtual School OpenEd Solutions Whitfield County School District (GA)
Kentucky Statewide Consortium (statewide) Oregon Department of Education
Hudson Schools of Technology (NJ) Open High School of Utah (NV) Partner Advisors:
Idaho Digital Learning Academy OTAN Council of Chief State Supervisors and
IDEAL-New Mexico Palmetto State e-Cadamy Officers (CCSSO)
Southern Regional Education Board (SREB)
Data as of July 2011

36. Free, personalized learning for every person Classic Hippocampus

37. HippoCampus Classic
24 hours

38. Instructor Feedback Survey, February 2011

39. Free, personalized learning for every person The new Hippocampus
Link

40. Free, personalized learning for every person
Easy to navigate.

41. Free, personalized learning for every person
Easy to resize. Includes closed captions.

42. Free, personalized learning for every person
Khan Academy worked examples.

43. Free, personalized learning for every person
PhET interactive simulations.

44. Free, personalized learning for every person
Interactive tutors at the unit level.

45. Free, personalized learning for every person
PE!!
And playlists for customized pathways.

46. Free, personalized learning for every person
Even the site itself can be customized.

47. Free, personalized learning for every person
The institutional and personalized channels
merge on branded Hippocampus sites.

48. Why this design?
http://www.flickr.com/photos/maxx361/4205316487/
http://www.flickr.com/photos/qthomasbower/3640362081/
http://www.flickr.com/photos/23021987@
N06/4040082375/in/pool-638814@N24/
An object
size...
...that enables
creativity with less risk.

49. Research and outcomes How are we doing...?
250,000 visits/month
8 million page views/month
25 minutes per visit average

50. How are we doing...?
Algebra 1, Semesters 1 and 2 published, March 2011.
Dev Math Arithmetic beta released, June 2011.
34 Algebra 1 research pilots nationwide (54 more queued).
20 Dev Math Arithmetic pilots queued in 15 states.
MITE Algebra 1 Pilot Study Evaluation
Research and outcomes
Draft, June 30, 2011
Algebra 1 published to HippoCampus,
August 2011.
Documentary film on use of of Algebra 1
at Sierra Vista Alt. HS (Whittier USD).
NROC Mobile Math App under dev.

51. But what about learning outcomes?
We are still gathering data from our pilot projects.
We are striving for maximum variation in
implementations, including along the following
categories of interest:
DISTRICT STATISTICS INSTRUCTIONAL DESIGNS
●
District Enrollment ●
Classroom‐Based Instruction
Research and outcomes
●
Teacher FTE ●
Blended/Hybrid Learning
●
Years of NROC ●
Accelerated Course
Membership ●
Credit Recovery
●
Learning Management ●
Online Independent Study
System and Learning ●
Individualized Instruction
Object Repository ●
Asynchronous Learning
●
Self‐Paced Learning
●
Online Teacher‐Led
●
Remedial Education
●
Adult Education / GED

52. But what about learning outcomes?
A detailed look at one pilot study:
Sierra Vista Alternative High School
●
HS juniors and seniors
●
62% credit deficiencies, 80% teen parents, 20% homeless
Research and outcomes
●
Most repeating Algebra (school data)
●
Hispanic 94%, White 13%; 75% low income
●
Whittier USD district algebra benchmark unit tests
●
Traditional classroom: textbook, lecture
●
Hybrid-Lab: NROC Alg, teacher-coach, self-paced

53. Research and outcomes Sierra Vista Pilot Study

54. Research and outcomes Sierra Vista Pilot Study
Student ID (anonymized)

55. Research and outcomes Sierra Vista Pilot Study
Student ID (anonymized)

56. Sierra Vista Pilot Study
Results -- preliminary, 40 students
Algebra benchmark test passing score = 60%
Traditional class
Research and outcomes
●
Test on a schedule
●
Half scored below 60%,
●
One student scored over 80%
Hybrid class: NROC Alg 1
●
Test when ready
●
95% scored 60% or better
●
Half scored over 80%

57. Sierra Vista Pilot Study
Results -- preliminary, 40 students
Algebra benchmark test passing score = 60%
Research and outcomes
Hybrid class: NROC Alg 1
●
Test when ready
●
95% scored 60% or better
●
Half scored over 80%
Play video here

58. More studies are under way...
www.NROCmath.org
Research and outcomes

59. A set of rules that describes
our reactions to technologies:
1. Anything that is in the world when you're born
Douglass Adams. The Salmon of Doubt. 2002.
is normal and ordinary and is just a natural
part of the way the world works.
2. Anything that's invented between when you're
fifteen and thirty-five is new and exciting and
revolutionary and you can probably get a
career in it.
3. Anything invented after you're thirty-five is
against the natural order of things.
Whom do we serve, and how?

60. Questions?
abissell@montereyinstitute.org