O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
1 de 17

The RISE Framework: Using learning analytics for the continuous improvement of open educational resources



Baixar para ler offline

We present the Resource Inspection, Selection, and Enhancement (RISE) framework, a learning analytics framework designed to enable teachers to engage in the continuous improvement process. This framework helps identify resources that should be evaluated by a teacher or an instructional designer.

Visit BobBodily.com for more information about my research.

The RISE Framework: Using learning analytics for the continuous improvement of open educational resources

  1. 1. The RISE Framework: Using learning analytics for the continuous improvement of open educational resources Robert Bodily, Rob Nyland, and David Wiley
  2. 2. Project Background ◈ Bob and Rob were members of John Hilton’s OER Research Class at BYU ◈ In the class we were using the COUP framework for OER Research ◆ Cost ◆ Outcomes ◆ Use ◆ Perceptions ◈ We wanted to understand what learning analytics could bring to the conversation around OER
  3. 3. The 5 R’s of OER Retain Reuse Revise Remix Redistribute
  4. 4. “Because the Open Curriculum is licensed in such a way that we can revise materials directly, OHSU is able to engage in a highly data-driven curriculum improvement process. As students go through the online courses, their clicks and learning paths can be tracked and recorded. This information can be combined with item response theory and learning outcome analysis data to set priorities for curriculum or assessment revision empirically. This data-driven process of curriculum improvement should allow the Open Curriculum to reach a very high level of quality very quickly.” - David Wiley, 2009
  5. 5. Continuous Improvement Learning Analytics OER Data-Driven Continuous Improvement
  6. 6. Continuous Improvement at OLI “A key attribute of the OLI environment is that while students are working through the course, we are collecting analytics data and using those data to drive multiple feedback loops” -Strader & Thille (2012)
  7. 7. Resource Inspection, Selection, and Enhancement (RISE) Inspection: Analyze course resources using a variety of automated methods Selection: Identify resources that need additional evaluation Enhancement: Facilitate enhancement through an automated inspection and selection process (enhancement is usually done manually by an instructional designer).
  8. 8. Four Quadrant Evaluation Structure High Grades High student prior knowledge, inherently easy learning outcome, highly effective content, poorly written assessment Effective resources, effective assessment, strong outcome alignment Low Grades Low motivation or high life distraction of students, too much material, technical or other difficulties accessing resources Poorly designed resources, poorly written assessments, poor outcome alignment, difficult learning outcome Low Use High Use
  9. 9. Our Approach: Case Study ◈ Fall 2015 Introduction to Business Course (N = 1002) ◈ OER courseware platform ◆ OER online text ◆ Videos ◆ Simulations ◆ Formative assessments ◆ Practice assessments ◆ Summative assessments ◈ Assessments and resources are all linked to course outcomes
  10. 10. Data Tracking Resource Use Data ◈ Tracked with Google Analytics ◈ Defined as pageviews (other options include time spent and ratings) Assessment Data ◈ Tracked with Open Assessments ◈ Defined as grade on summative assessments
  11. 11. Types of Analyses 1. Outer quadrant analysis 2. Page type analysis 3. Extreme outlier analysis
  12. 12. Outer Quadrant Analysis ◈ Blue dotted line is the additive z-score threshold. ◈ Black lines are the median values for pageviews and grades
  13. 13. Outer Quadrant Analysis Page Title Grade PV/student A.Z-Score Quadrant Reading: The Four Ps of Marketing 0.81 1.64 3.65 1 Outcome: Recruiting Employees 0.88 0.53 3.4 2 Video: Legal Rights Under Implied Warranties 0.53 0.57 3.6 3 Reading: The Organization Chart and Reporting Structure 0.64 1.89 4.56 4
  14. 14. Page Type Analysis Reading Pages Self-Check Pages Outcome Pages Video Pages Simulation Pages
  15. 15. ◈ External search traffic: 1500 pageviews ◈ Course search traffic: 1626 pageviews ◈ Once external search traffic was removed, it was no longer an outlier. Extreme Outlier Analysis Page Title Grade PV/student A.Z-Score Reading: Stages of an Economy 0.83 2.82 7.79
  16. 16. Enhancement ◈ Where the continuous improvement “rubber meets the road” ◈ Start with a theory of the domain ◈ Examine assessments ◆ Evaluate student response patterns / IRT / CFA ◆ Review assessments for design problems (negative language) ◆ Redesign or replace assessments ◈ Move on to content ◆ Examine content for assessment alignment (outcome, words) ◆ Examine content for micro-design problems (mismatch) ◆ Redesign or replace content
  17. 17. Questions?