SlideShare uma empresa Scribd logo
1 de 28
Baixar para ler offline
© 2016 Boise State University 1
Kerry Rice, Jui-Long Hung, Yu-Chang Hsu, Brett E. Shelton
Department of Educational Technology
Boise State University
Educational Data Mining in Program
Evaluation: Lessons Learned
AECT 2016, Las Vegas
© 2016 Boise State University 2
MET
Ed. D.
Ed. S.
Graduate Certificates:
Online Teaching
Technology Integration Specialist
School Technology Coordinator
K-12 Online Teaching Endorsement
College of Education
© 2016 Boise State University 3
Go Broncos!
© 2016 Boise State University 4
Decision Tree
Analysis
(performance
prediction)
Cluster Analysis
(engagement)
Sequential
Association
Analysis (path
analysis)
Educational Data Mining Applications
Time Series
Analysis
(future
performance
prediction)
© 2016 Boise State University 5
Study #1: Teacher Training Workshops 2010
• Survey Data + Data
Mining + Student
Outcomes
• Research Goal:
– Program
improvement
– Satisfaction
– Impact on practice
• Blackboard
• 103 participants
• 31,417 learning logs
• Cluster Analysis;
Sequential Association
Analysis; Decision Tree
Analysis;
© 2016 Boise State University 6
Study #2: Online Graduate Teacher Education 2010
• Data Mining + Student
Outcomes (no demographic
data)
• Research Goal:
– Identify struggling students
– Adjust teaching strategies
– Improve course design
– Data Visualization
• Study Design
– Comparative (between and
within courses)
– Random course selection
• Moodle
• Two graduate courses
(X and Y)
• Each with two sections
– X1 (18 students)
– X2 (19 students)
– Y1 (18 students)
– Y2 (22 students)
• 2,744,433 server logs
• Cluster Analysis; Sequential
Association Analysis;
Decision Tree Analysis; Data
Visualization
© 2016 Boise State University 7
• Data mining + Demographics +
Survey Data + Student
Outcomes
• Research Goal: Large scale
program evaluation
– Support decision making at the
course and institutional level
– Identify key variables and
relationships between teacher
and course satisfaction, student
behaviors, and performance
outcomes
Study #3: End of Year K-12 Online Program Evaluation 2012
• Blackboard LMS
• 7500 students
• 883 courses
• 23,854,527 learning logs
(over 1 billion records)
• Cluster Analysis; Decision
Tree Analysis
© 2016 Boise State University 8
Study #4: End of Year K-12 Blended Program Evaluation 2012
• Blackboard LMS
• 255 Enrollments
• 33 course sections
• 17 unique courses
• Satisfaction Survey
• Data from 2011 pilot
study
• Cluster Analysis;
Decision Tree Analysis
• Demographics + Survey Data +
Data Mining + Student
Outcomes
• Research Goal: Test Framework
in Blended Learning
– Support decision making at the
course and institutional level?
– Identify key variables and
relationships between teacher
and course satisfaction, student
behaviors, and performance
outcomes
© 2016 Boise State University 9
Study #5: Online Graduate Teacher Education, 2014
• Moodle LMS
• 509 Enrollments
• 25 course sections
• 12 unique courses
• 431,708 records
• Time Series Analysis
• 34 original and derived
variables
–static (demographic)
–dynamic (engagement)
• Demographics + Data
Mining + Student
Outcomes
• Research Goal:
– Could we identify at-risk
students in real time?
– When?
© 2016 Boise State University 10
Study #6: Online Graduate Teacher Education, 2015
• Moodle LMS
• 661 Enrollments
• 31 course sections
• 18 unique courses
• 546,965 records
• Time Series Analysis
• 34 original and derived
variables
–static (demographic)
–dynamic (engagement)
• Demographics + Data
Mining + Student
Outcomes
• Research Goal:
– Did the model developed in
Study #5 work with new
semester data?
– Were the predictive (timing)
results the same?
– Were frequency data more
predictive?
© 2016 Boise State University 11
Variables - actual and derived
Engagement
© 2016 Boise State University 12
Cluster Analysis
Student clustering which describes
shared characteristics of students who passed or failed their courses
© 2016 Boise State University 13
Cluster Analysis:
Relative Participation Levels and Final Grades
• Average Time Spent
• Average Days Participated
• Average Frequency of Mouse Clicks
• Average Time Spent per Session
• Average Frequency of Mouse Clicks per Session
Participation Variables
(Engagement)
© 2016 Boise State University 14
Cluster Analysis: Student Characteristics
Cluster 1 – Low-High, 119 students: Low average participation and higher
performance levels.
Cluster 2 – High-High, 60 students: High average participation and high
performance levels.
Cluster 3 – Low-Low, 76 students (46% remedial): Low average
participation and low performance levels.
© 2016 Boise State University 15
Cluster Distributions in Courses
High percentage of
Low-Low students in
Chemistry CP, CP
Pre-Calculus,
English II, English III,
and Pre-Calculus (H)
© 2016 Boise State University 16
Decision Tree Analysis
Perception and performance predictions which identify key predictors of
course satisfaction, instruction satisfaction, and final grade
© 2016 Boise State University 17
Decision Tree Analysis:
Predictors of Student Performance
Decision Tree Analysis
• Identified at risk
• Number of Courses Taken
• Average Clicks per Week
• Average Time Spent per
Week, and
• Average Time Spent per
Session
Other factors
• Gender
• Ethnicity
• Reason for taking a course
© 2016 Boise State University 18
Sequential Association Analysis
Course X Course Y
Does the design of the course (path to learning) predict learner outcomes?
© 2016 Boise State University 19
Sequential Association Analysis
© 2016 Boise State University 20
Time Series Analysis
© 2016 Boise State University 21
Time Series Analysis – course access
A
B
F
Week 10 Spring Break
© 2016 Boise State University 22
Time Series Analysis – DB Replies
A
B
F
Week 10 Spring Break
© 2016 Boise State University 23
Overall Analysis
• Students who took fewer courses performed significantly better than
those who took more courses.
• Engagement is a significant factor. High-engaged students performed
better than low-engaged students.
• Students identified as at risk performed differently than all other
students.
• Type of engagement matters. Students who accessed their courses more
often performed better than those who had more interactions within the
course. Consistent interaction over time is a better predictor of
performance. (higher ed only)
• Advanced courses - High-engagement and high performance (K-12)
• Entry level courses - Low performance regardless of engagement (K-12)
• Gender and ethnicity (higher ed) were identified as significant factors
• Satisfaction did not always equate to higher performance
© 2016 Boise State University 24
Characteristics of successful students
• Female (k-12)
• Younger (k-12)
• Were enrolled in advanced courses (k-12)
• Took fewer courses
• Were more engaged overall
• Were consistently engaged
© 2016 Boise State University 25
Characteristics of at-risk students
• Male (K-12)
• Older (K-12)
• Took entry-level courses (K-12)
• Took a greater number of courses
• Were low engaged overall
• Were inconsistent in their engagement
© 2016 Boise State University 26
Data Collection Challenges
• Bb activity accumulator grouped wide ranging behaviors into
only five useful categories
• Missing data (empty fields – ex. internal handler)
• Mismatched data fields/data stored in the wrong fields
• Inconsistent data collection (i.e. failure to track every forum
reply)
• Partial or missing timestamp (needed for sequential analysis)
• Course or student ID not linked to survey
• Demographic data not linked to course or program
• Inconsistent course models (blended)
© 2016 Boise State University 27
Educational Data Mining
Special Challenges
• Learning behaviors are complex
• Target variables (learning
outcomes/performance) require
wide range of assessments and
indicators
• Goal of improving teaching and
learning is hard to quantify
• Limited number of DM techniques
suitable to meet educational goals
• Only interactions that occur in the
LMS can be tracked through data
mining
• Still a very intensive process to
identify rules and patterns
© 2016 Boise State University 28
References
•Hung, J. L., Rice, K., & Saba, A. (2012). An educational data mining model for online
teaching and learning. Journal of Educational Technology Development and Exchange,
5(2), 77-94.
•Hung, J. L., Hsu, Y.-C., & Rice, K. (2012). Integrating data mining in program
evaluation of K-12 online education. Educational Technology & Society, 15(3), 27-41.
•Rice, K., & Hung. J. (2015). Data mining in online professional development program:
An exploratory case study. International Journal of Technology in Teaching and
Learning, 11(1), 1-20.
•Shelton, B., Hung, J. L., & Baughman, S. (2015). Online graduate teacher education:
Establishing and EKG for student success intervention. Technology, Knowledge and
Learning.
•Rice, K., & Hung, J. L. (2015). Identifying variables important to the success of K-12
students in blended learning. Paper presented at the Northern Rocky Mountain
Educational Research Association Conference, Boise, Idaho.
•Shelton, B. E., Hung, J. L.., & Lowenthal, P. (under review). Predicting student success
by modeling student interaction in online courses.

Mais conteúdo relacionado

Mais procurados

Every Teacher is a Counselor_Sufia Azmat_2014
Every Teacher is a Counselor_Sufia Azmat_2014Every Teacher is a Counselor_Sufia Azmat_2014
Every Teacher is a Counselor_Sufia Azmat_2014ZaynabA
 
Educational research need concept and types by purpose
Educational research need concept and types by purposeEducational research need concept and types by purpose
Educational research need concept and types by purposeInam Jahangir
 
Obe training final
Obe training finalObe training final
Obe training finalErNITIN3
 
Research proposal defense-powerpoint-
Research proposal defense-powerpoint-Research proposal defense-powerpoint-
Research proposal defense-powerpoint-sanimuhammad001
 

Mais procurados (7)

Every Teacher is a Counselor_Sufia Azmat_2014
Every Teacher is a Counselor_Sufia Azmat_2014Every Teacher is a Counselor_Sufia Azmat_2014
Every Teacher is a Counselor_Sufia Azmat_2014
 
Blended Learning
Blended Learning Blended Learning
Blended Learning
 
Educational research need concept and types by purpose
Educational research need concept and types by purposeEducational research need concept and types by purpose
Educational research need concept and types by purpose
 
Regression Analysis
Regression AnalysisRegression Analysis
Regression Analysis
 
Obe training final
Obe training finalObe training final
Obe training final
 
Outcome based Education
Outcome based EducationOutcome based Education
Outcome based Education
 
Research proposal defense-powerpoint-
Research proposal defense-powerpoint-Research proposal defense-powerpoint-
Research proposal defense-powerpoint-
 

Destaque

Performance Assessment of Faculties of Management Discipline From Student Per...
Performance Assessment of Faculties of Management Discipline From Student Per...Performance Assessment of Faculties of Management Discipline From Student Per...
Performance Assessment of Faculties of Management Discipline From Student Per...Waqas Tariq
 
An insight into Educational Data Mining at Muğla Sıtkı Koçman University, Turkey
An insight into Educational Data Mining at Muğla Sıtkı Koçman University, TurkeyAn insight into Educational Data Mining at Muğla Sıtkı Koçman University, Turkey
An insight into Educational Data Mining at Muğla Sıtkı Koçman University, Turkeystrehlst
 
Knowledge Discovery from Academic Data using Association Rule Mining, Paper P...
Knowledge Discovery from Academic Data using Association Rule Mining, Paper P...Knowledge Discovery from Academic Data using Association Rule Mining, Paper P...
Knowledge Discovery from Academic Data using Association Rule Mining, Paper P...shibbirtanvin
 
STUDENT PERFORMANCE ANALYSIS USING DECISION TREE
STUDENT PERFORMANCE ANALYSIS USING DECISION TREESTUDENT PERFORMANCE ANALYSIS USING DECISION TREE
STUDENT PERFORMANCE ANALYSIS USING DECISION TREEAkshay Jain
 
Predicting Student Performance in Solving Parameterized Exercises
Predicting Student Performance in Solving Parameterized ExercisesPredicting Student Performance in Solving Parameterized Exercises
Predicting Student Performance in Solving Parameterized ExercisesShaghayegh (Sherry) Sahebi
 
USING LEARNING ANALYTICS TO PREDICT STUDENTS’ PERFORMANCE IN MOODLE LMS
USING LEARNING ANALYTICS TO PREDICT STUDENTS’ PERFORMANCE IN MOODLE LMSUSING LEARNING ANALYTICS TO PREDICT STUDENTS’ PERFORMANCE IN MOODLE LMS
USING LEARNING ANALYTICS TO PREDICT STUDENTS’ PERFORMANCE IN MOODLE LMSAfrican Virtual University
 
Data mining to predict academic performance.
Data mining to predict academic performance. Data mining to predict academic performance.
Data mining to predict academic performance. Ranjith Gowda
 
Final project viva presentation
Final project   viva presentationFinal project   viva presentation
Final project viva presentationteshan
 
DATA MINING TOOL- ORANGE
DATA MINING TOOL- ORANGEDATA MINING TOOL- ORANGE
DATA MINING TOOL- ORANGENeeraj Goswami
 

Destaque (9)

Performance Assessment of Faculties of Management Discipline From Student Per...
Performance Assessment of Faculties of Management Discipline From Student Per...Performance Assessment of Faculties of Management Discipline From Student Per...
Performance Assessment of Faculties of Management Discipline From Student Per...
 
An insight into Educational Data Mining at Muğla Sıtkı Koçman University, Turkey
An insight into Educational Data Mining at Muğla Sıtkı Koçman University, TurkeyAn insight into Educational Data Mining at Muğla Sıtkı Koçman University, Turkey
An insight into Educational Data Mining at Muğla Sıtkı Koçman University, Turkey
 
Knowledge Discovery from Academic Data using Association Rule Mining, Paper P...
Knowledge Discovery from Academic Data using Association Rule Mining, Paper P...Knowledge Discovery from Academic Data using Association Rule Mining, Paper P...
Knowledge Discovery from Academic Data using Association Rule Mining, Paper P...
 
STUDENT PERFORMANCE ANALYSIS USING DECISION TREE
STUDENT PERFORMANCE ANALYSIS USING DECISION TREESTUDENT PERFORMANCE ANALYSIS USING DECISION TREE
STUDENT PERFORMANCE ANALYSIS USING DECISION TREE
 
Predicting Student Performance in Solving Parameterized Exercises
Predicting Student Performance in Solving Parameterized ExercisesPredicting Student Performance in Solving Parameterized Exercises
Predicting Student Performance in Solving Parameterized Exercises
 
USING LEARNING ANALYTICS TO PREDICT STUDENTS’ PERFORMANCE IN MOODLE LMS
USING LEARNING ANALYTICS TO PREDICT STUDENTS’ PERFORMANCE IN MOODLE LMSUSING LEARNING ANALYTICS TO PREDICT STUDENTS’ PERFORMANCE IN MOODLE LMS
USING LEARNING ANALYTICS TO PREDICT STUDENTS’ PERFORMANCE IN MOODLE LMS
 
Data mining to predict academic performance.
Data mining to predict academic performance. Data mining to predict academic performance.
Data mining to predict academic performance.
 
Final project viva presentation
Final project   viva presentationFinal project   viva presentation
Final project viva presentation
 
DATA MINING TOOL- ORANGE
DATA MINING TOOL- ORANGEDATA MINING TOOL- ORANGE
DATA MINING TOOL- ORANGE
 

Semelhante a Educational Data Mining in Program Evaluation: Lessons Learned

Katarina Thomson and Karl Molden - Turning Course Evaluation into Information
Katarina Thomson and Karl Molden - Turning Course Evaluation into InformationKatarina Thomson and Karl Molden - Turning Course Evaluation into Information
Katarina Thomson and Karl Molden - Turning Course Evaluation into InformationAssociation of University Administrators
 
Learner Engagement, QM Standards and the Blended Course
Learner Engagement, QM Standards and the Blended CourseLearner Engagement, QM Standards and the Blended Course
Learner Engagement, QM Standards and the Blended CourseLakeland Community College
 
Open Academic Analytics Initiative - Campus Technology Innovator Award Presen...
Open Academic Analytics Initiative - Campus Technology Innovator Award Presen...Open Academic Analytics Initiative - Campus Technology Innovator Award Presen...
Open Academic Analytics Initiative - Campus Technology Innovator Award Presen...Joshua
 
ITHAKA The Next Wave 2017: Jade Winn - Student Success Research: Information ...
ITHAKA The Next Wave 2017: Jade Winn - Student Success Research: Information ...ITHAKA The Next Wave 2017: Jade Winn - Student Success Research: Information ...
ITHAKA The Next Wave 2017: Jade Winn - Student Success Research: Information ...ITHAKA
 
College Textbook Affordability Student Survey Findings
College Textbook Affordability Student Survey FindingsCollege Textbook Affordability Student Survey Findings
College Textbook Affordability Student Survey FindingsUna Daly
 
btc2018-19informationwebinarfinal.pptx
btc2018-19informationwebinarfinal.pptxbtc2018-19informationwebinarfinal.pptx
btc2018-19informationwebinarfinal.pptxSapnaSoni30
 
Expanding opportunities a k12 strategic approach slide share
Expanding opportunities a k12 strategic approach slide shareExpanding opportunities a k12 strategic approach slide share
Expanding opportunities a k12 strategic approach slide shareaccoker
 
Assessing the Impact of Mentoring: Lessons Learned from a Research Study in W...
Assessing the Impact of Mentoring: Lessons Learned from a Research Study in W...Assessing the Impact of Mentoring: Lessons Learned from a Research Study in W...
Assessing the Impact of Mentoring: Lessons Learned from a Research Study in W...ICF
 
The Higher Ed Canvas: Connecting Challenges and Tools
The Higher Ed Canvas: Connecting Challenges and ToolsThe Higher Ed Canvas: Connecting Challenges and Tools
The Higher Ed Canvas: Connecting Challenges and ToolsChristina Sax
 
Professor Graham Gibbs Presentation
Professor Graham Gibbs PresentationProfessor Graham Gibbs Presentation
Professor Graham Gibbs PresentationChris McEwan
 
Data Collection and Accessibility for Inclusive Excellence
Data Collection and Accessibility for Inclusive ExcellenceData Collection and Accessibility for Inclusive Excellence
Data Collection and Accessibility for Inclusive ExcellencePresence
 
Counternarratives and HBCU Student Success - NASPA 3.24.15
Counternarratives and HBCU Student Success - NASPA 3.24.15Counternarratives and HBCU Student Success - NASPA 3.24.15
Counternarratives and HBCU Student Success - NASPA 3.24.15saUGA411
 
Pam Muth and Lisa Bolton: Optimising QILT to improve the student experience
Pam Muth and Lisa Bolton: Optimising QILT to improve the student experiencePam Muth and Lisa Bolton: Optimising QILT to improve the student experience
Pam Muth and Lisa Bolton: Optimising QILT to improve the student experienceStudiosity.com
 
The power of learning analytics to measure learning gains: an OU, Surrey and ...
The power of learning analytics to measure learning gains: an OU, Surrey and ...The power of learning analytics to measure learning gains: an OU, Surrey and ...
The power of learning analytics to measure learning gains: an OU, Surrey and ...Bart Rienties
 
NASPA.FV.AVPSSCollectiveImpact.AND.Outlook
NASPA.FV.AVPSSCollectiveImpact.AND.OutlookNASPA.FV.AVPSSCollectiveImpact.AND.Outlook
NASPA.FV.AVPSSCollectiveImpact.AND.OutlookRenee Delgado-Riley
 
Standing at the Intersection of College Choice and Competency-based Education...
Standing at the Intersection of College Choice and Competency-based Education...Standing at the Intersection of College Choice and Competency-based Education...
Standing at the Intersection of College Choice and Competency-based Education...American Public University System
 
Star Pilot Project Review of Year 1
Star Pilot Project Review of Year 1Star Pilot Project Review of Year 1
Star Pilot Project Review of Year 1Texas ACE
 

Semelhante a Educational Data Mining in Program Evaluation: Lessons Learned (20)

Katarina Thomson and Karl Molden - Turning Course Evaluation into Information
Katarina Thomson and Karl Molden - Turning Course Evaluation into InformationKatarina Thomson and Karl Molden - Turning Course Evaluation into Information
Katarina Thomson and Karl Molden - Turning Course Evaluation into Information
 
Learner Engagement, QM Standards and the Blended Course
Learner Engagement, QM Standards and the Blended CourseLearner Engagement, QM Standards and the Blended Course
Learner Engagement, QM Standards and the Blended Course
 
Open Academic Analytics Initiative - Campus Technology Innovator Award Presen...
Open Academic Analytics Initiative - Campus Technology Innovator Award Presen...Open Academic Analytics Initiative - Campus Technology Innovator Award Presen...
Open Academic Analytics Initiative - Campus Technology Innovator Award Presen...
 
ITHAKA The Next Wave 2017: Jade Winn - Student Success Research: Information ...
ITHAKA The Next Wave 2017: Jade Winn - Student Success Research: Information ...ITHAKA The Next Wave 2017: Jade Winn - Student Success Research: Information ...
ITHAKA The Next Wave 2017: Jade Winn - Student Success Research: Information ...
 
College Textbook Affordability Student Survey Findings
College Textbook Affordability Student Survey FindingsCollege Textbook Affordability Student Survey Findings
College Textbook Affordability Student Survey Findings
 
WPU Scholarship Day 2013
WPU Scholarship Day 2013WPU Scholarship Day 2013
WPU Scholarship Day 2013
 
btc2018-19informationwebinarfinal.pptx
btc2018-19informationwebinarfinal.pptxbtc2018-19informationwebinarfinal.pptx
btc2018-19informationwebinarfinal.pptx
 
Expanding opportunities a k12 strategic approach slide share
Expanding opportunities a k12 strategic approach slide shareExpanding opportunities a k12 strategic approach slide share
Expanding opportunities a k12 strategic approach slide share
 
Assessing the Impact of Mentoring: Lessons Learned from a Research Study in W...
Assessing the Impact of Mentoring: Lessons Learned from a Research Study in W...Assessing the Impact of Mentoring: Lessons Learned from a Research Study in W...
Assessing the Impact of Mentoring: Lessons Learned from a Research Study in W...
 
ABCs of Research
ABCs of ResearchABCs of Research
ABCs of Research
 
The Higher Ed Canvas: Connecting Challenges and Tools
The Higher Ed Canvas: Connecting Challenges and ToolsThe Higher Ed Canvas: Connecting Challenges and Tools
The Higher Ed Canvas: Connecting Challenges and Tools
 
Professor Graham Gibbs Presentation
Professor Graham Gibbs PresentationProfessor Graham Gibbs Presentation
Professor Graham Gibbs Presentation
 
Data Collection and Accessibility for Inclusive Excellence
Data Collection and Accessibility for Inclusive ExcellenceData Collection and Accessibility for Inclusive Excellence
Data Collection and Accessibility for Inclusive Excellence
 
Counternarratives and HBCU Student Success - NASPA 3.24.15
Counternarratives and HBCU Student Success - NASPA 3.24.15Counternarratives and HBCU Student Success - NASPA 3.24.15
Counternarratives and HBCU Student Success - NASPA 3.24.15
 
Pam Muth and Lisa Bolton: Optimising QILT to improve the student experience
Pam Muth and Lisa Bolton: Optimising QILT to improve the student experiencePam Muth and Lisa Bolton: Optimising QILT to improve the student experience
Pam Muth and Lisa Bolton: Optimising QILT to improve the student experience
 
The power of learning analytics to measure learning gains: an OU, Surrey and ...
The power of learning analytics to measure learning gains: an OU, Surrey and ...The power of learning analytics to measure learning gains: an OU, Surrey and ...
The power of learning analytics to measure learning gains: an OU, Surrey and ...
 
NASPA.FV.AVPSSCollectiveImpact.AND.Outlook
NASPA.FV.AVPSSCollectiveImpact.AND.OutlookNASPA.FV.AVPSSCollectiveImpact.AND.Outlook
NASPA.FV.AVPSSCollectiveImpact.AND.Outlook
 
Standing at the Intersection of College Choice and Competency-based Education...
Standing at the Intersection of College Choice and Competency-based Education...Standing at the Intersection of College Choice and Competency-based Education...
Standing at the Intersection of College Choice and Competency-based Education...
 
Star Pilot Project Review of Year 1
Star Pilot Project Review of Year 1Star Pilot Project Review of Year 1
Star Pilot Project Review of Year 1
 
Jan 31 2014 SAAS Division Meeting
Jan 31 2014 SAAS Division MeetingJan 31 2014 SAAS Division Meeting
Jan 31 2014 SAAS Division Meeting
 

Mais de Kerry Rice

Blended Learning: Practical Strategies for the Classroom
Blended Learning: Practical Strategies for the ClassroomBlended Learning: Practical Strategies for the Classroom
Blended Learning: Practical Strategies for the ClassroomKerry Rice
 
MSMU Best Practice in Online Teaching
MSMU Best Practice in Online TeachingMSMU Best Practice in Online Teaching
MSMU Best Practice in Online TeachingKerry Rice
 
Innovation in Teaching: Challenges, Risks, and Rewards
Innovation in Teaching: Challenges, Risks, and RewardsInnovation in Teaching: Challenges, Risks, and Rewards
Innovation in Teaching: Challenges, Risks, and RewardsKerry Rice
 
Using ePortfolios to Evaluate Teachers for Idaho's K-12 Online Teaching Endor...
Using ePortfolios to Evaluate Teachers for Idaho's K-12 Online Teaching Endor...Using ePortfolios to Evaluate Teachers for Idaho's K-12 Online Teaching Endor...
Using ePortfolios to Evaluate Teachers for Idaho's K-12 Online Teaching Endor...Kerry Rice
 
Common Core Gamified: Technology Supported CCSS Mathematics
Common Core Gamified: Technology Supported CCSS Mathematics Common Core Gamified: Technology Supported CCSS Mathematics
Common Core Gamified: Technology Supported CCSS Mathematics Kerry Rice
 
Common Core Gamified: Technology Supported CCSS for English Language Arts and...
Common Core Gamified: Technology Supported CCSS for English Language Arts and...Common Core Gamified: Technology Supported CCSS for English Language Arts and...
Common Core Gamified: Technology Supported CCSS for English Language Arts and...Kerry Rice
 
TxVSN Speaks Volumes Conference July 2014
TxVSN Speaks Volumes Conference July 2014TxVSN Speaks Volumes Conference July 2014
TxVSN Speaks Volumes Conference July 2014Kerry Rice
 
Idaho's K-12 Online Teaching Endorsement: Lessons Learned in Online Teacher P...
Idaho's K-12 Online Teaching Endorsement: Lessons Learned in Online Teacher P...Idaho's K-12 Online Teaching Endorsement: Lessons Learned in Online Teacher P...
Idaho's K-12 Online Teaching Endorsement: Lessons Learned in Online Teacher P...Kerry Rice
 
Status of Online Education in the U.S.
Status of Online Education in the U.S.Status of Online Education in the U.S.
Status of Online Education in the U.S.Kerry Rice
 
From Online Revolution to Mainstream Evolution: Are We There Yet?
From Online Revolution to Mainstream Evolution: Are We There Yet?From Online Revolution to Mainstream Evolution: Are We There Yet?
From Online Revolution to Mainstream Evolution: Are We There Yet?Kerry Rice
 
Online Learning: Where are we now?
Online Learning: Where are we now?Online Learning: Where are we now?
Online Learning: Where are we now?Kerry Rice
 
Digital Age PBL
Digital Age PBLDigital Age PBL
Digital Age PBLKerry Rice
 
VSS 2011 Data Mining (Thursday, 10:45)
VSS 2011 Data Mining (Thursday, 10:45)VSS 2011 Data Mining (Thursday, 10:45)
VSS 2011 Data Mining (Thursday, 10:45)Kerry Rice
 
Idaho's K-12 Online Teaching Endorsement
Idaho's K-12 Online Teaching EndorsementIdaho's K-12 Online Teaching Endorsement
Idaho's K-12 Online Teaching EndorsementKerry Rice
 
Idaho Region III Superintendents
Idaho Region III SuperintendentsIdaho Region III Superintendents
Idaho Region III SuperintendentsKerry Rice
 
PBL for a Digital Age
PBL for a Digital AgePBL for a Digital Age
PBL for a Digital AgeKerry Rice
 
Going Virtual! 2010
Going Virtual! 2010Going Virtual! 2010
Going Virtual! 2010Kerry Rice
 
VSS 2010 PBL Virtually Authentic
VSS 2010 PBL Virtually AuthenticVSS 2010 PBL Virtually Authentic
VSS 2010 PBL Virtually AuthenticKerry Rice
 
Adjunct Orientation, January 11, 2011
Adjunct Orientation, January 11, 2011Adjunct Orientation, January 11, 2011
Adjunct Orientation, January 11, 2011Kerry Rice
 

Mais de Kerry Rice (20)

Blended Learning: Practical Strategies for the Classroom
Blended Learning: Practical Strategies for the ClassroomBlended Learning: Practical Strategies for the Classroom
Blended Learning: Practical Strategies for the Classroom
 
MSMU Best Practice in Online Teaching
MSMU Best Practice in Online TeachingMSMU Best Practice in Online Teaching
MSMU Best Practice in Online Teaching
 
Innovation in Teaching: Challenges, Risks, and Rewards
Innovation in Teaching: Challenges, Risks, and RewardsInnovation in Teaching: Challenges, Risks, and Rewards
Innovation in Teaching: Challenges, Risks, and Rewards
 
Using ePortfolios to Evaluate Teachers for Idaho's K-12 Online Teaching Endor...
Using ePortfolios to Evaluate Teachers for Idaho's K-12 Online Teaching Endor...Using ePortfolios to Evaluate Teachers for Idaho's K-12 Online Teaching Endor...
Using ePortfolios to Evaluate Teachers for Idaho's K-12 Online Teaching Endor...
 
Common Core Gamified: Technology Supported CCSS Mathematics
Common Core Gamified: Technology Supported CCSS Mathematics Common Core Gamified: Technology Supported CCSS Mathematics
Common Core Gamified: Technology Supported CCSS Mathematics
 
Common Core Gamified: Technology Supported CCSS for English Language Arts and...
Common Core Gamified: Technology Supported CCSS for English Language Arts and...Common Core Gamified: Technology Supported CCSS for English Language Arts and...
Common Core Gamified: Technology Supported CCSS for English Language Arts and...
 
TxVSN Speaks Volumes Conference July 2014
TxVSN Speaks Volumes Conference July 2014TxVSN Speaks Volumes Conference July 2014
TxVSN Speaks Volumes Conference July 2014
 
Idaho's K-12 Online Teaching Endorsement: Lessons Learned in Online Teacher P...
Idaho's K-12 Online Teaching Endorsement: Lessons Learned in Online Teacher P...Idaho's K-12 Online Teaching Endorsement: Lessons Learned in Online Teacher P...
Idaho's K-12 Online Teaching Endorsement: Lessons Learned in Online Teacher P...
 
Status of Online Education in the U.S.
Status of Online Education in the U.S.Status of Online Education in the U.S.
Status of Online Education in the U.S.
 
From Online Revolution to Mainstream Evolution: Are We There Yet?
From Online Revolution to Mainstream Evolution: Are We There Yet?From Online Revolution to Mainstream Evolution: Are We There Yet?
From Online Revolution to Mainstream Evolution: Are We There Yet?
 
Online Learning: Where are we now?
Online Learning: Where are we now?Online Learning: Where are we now?
Online Learning: Where are we now?
 
Digital Age PBL
Digital Age PBLDigital Age PBL
Digital Age PBL
 
VSS 2011 Data Mining (Thursday, 10:45)
VSS 2011 Data Mining (Thursday, 10:45)VSS 2011 Data Mining (Thursday, 10:45)
VSS 2011 Data Mining (Thursday, 10:45)
 
Idaho's K-12 Online Teaching Endorsement
Idaho's K-12 Online Teaching EndorsementIdaho's K-12 Online Teaching Endorsement
Idaho's K-12 Online Teaching Endorsement
 
Idaho Region III Superintendents
Idaho Region III SuperintendentsIdaho Region III Superintendents
Idaho Region III Superintendents
 
PBL for a Digital Age
PBL for a Digital AgePBL for a Digital Age
PBL for a Digital Age
 
ASCD 2011
ASCD 2011ASCD 2011
ASCD 2011
 
Going Virtual! 2010
Going Virtual! 2010Going Virtual! 2010
Going Virtual! 2010
 
VSS 2010 PBL Virtually Authentic
VSS 2010 PBL Virtually AuthenticVSS 2010 PBL Virtually Authentic
VSS 2010 PBL Virtually Authentic
 
Adjunct Orientation, January 11, 2011
Adjunct Orientation, January 11, 2011Adjunct Orientation, January 11, 2011
Adjunct Orientation, January 11, 2011
 

Último

Education and training program in the hospital APR.pptx
Education and training program in the hospital APR.pptxEducation and training program in the hospital APR.pptx
Education and training program in the hospital APR.pptxraviapr7
 
CapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapitolTechU
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRATanmoy Mishra
 
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptx
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptxPractical Research 1: Lesson 8 Writing the Thesis Statement.pptx
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptxKatherine Villaluna
 
Practical Research 1 Lesson 9 Scope and delimitation.pptx
Practical Research 1 Lesson 9 Scope and delimitation.pptxPractical Research 1 Lesson 9 Scope and delimitation.pptx
Practical Research 1 Lesson 9 Scope and delimitation.pptxKatherine Villaluna
 
UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024UKCGE
 
What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?TechSoup
 
Benefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive EducationBenefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive EducationMJDuyan
 
How to Solve Singleton Error in the Odoo 17
How to Solve Singleton Error in the  Odoo 17How to Solve Singleton Error in the  Odoo 17
How to Solve Singleton Error in the Odoo 17Celine George
 
Patterns of Written Texts Across Disciplines.pptx
Patterns of Written Texts Across Disciplines.pptxPatterns of Written Texts Across Disciplines.pptx
Patterns of Written Texts Across Disciplines.pptxMYDA ANGELICA SUAN
 
How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17Celine George
 
Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.raviapr7
 
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...Nguyen Thanh Tu Collection
 
Diploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdfDiploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdfMohonDas
 
In - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptxIn - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptxAditiChauhan701637
 
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptxPISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptxEduSkills OECD
 
How to Use api.constrains ( ) in Odoo 17
How to Use api.constrains ( ) in Odoo 17How to Use api.constrains ( ) in Odoo 17
How to Use api.constrains ( ) in Odoo 17Celine George
 
Presentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphPresentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphNetziValdelomar1
 
The Stolen Bacillus by Herbert George Wells
The Stolen Bacillus by Herbert George WellsThe Stolen Bacillus by Herbert George Wells
The Stolen Bacillus by Herbert George WellsEugene Lysak
 

Último (20)

Education and training program in the hospital APR.pptx
Education and training program in the hospital APR.pptxEducation and training program in the hospital APR.pptx
Education and training program in the hospital APR.pptx
 
CapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptx
 
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRADUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
DUST OF SNOW_BY ROBERT FROST_EDITED BY_ TANMOY MISHRA
 
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptx
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptxPractical Research 1: Lesson 8 Writing the Thesis Statement.pptx
Practical Research 1: Lesson 8 Writing the Thesis Statement.pptx
 
Practical Research 1 Lesson 9 Scope and delimitation.pptx
Practical Research 1 Lesson 9 Scope and delimitation.pptxPractical Research 1 Lesson 9 Scope and delimitation.pptx
Practical Research 1 Lesson 9 Scope and delimitation.pptx
 
UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024UKCGE Parental Leave Discussion March 2024
UKCGE Parental Leave Discussion March 2024
 
What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?
 
Prelims of Kant get Marx 2.0: a general politics quiz
Prelims of Kant get Marx 2.0: a general politics quizPrelims of Kant get Marx 2.0: a general politics quiz
Prelims of Kant get Marx 2.0: a general politics quiz
 
Benefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive EducationBenefits & Challenges of Inclusive Education
Benefits & Challenges of Inclusive Education
 
How to Solve Singleton Error in the Odoo 17
How to Solve Singleton Error in the  Odoo 17How to Solve Singleton Error in the  Odoo 17
How to Solve Singleton Error in the Odoo 17
 
Patterns of Written Texts Across Disciplines.pptx
Patterns of Written Texts Across Disciplines.pptxPatterns of Written Texts Across Disciplines.pptx
Patterns of Written Texts Across Disciplines.pptx
 
How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17How to Make a Field read-only in Odoo 17
How to Make a Field read-only in Odoo 17
 
Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.Drug Information Services- DIC and Sources.
Drug Information Services- DIC and Sources.
 
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
 
Diploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdfDiploma in Nursing Admission Test Question Solution 2023.pdf
Diploma in Nursing Admission Test Question Solution 2023.pdf
 
In - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptxIn - Vivo and In - Vitro Correlation.pptx
In - Vivo and In - Vitro Correlation.pptx
 
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptxPISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
PISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
 
How to Use api.constrains ( ) in Odoo 17
How to Use api.constrains ( ) in Odoo 17How to Use api.constrains ( ) in Odoo 17
How to Use api.constrains ( ) in Odoo 17
 
Presentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a ParagraphPresentation on the Basics of Writing. Writing a Paragraph
Presentation on the Basics of Writing. Writing a Paragraph
 
The Stolen Bacillus by Herbert George Wells
The Stolen Bacillus by Herbert George WellsThe Stolen Bacillus by Herbert George Wells
The Stolen Bacillus by Herbert George Wells
 

Educational Data Mining in Program Evaluation: Lessons Learned

  • 1. © 2016 Boise State University 1 Kerry Rice, Jui-Long Hung, Yu-Chang Hsu, Brett E. Shelton Department of Educational Technology Boise State University Educational Data Mining in Program Evaluation: Lessons Learned AECT 2016, Las Vegas
  • 2. © 2016 Boise State University 2 MET Ed. D. Ed. S. Graduate Certificates: Online Teaching Technology Integration Specialist School Technology Coordinator K-12 Online Teaching Endorsement College of Education
  • 3. © 2016 Boise State University 3 Go Broncos!
  • 4. © 2016 Boise State University 4 Decision Tree Analysis (performance prediction) Cluster Analysis (engagement) Sequential Association Analysis (path analysis) Educational Data Mining Applications Time Series Analysis (future performance prediction)
  • 5. © 2016 Boise State University 5 Study #1: Teacher Training Workshops 2010 • Survey Data + Data Mining + Student Outcomes • Research Goal: – Program improvement – Satisfaction – Impact on practice • Blackboard • 103 participants • 31,417 learning logs • Cluster Analysis; Sequential Association Analysis; Decision Tree Analysis;
  • 6. © 2016 Boise State University 6 Study #2: Online Graduate Teacher Education 2010 • Data Mining + Student Outcomes (no demographic data) • Research Goal: – Identify struggling students – Adjust teaching strategies – Improve course design – Data Visualization • Study Design – Comparative (between and within courses) – Random course selection • Moodle • Two graduate courses (X and Y) • Each with two sections – X1 (18 students) – X2 (19 students) – Y1 (18 students) – Y2 (22 students) • 2,744,433 server logs • Cluster Analysis; Sequential Association Analysis; Decision Tree Analysis; Data Visualization
  • 7. © 2016 Boise State University 7 • Data mining + Demographics + Survey Data + Student Outcomes • Research Goal: Large scale program evaluation – Support decision making at the course and institutional level – Identify key variables and relationships between teacher and course satisfaction, student behaviors, and performance outcomes Study #3: End of Year K-12 Online Program Evaluation 2012 • Blackboard LMS • 7500 students • 883 courses • 23,854,527 learning logs (over 1 billion records) • Cluster Analysis; Decision Tree Analysis
  • 8. © 2016 Boise State University 8 Study #4: End of Year K-12 Blended Program Evaluation 2012 • Blackboard LMS • 255 Enrollments • 33 course sections • 17 unique courses • Satisfaction Survey • Data from 2011 pilot study • Cluster Analysis; Decision Tree Analysis • Demographics + Survey Data + Data Mining + Student Outcomes • Research Goal: Test Framework in Blended Learning – Support decision making at the course and institutional level? – Identify key variables and relationships between teacher and course satisfaction, student behaviors, and performance outcomes
  • 9. © 2016 Boise State University 9 Study #5: Online Graduate Teacher Education, 2014 • Moodle LMS • 509 Enrollments • 25 course sections • 12 unique courses • 431,708 records • Time Series Analysis • 34 original and derived variables –static (demographic) –dynamic (engagement) • Demographics + Data Mining + Student Outcomes • Research Goal: – Could we identify at-risk students in real time? – When?
  • 10. © 2016 Boise State University 10 Study #6: Online Graduate Teacher Education, 2015 • Moodle LMS • 661 Enrollments • 31 course sections • 18 unique courses • 546,965 records • Time Series Analysis • 34 original and derived variables –static (demographic) –dynamic (engagement) • Demographics + Data Mining + Student Outcomes • Research Goal: – Did the model developed in Study #5 work with new semester data? – Were the predictive (timing) results the same? – Were frequency data more predictive?
  • 11. © 2016 Boise State University 11 Variables - actual and derived Engagement
  • 12. © 2016 Boise State University 12 Cluster Analysis Student clustering which describes shared characteristics of students who passed or failed their courses
  • 13. © 2016 Boise State University 13 Cluster Analysis: Relative Participation Levels and Final Grades • Average Time Spent • Average Days Participated • Average Frequency of Mouse Clicks • Average Time Spent per Session • Average Frequency of Mouse Clicks per Session Participation Variables (Engagement)
  • 14. © 2016 Boise State University 14 Cluster Analysis: Student Characteristics Cluster 1 – Low-High, 119 students: Low average participation and higher performance levels. Cluster 2 – High-High, 60 students: High average participation and high performance levels. Cluster 3 – Low-Low, 76 students (46% remedial): Low average participation and low performance levels.
  • 15. © 2016 Boise State University 15 Cluster Distributions in Courses High percentage of Low-Low students in Chemistry CP, CP Pre-Calculus, English II, English III, and Pre-Calculus (H)
  • 16. © 2016 Boise State University 16 Decision Tree Analysis Perception and performance predictions which identify key predictors of course satisfaction, instruction satisfaction, and final grade
  • 17. © 2016 Boise State University 17 Decision Tree Analysis: Predictors of Student Performance Decision Tree Analysis • Identified at risk • Number of Courses Taken • Average Clicks per Week • Average Time Spent per Week, and • Average Time Spent per Session Other factors • Gender • Ethnicity • Reason for taking a course
  • 18. © 2016 Boise State University 18 Sequential Association Analysis Course X Course Y Does the design of the course (path to learning) predict learner outcomes?
  • 19. © 2016 Boise State University 19 Sequential Association Analysis
  • 20. © 2016 Boise State University 20 Time Series Analysis
  • 21. © 2016 Boise State University 21 Time Series Analysis – course access A B F Week 10 Spring Break
  • 22. © 2016 Boise State University 22 Time Series Analysis – DB Replies A B F Week 10 Spring Break
  • 23. © 2016 Boise State University 23 Overall Analysis • Students who took fewer courses performed significantly better than those who took more courses. • Engagement is a significant factor. High-engaged students performed better than low-engaged students. • Students identified as at risk performed differently than all other students. • Type of engagement matters. Students who accessed their courses more often performed better than those who had more interactions within the course. Consistent interaction over time is a better predictor of performance. (higher ed only) • Advanced courses - High-engagement and high performance (K-12) • Entry level courses - Low performance regardless of engagement (K-12) • Gender and ethnicity (higher ed) were identified as significant factors • Satisfaction did not always equate to higher performance
  • 24. © 2016 Boise State University 24 Characteristics of successful students • Female (k-12) • Younger (k-12) • Were enrolled in advanced courses (k-12) • Took fewer courses • Were more engaged overall • Were consistently engaged
  • 25. © 2016 Boise State University 25 Characteristics of at-risk students • Male (K-12) • Older (K-12) • Took entry-level courses (K-12) • Took a greater number of courses • Were low engaged overall • Were inconsistent in their engagement
  • 26. © 2016 Boise State University 26 Data Collection Challenges • Bb activity accumulator grouped wide ranging behaviors into only five useful categories • Missing data (empty fields – ex. internal handler) • Mismatched data fields/data stored in the wrong fields • Inconsistent data collection (i.e. failure to track every forum reply) • Partial or missing timestamp (needed for sequential analysis) • Course or student ID not linked to survey • Demographic data not linked to course or program • Inconsistent course models (blended)
  • 27. © 2016 Boise State University 27 Educational Data Mining Special Challenges • Learning behaviors are complex • Target variables (learning outcomes/performance) require wide range of assessments and indicators • Goal of improving teaching and learning is hard to quantify • Limited number of DM techniques suitable to meet educational goals • Only interactions that occur in the LMS can be tracked through data mining • Still a very intensive process to identify rules and patterns
  • 28. © 2016 Boise State University 28 References •Hung, J. L., Rice, K., & Saba, A. (2012). An educational data mining model for online teaching and learning. Journal of Educational Technology Development and Exchange, 5(2), 77-94. •Hung, J. L., Hsu, Y.-C., & Rice, K. (2012). Integrating data mining in program evaluation of K-12 online education. Educational Technology & Society, 15(3), 27-41. •Rice, K., & Hung. J. (2015). Data mining in online professional development program: An exploratory case study. International Journal of Technology in Teaching and Learning, 11(1), 1-20. •Shelton, B., Hung, J. L., & Baughman, S. (2015). Online graduate teacher education: Establishing and EKG for student success intervention. Technology, Knowledge and Learning. •Rice, K., & Hung, J. L. (2015). Identifying variables important to the success of K-12 students in blended learning. Paper presented at the Northern Rocky Mountain Educational Research Association Conference, Boise, Idaho. •Shelton, B. E., Hung, J. L.., & Lowenthal, P. (under review). Predicting student success by modeling student interaction in online courses.