Author: Jingyan Lu, The University of Hong Kong -------------------------------------------------------- http://www.cite.hku.hk/news.php?id=501&category=cite

1. Understanding, predicting and optimizing
learning with Learning Analytics
Jingyan Lu, The University of Hong Kong
July 5th, LASI-Hong Kong
7/6/2013Jingyan Lu@hku
1

2. Assessment in the Education Triangle
Instruction Assessment
Curriculum
7/6/2013Jingyan Lu@hku
2

3. Assessment triangle
Observation Interpretation
Cognition
Pellegrino, J. W., Chudowsky, N., & Glaser, R. (Eds.). (2002). Knowing what
students know: The science and design of educational assessment.
Washington, DC: National Academy Press.
7/6/2013Jingyan Lu@hku
3

4. Conceptual Integration of Learning
Analytics and Assessment
Instruction Assessment
Curriculum
Observation Interpretation
Cognition
7/6/2013Jingyan Lu@hku
4

5. Contribution of measurement
and statistical modeling in
Assessment
7/6/2013Jingyan Lu@hku
5

6. Assessment models
X = T + E
Argumentative
Representation
Structure
Justification
Position
Conceptual Clarification
Conclusion
Multiple Perspectives
Identify Stakeholder
Number of Stakeholder
Types of Stakeholder
Matching
Citing Case Information
Outside Information
Explanation
7/6/2013Jingyan Lu@hku
6

7. Conceptual Assessment Framework:
Evidence Center Assessment Design (ECD)
Mislevey, R. J., Steinberg, L. S., Almond, R. G., Haertel, G. D., & Penuel, W. R. (2001).
Leverage points for improving education assessment. Princeton: Educational
testing Service.
7/6/2013Jingyan Lu@hku
7

8. Assessment in the 21st Century
Classroom
 “This evidence-based approach (Mislevy et al,
2003) is particularly relevant in the 21st century
technology rich classroom where student’s
use of technology tools for learning create a
multitude of data (e.g., artefacts, log data)
which can be mined, assessed, and presented in
ways that students and teachers can interpret it
to support learning.”
 Hansen & Wasson (forthcoming)
Hansen, C. & Wasson, B. (forthcoming). Formaive e-assessment in the 21st century
Classroom. NEAR, Iceland, March 7/6/2013Jingyan Lu@hku
8

9. Applications--Where
 Online Learning Systems - online courses or
learning software or interactive environments that
use intelligent tutoring systems, virtual labs, or
simulations, such as Moodle.
 Why: Big data
7/6/2013Jingyan Lu@hku
9

10. Application of LA—Predictive
Models in Instructional System
 When are students ready to move onto the next topic?
 When are students falling behind in a course?
 When is a student at risk for not completing a course?
 What grade is a student likely to get without
intervention?
 What is the best next course for a given student?
 Should a student be referred to a counselor for help?
7/6/2013Jingyan Lu@hku
10

11. Typical
Adaptive
Learning
System
Predictive
nature of
LA
7/6/2013Jingyan Lu@hku
11

12. Why are we measuring (1):
Modeling and theory building
User knowledge modeling
User behavior modeling
User experience modeling
User Profiling
Domain Modeling
7/6/2013Jingyan Lu@hku
12

13. Examples of interactive
learning environments
Khan Academy
7/6/2013Jingyan Lu@hku
13

14. Stakeholders
ADAPTIVE
LEARNING
SYSTEM
7/6/2013Jingyan Lu@hku
14

15. Some Examples
7/6/2013Jingyan Lu@hku
15

16. Examples (1): Online
learning behavior
7/6/2013Jingyan Lu@hku
16

17. What are we measuring on
LMS: Student model
 How peer assessment
affect learning
Lu, J., & Zhang, Z. (2012). Understanding the effectiveness of online peer assessment: A path model.
Journal of Educational Computing Research, 46(3), 313-333.
7/6/2013Jingyan Lu@hku
17

18. Where do we measure it: Task
model
 Peer assessment
 Feedback
 Grading
7/6/2013Jingyan Lu@hku
18

19. How do we measure: Evidence
model
 Online behavior
 Log data
 Activities
 Content
7/6/2013Jingyan Lu@hku
19

20. Example 2: Modeling online
critical reading
7/6/2013Jingyan Lu@hku
20

21. Student Model
 Critical reading behavior
 Critical reading and writing argument
7/6/2013Jingyan Lu@hku
21

22. Task Model
DiigoOASIS
7/6/2013Jingyan Lu@hku
22

23. Evidence Model
 Understanding online
reading behavior
 Building up model from
Reading to writing
7/6/2013Jingyan Lu@hku
23

24. Thank You
jingyan@hku.hk
7/6/2013Jingyan Lu@hku
24