2. Kostas Perifanos, Learner Analytics & Data Science
EdTech Meetup, July 10, 2014
Introduction to
Adaptive Learning
3. Kostas Perifanos, Learner Analytics & Data Science
Aim:
“[...] Computers adapt the presentation of educational material according to student’s
learning needs, as indicated by their responses to questions and tasks.” [wikipedia]
● Knowledge inference: Measure what a student knows at a specific time
● Scientific understanding of Learning:
● Ignore domain differences and focus on the kind of knowledge being taught
● Knowledge Components - KC’s
Introduction to Adaptive Learning
4. Kostas Perifanos, Learner Analytics & Data Science
Abstracting the educational material:
“A knowledge component is a description of a mental structure or process that a learner
uses, alone or in combination with other knowledge components, to accomplish steps
in a task or a problem.” [Pittsburgh Science for Learning Center]
http://www.learnlab.org/research/wiki/index.php/Knowledge_component
http://www.learnlab.org/opportunities/summer/presentations/2010/2010-pslc-summer-
school%20Geoff%20Gordon.pdf
Introduction to Adaptive Learning
5. Kostas Perifanos, Learner Analytics & Data Science
Introduction to Adaptive Learning
● KC: Anything a student can know/do:
● Skill
● Fact
● Concept
● Principle etc
● KC’s can be:
● Low Level Things
● High Level things
● Motivational Things etc
6. Kostas Perifanos, Learner Analytics & Data Science
Introduction to Adaptive Learning
If we can measure knowledge:
● We can make it better
● We can provide tutors with meaningful feedback
● We can make automated decisions
… thus, we can implement “Adaptive Learning” solutions
● Knowledge Inference / Latent Knowledge Estimation
● Latent = Not directly observable (measurable)
7. Kostas Perifanos, Learner Analytics & Data Science
Introduction to Adaptive Learning
How to infer knowledge?
We can look at student performance over time
Build and evaluate models
Two views of KC’s
Statistical Model: what latent factors in a student/step explain observed data
Cognitive Model: what is the structure of the internal reasoning system students use
to solve problems
8. Kostas Perifanos, Learner Analytics & Data Science
Introduction to Adaptive Learning
Statistical/Machine Learning Vision
Given a cognitive model:
Evaluate the model
Evaluate the student
Provide feedback
Pure ML approaches are not heavily used just yet, human interaction and interpretation is
required
9. Kostas Perifanos, Learner Analytics & Data Science
Introduction to Adaptive Learning
Common Models
● Bayesian Knowledge Tracing
● Performance Factors Analysis
● Item Response Theory - Rasch Model
● Additive Factor Model (AFM)
Two main approaches:
- Does student X knows skill K? [Knowledge Tracing]
- Calculate the probability of a correct answer given student and skill [IRT, AFM, PFA]
P(correct| features of student and step at time t)
10. Kostas Perifanos, Learner Analytics & Data Science
Introduction to Adaptive Learning
Bayesian Knowledge Tracing
How well a student knows a specific skill/component at a specific time, based on their
performance
Each item corresponds to a single skill
Measure skill/KC knowledge at a specific time
Two learning parameters [P(L), P(T)]
Two performance parameters [P(G), P(S)]
P(L): Probability the skill is already known
P(T): Probability the skill will be learned
P(G): Probability of correct guessing [I don’t know the skill and I am guessing]
P(S): Probability of slip [I know the skill but I made a mistake]
Model fitting: Expectation Maximization [Hidden Markov Model]
11. Kostas Perifanos, Learner Analytics & Data Science
Introduction to Adaptive Learning
Item Response Theory
Model probability of correct as function of student knowledge level and item difficulty
Additive Factor Model
Model probability of correct as function of student knowledge level and item difficulty,
but also take into account skill learning rate.
Each item has a KC and this determines the difficulty of the item.
Learning rates: How fast students are learning specific skills
12. Kostas Perifanos, Learner Analytics & Data Science
Introduction to Adaptive Learning
Performance Factors Analysis
Measure the latent skill as the probability of correctness the next time we
encounter this skill
Multiple KC’s per item
Parametrized skills [success learning rate, failure learning rate] and item
difficulty
Take into account success and failure
13. Kostas Perifanos, Learner Analytics & Data Science
Introduction to Adaptive Learning
What if we don’t know the KC’s?
Principal Components Analysis (PCA):
Factor student-step data in “eigenskills” to obtain most important “interactions”
Good at making predictions
Features are not easily interpretable
14. Kostas Perifanos, Learner Analytics & Data Science
Introduction to Adaptive Learning
Overview:
● Knowledge tracing
● Directly model “Does the student know X”, suitable for good instructions
● Knowledge is latent, harder to fit but more flexible
● AFM
● Used for refining KC models, detect a bad KC
● Knowledge is observable [Fully Markov Model]
● assumes all students accumulate knowledge in the same manner and ignores the
correctness of their individual responses
● PFA: LR model, similar to AFM but take into account individual responses (Successes
vs Failures)
16. Kostas Perifanos, Learner Analytics & Data Science
Introduction to Adaptive Learning
(Warning: Next presentation will have all the maths and implementation details)
Thank you
