Presentation at the NUI Galway library - Research Data Management Group. 02/05/2018

1. Learning Analytics:
understand learning and
support the learner
Mathieu d’Aquin - @mdaquin
Data Science Institute
National University of Ireland Galway
Insight Centre for Data Analytics
AFEL project (@afelproject)

2. Learning analytics
According to Wikipedia
(and past LAK CFPs, and some papers from relevant people)
Learning analytics is the measurement, collection, analysis and
reporting of data about learners and their contexts, for purposes of
understanding and optimizing learning and the environments in
which it occurs.

3. Learning analytics
According to Wikipedia
(and past LAK CFPs, and some papers from relevant people)
Learning analytics is the measurement, collection, analysis and
reporting of data about learners and their contexts, for purposes of
understanding and optimizing learning and the environments in
which it occurs.
data analytics applied to
data from learning activities

4. Learning
(from a system’s point of view)
Learner
Platform
VLE | Website | Library
Assessment | Enrollment
School/University

5. Learning analytics: Basics
Example: Vital for Doctor
Dashboard comparing
engagement of junior
doctors with learning
resources, and the effect on
their “performance”.

6. Learning analytics: Prediction
Example: OU
Analyse
Predicting student
success based on
engagement with
learning resources
and demographic
information.

7. Learning analytics: Exploration
Example: Student
location
Clustering students
based on their
location and the
subjects to which
they enroll.

8. Learning analytics: Interpretation
Example:
Understanding
how sequences
of modules are
chosen
Using sequence
mining and
formal concept
analysis.
d'Aquin, Mathieu, and N. Jay. "Interpreting data mining results with linked data for
learning analytics: motivation, case study and directions." In Proceedings of the
Third International Conference on Learning Analytics and Knowledge, LAK 2013.

9. But...
According to Wikipedia
(and past LAK CFPs, and some papers from relevant people)
Learning analytics is the measurement, collection, analysis and
reporting of data about learners and their contexts, for purposes of
understanding and optimizing learning and the environments in
which it occurs.

10. But...
According to Wikipedia
(and past LAK CFPs, and some papers from relevant people)
Learning analytics is the measurement, collection, analysis and
reporting of data about learners and their contexts, for purposes of
understanding and optimizing learning and the environments in
which it occurs.

11. But...
According to Wikipedia
(and past LAK CFPs, and some papers from relevant people)
Learning analytics is the measurement, collection, analysis and
reporting of data about learners and their contexts, for purposes of
understanding and optimizing learning and the environments in
which it occurs.
i.e. not only students in the classroom/on campus!

12. Learning
(from a system’s point of view)
Learner
Platform
VLE | Website | Library
Assessment | Enrollment
School/University
Edu on

13. Education/Learning
(still from a system’s point of view)
Learner
Platform
VLE | Website | Library
Assessment | Enrollment
School/University

14. Learning
(still from a system’s point of view)
Learner
Platform
VLE | Website | Library
Assessment | Enrollment
School/University

15. Parenthesis
Learner
Platform
VLE | Website | Library
Assessment | Enrollment
School/University
This needs to
evolve to become
more open and
connected

16. data.open.ac.uk

17. Applications - Recommendation

18. Applications - Recommendation

19. Back to…
Learner
Platform
VLE | Website | Library
Assessment | Enrollment
School/University

20. Objective: To create theory-backed methods and tools
supporting self-directed learners and the people helping them
in making more effective use of online resources, platforms and
networks according to their own goals.

21. Scenario
Jane is 37 and works as an administrative assistant in a local medium-sized company. As a hobbies, she enjoyed sewing and
cycling in the local forests. She is also interested in business management, and is considering either developing in her current
job to a more senior level or making a career change.
Jane spends a lot of time online at home and at her job. She has friends on facebook with whom she shares and discusses
local places to go biking, and others with whom she discusses sewing techniques and possible projects, often through sharing
youtube videos.
Jane also follows MOOCs and forums related to business management, on different topics. She often uses online resources
such as Wikipedia and online magazine on the topics. At school, she was not very interested in maths, which is needed if she
want to progress in her job. She is therefore registered on Didactalia, connecting to resources and communities on maths,
especially statistics.
Jane has also decided to take her learning seriously: She has registered to use the AFEL dashboard through the Didactalia
interface. She has also installed the browser extension to include her browsing history, as well as the facebook app. She has
not included in her dashboard her emails, as they are mostly related to her current job, or twitter, since she rarely uses it.
Jane looks at the dashboard more or less once a day, as she is prompted by a notification from the AFEL smartphone
application or from the facebook app, to see how she has been doing the previous day in her online social learning. It might
for example say “It looks like you progressed well with sewing yesterday! See how you are doing on other topics…”
Jane, as she looks at the dashboard, realises that she has been focusing a lot on her hobbies and procrastinated on the
topics she enjoys less, especially statistics. Looking specifically at statistics, she realises that she almost only works on it in
Friday evenings, because she feels guilty of not having done much during the week. She also sees that she is not putting as
much effort into her learning of statistics as other learners, and not making as much progress. She therefore makes a
conscious decision to put more focus on it. She adds the dashboard goals of the form “to work on statistics during my lunch
break every week day” or “to have achieved a 10% progress compared to now by the same time next week”. The dashboard will
remind her how she is doing against those goals as she go about her usual online social learning activities. She also gets
recommendation of things to do on Didactalia and Facebook based on the indicators shown on the dashboard and her
stated goals.

22. Scenario
Jane is 37 and works as an administrative assistant in a local medium-sized company. As a hobbies, she enjoyed sewing and
cycling in the local forests. She is also interested in business management, and is considering either developing in her current
job to a more senior level or making a career change.
Jane spends a lot of time online at home and at her job. She has friends on facebook with whom she shares and discusses
local places to go biking, and others with whom she discusses sewing techniques and possible projects, often through sharing
youtube videos.
Jane also follows MOOCs and forums related to business management, on different topics. She often uses online resources
such as Wikipedia and online magazine on the topics. At school, she was not very interested in maths, which is needed if she
want to progress in her job. She is therefore registered on Didactalia, connecting to resources and communities on maths,
especially statistics.
Jane has also decided to take her learning seriously: She has registered to use the AFEL dashboard through the Didactalia
interface. She has also installed the browser extension to include her browsing history, as well as the facebook app. She has
not included in her dashboard her emails, as they are mostly related to her current job, or twitter, since she rarely uses it.
Jane looks at the dashboard more or less once a day, as she is prompted by a notification from the AFEL smartphone
application or from the facebook app, to see how she has been doing the previous day in her online social learning. It might
for example say “It looks like you progressed well with sewing yesterday! See how you are doing on other topics…”
Jane, as she looks at the dashboard, realises that she has been focusing a lot on her hobbies and procrastinated on the
topics she enjoys less, especially statistics. Looking specifically at statistics, she realises that she almost only works on it in
Friday evenings, because she feels guilty of not having done much during the week. She also sees that she is not putting as
much effort into her learning of statistics as other learners, and not making as much progress. She therefore makes a
conscious decision to put more focus on it. She adds the dashboard goals of the form “to work on statistics during my lunch
break every week day” or “to have achieved a 10% progress compared to now by the same time next week”. The dashboard will
remind her how she is doing against those goals as she go about her usual online social learning activities. She also gets
recommendation of things to do on Didactalia and Facebook based on the indicators shown on the dashboard and her
stated goals.

23. Challenges
How do we recognise learning in (the data of) open, generic
unconstrained environments?
How do we measure learning in (the data of) open, generic
unconstrained environments?

24. Cognitive model: Learning and knowledge
construction through co-evolution
The dynamic processes of learning and knowledge construction
from Kimmerle, Moskaliuk, Oeberst, and Cress, 2015.

25. Cognitive model: Learning and knowledge
construction through co-evolution
The dynamic processes of learning and knowledge construction
from Kimmerle, Moskaliuk, Oeberst, and Cress, 2015.

26. Cognitive model: Learning and knowledge
construction through co-evolution
“constructive friction is the driving force behind
learning” -- AFEL Deliverable 4.1, [CK08]

27. Identified types of constructive frictions, indicators
of learning (in a given learning scope)
- Coverage: Most obvious indicator. How much of the
concepts covered by the given learning scope (topic) have
been covered by captured learning activities.
- Complexity: How the learner difficult at the resources used
by the learner in exploring this learning scope.
- Diversity: How diverse the resources and activities used by
the learner have been in the given learning scope.

28. Current results - the AFEL personal analytics app

29. What about ethics?
This is about behavioural analysis and supporting
behavioural changes for learners… ethics
questions obviously relevant.
Not only about privacy: self-regulation, black-box
effect, data-bias, unbalanced access need to be
considered.
Basic question: Is it OK to process student data for
analysing retention, success, learning design,
learning behaviours?
Reverse question: We already have all those data.
Is it OK not to use it to provide the best possible
chances of success to our student.
Bobbie Eicher et al., Jill Watson Doesn’t Care
if You’re Pregnant: Grounding AI Ethics in
Empirical Studies, AIES 2018

30. What about ethics?
‘Ethics in
Design’ for Data
Science
Dialectic
The process is based on a conversational
approach between data and critical social
scientists throughout the project’s life-cycle.
Reflective
Ethical concerns are not pre-fixed; they may
emanate from any stage of the project; thus,
constant reflexivity on activities and
researchers is needed.
Creative, not disruptive
The objective of this process is to achieve a
positive impact on the research, increase its
value addressing ethics throughout the
project’s life-cycle.
All- encompassing
Ethical concerns appear as much in the
research activities as in their outcomes, their
use and exploitation; the process needs to
expand on all stages.
d’Aquin et al, Towards an “Ethics in Design” methodology for AI research projects, in AIES 2018

31. Conclusion
Collecting Data
about learning
Data Analytics
Results
Interpretation
Intervention

32. Conclusion
Collecting Data
about learning
Data Analytics
Results
Interpretation
Intervention

33. Conclusion
Collecting Data
about learning
Data Analytics
Results
Interpretation
Intervention

34. Conclusion
Collecting Data
about learning
Data Analytics
Results
Interpretation
Intervention
Educational psychology
Education Science
Ethics

35. Thank you!
@mdaquin
mathieu.daquin@nuigalway.ie
mdaquin.net
@afelproject
afel-project.eu