Slides from keynote at moodlemoot.net 2014.

1. ANALYSING ANALYTICS
Gavin Henrick
Learning Technology Services

2. This Wikipedia and Wikimedia Commons image is from the user Chris 73 and is freely available at
commons.wikimedia.org/wiki/File:Tokyo_University_Entrance_Exam_Results_6.JPG under the creative commons cc-by-sa 3.0
license.

3. LEARNING ANALYTICS
What is it?

4. What is Learning Analytics?
“Learning analytics is the measurement, collection, analysis
and reporting of data about learners and their contexts, for
purposes of understanding and optimising learning and the
environments in which it occurs. ”
Wikipedia http://en.wikipedia.org/wiki/Learning_analytics
“Field associated with deciphering trends and patterns from
educational big data, or huge sets of student-related data,
to further the advancement of a personalized, supportive
system of higher education”
2013 Horizon Report http://net.educause.edu/ir/library/pdf/HR2013.pdfz

5. How?
“Learning analytics is the measurement, collection, analysis
and reporting of data about learners and their contexts, for
purposes of understanding and optimising learning and the
environments in which it occurs. ”
Wikipedia http://en.wikipedia.org/wiki/Learning_analytics
“Field associated with deciphering trends and patterns from
educational big data, or huge sets of student-related data,
to further the advancement of a personalized, supportive
system of higher education”
2013 Horizon Report http://net.educause.edu/ir/library/pdf/HR2013.pdfz

6. Why?
“Learning analytics is the measurement, collection, analysis
and reporting of data about learners and their contexts, for
purposes of understanding and optimising learning and the
environments in which it occurs. ”
Wikipedia http://en.wikipedia.org/wiki/Learning_analytics
“Field associated with deciphering trends and patterns from
educational big data, or huge sets of student-related data,
to further the advancement of a personalized, supportive
system of higher education”
2013 Horizon Report http://net.educause.edu/ir/library/pdf/HR2013.pdfz

7. DESCRIPTIVE ANALYTICS
What has happened?

8. DIAGNOSTIC ANALYTICS
Why this happened?

9. PREDICTIVE ANALYTICS
What will happen?

10. PRESCRIPTIVE ANALYTICS
What to do?

11. What data is there on students?
Profile Activity
Content Results

12. Profile
• Prior skill set
• Prior examination results
• Prior subject choice
• Prior examination levels
• Demographics

13. Activity
• Library visits
• Number of books / resources used
• Class attendence
• Wifi access
• Online systems access

14. Content
• Which modules
• How many modules
• Level of modules
• Workload of modules

15. Results
• Year completion
• Module completion
• Module grades
• Assignment grades
• Question level success
• Surveys
• Competency assessments
• Competency related success

16. Key Goals
• Improve student success
• Improve student retention
• Improve the learning experience

17. WHO
WHAT
WHERE
WHEN
WHY

18. Who are we thinking about?
Consider each of the following questions from the
position of
• A student
• A teacher/lecturer
• A programme /course coordinator
• Student support staff
• Central registry

19. Who
Who is going to be using the data or the reports
using the data?
What controls are needed to ensure only those
who should access them get access?

20. What
What data and reports are they going to need for
their usage?

21. Where
Where do they need these reports and data?
Where and how will they be accessing them

22. When
When do they need to get the data, reports
- Different data sources will have different potential
latency
- Different data sets may require different
timeframes for usefulness
- Different data sets may be useful at different
times of year

23. Why
Why are they going to use it?

24. Useful vs Used
• Lots of data may be useful but not used
• Having reports available to access is no good if
they are not accessed
• Important to identify what will be used and how

25. WHY ANALYSE
Applying Analytics to learning

26. “With this data available
it is wrong to withhold it from the
students themselves”

27. What would a student do with the
information he is given through learning
analytics?

28. What would a lecturer do with the
information he is given about a student
through learning analytics?

29. What would a lecturer do with the
information he is given about his
course through learning analytics?

30. SOME QUESTIONS

31. Student
• How well am I doing?
• How well am I doing compared to the class?
• How are my friends doing?
• Which subjects should I invest more time in for
greatest benefit?
• What am I not doing that others are doing?
• Is there anything I should be doing that I am not?

32. Teacher
• How well are my students doing?
• How well are they doing compared to the class?
• How well are they doing compared to other years?
• Which areas of the curriculum are getting the worst
/ best results?
• Which learning outcomes are not being met?
• Are students using the resources? Which
resources? When ?
• With which resources are students outcomes the
best in assessments?

33. Support
• How well are students doing?
• How well are they doing compared to the class?
• How well are they doing compared to other
years?
• Which students are in need of help on a specific
subject?
• Which students are in need of help across many
subjects / in general?

34. Admins
• Which courses are students not engaging in?
• Which courses are teachers not engaging in?
• Which courses are students underperforming in?
• Which courses are generating the highest?
• Which students are at risk in a course?
• Which students are at risk in multiple courses?

35. ETHICAL CONCERNS
Applying Morals to Analytics

36. Data and reporting concerns
Some issues for discussion:
• Transparency on data acquisition
• Secure data storage, retention periods
• Ownership of data
• Purpose for reporting on different themes
• Access to different data

37. Legal issues
• Data protection laws
• Security policies
• Access policies
• Terms of use
• Student awareness
• Student Impact

38. THE OPEN UNIVERSITY
An example of transparency in analytics

39. The Open University 8 key principles
Principle 1: Learning analytics is an ethical practice that should align with core
organisational principles, such as open entry to undergraduate level study.
Principle 2: The OU has a responsibility to all stakeholders to use and extract
meaning from student data for the benefit of students where feasible.
Principle 3: Students should not be wholly defined by their visible data or our
interpretation of that data.
Principle 4: The purpose and the boundaries regarding the use of learning
analytics should be well defined and visible.
Principle 5: The University is transparent regarding data collection, and will provide
students with the opportunity to update their own data and consent agreements at
regular intervals.
Principle 6: Students should be engaged as active agents in the implementation of
learning analytics (e.g. informed consent, personalised learning paths,
interventions).
Principle 7: Modelling and interventions based on analysis of data should be sound
and free from bias.
Principle 8: Adoption of learning analytics within the OU requires broad acceptance
of the values and benefits (organisational culture) and the development of
appropriate skills across the organisation.
See: http://www.open.ac.uk/students/charter/essential-documents/ethical-use-student-
data-learning-analytics-policy

40. Jisc work in the UK
• Code of practice for Learning Analytics – Public
consultation
• http://sclater.com/blog/code-of-practice-for-learning-analytics-
public-consultation/
• Final version of the Code in June
• Interesting breakdown including access, and action responsibilities

41. Mobile App. What students want
• http://sclater.com/blog/what-do-students-want-from-a-
learning-analytics-app/
• Some points to consider
• What is analytics to the student
• What do they want tracked
• What is the information that they want access to easily

42. References
Finding the Prodigal Student: Academics' Analytics at UCD
http://www.heanet.ie/conferences/2014/talks/id/97
Making Sense of Data from your LMS
http://www.heanet.ie/conferences/2014/talks/id/98
Code of practice for learning analytics – A literature review of the ethical and
legal issues
http://analytics.jiscinvolve.org/wp/2014/12/04/jisc-releases-report-on-ethical-and-
legal-challenges-of-learning-analytics/
Learning Analytics – The current state of play in UK Higher and further
education
http://analytics.jiscinvolve.org/wp/2014/11/20/jisc-releases-new-report-on-learning-
analytics-in-the-uk/
Ethical use of Student Data for Learning Analytics Policy – The Open
University
http://www.open.ac.uk/students/charter/essential-documents/ethical-use-student-
data-learning-analytics-policy