The concept of learning analytics is gaining traction in education as an approach to using learner data to gain insights into different trends and patterns but also to inform timely and appropriate support interventions. This webinar will explore a number of different approaches to integrating learning analytics into the context of assessment and feedback design; from overall assessment patterns and VLE usage in an institution, to creating student facing workshops, to developing principles for dashboards. The presentations will feature current thinking and approaches from teams from the following projects in the Jisc Assessment and Feedback programme: TRAFFIC, MMU ( speaker Rachel Forsyth) EBEAM, University of Hudersfield, (speaker Cath Ellis) iTeam, University of Hertfordshire (speaker Julie Vuolo) There will be opportunities for questions and discussion throughout the session.

1. 15/05/2013 Venue Name: Go to 'View' menu > 'Header and Footer' to change slide 1
Learning Analytics for Assessment
and Feedback
Presenters
Sheila MacNeill, Jisc Cetis
Rachel Forsyth, Manchester Metropolitan University
Cath Ellis, University of Huddersfield
Julie Vuolo, University of Hertfordshire
15 May 2013

2. Which best describes your role?
A. Supporting/managing e-learning or technology-enhanced learning
B. Staff and educational development
C. Lecturer/course leader
D. Professional services eg library, careers, student support
E. Other – please use chat box to give detail
15/05/2013 Developing Digital Literacies slide 2

3. 15/05/2013 slide 3
TRAFFIC project

4. 15/05/2013 slide 4
MMU TRAFFIC – the context
• Big institutional curriculum change
• One of aims: to address over-assessment
– Max 2 summative elements per 30 credit module
– Rolling out year-by-year to all UGs from Sep 2011

5. Analytics and systems development
15/05/2013 slide 5

6. Analytics and systems development
15/05/2013 slide 6
Personalised submission sheet
Student ID (+ Unit code)

7. Analytics and systems development
15/05/2013 slide 7

8. Analytics and systems development
15/05/2013 slide 8
0
50000
100000
150000
200000
250000
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 2 4 6 8 10 12 14 16
2012 2013
+ = 8.3M hits
(since Jan 1, 2012)

9. Analytics and systems development
15/05/2013 slide 9

10. Analytics and academic development
15/05/2013 slide 10

11. Analytics and academic development
15/05/2013 slide 11

12. Analytics and academic development
15/05/2013 slide 12

13. Analytics and academic development
15/05/2013 slide 13

14. Analytics and enhancement
15/05/2013 slide 14
30%
40%
50%
60%
70%
80%
90%
100%
The teaching on my
course
Academic support Learning resources Overall Satisfaction
30%
40%
50%
60%
70%
80%
90%
100%
The
teaching on
my course
Assessment
and feedback
Academic
support
Organisation
and
management
Learning
resources
Personal
development
Overall
Satisfaction
I am satisfied
with the
Students'
Union
(Association
or Guild) at
my institution
2011
MMU result
Similar institution results
2012

15. Analytics and enhancement
15/05/2013 slide 15
2011/12 (n=8,709) 2012/13 (n=8,646)
FB
SATISFCN
HAND-INS BUNCHING
FB
SATISFCN
HAND-INS BUNCHING
L4 3.75 5.61 1.29
3.79
(+1%)
5.59
(0%)
1.25 (-
3%)
L5 3.70 10.47 1.38
3.77
(+2%)
5.78 (-
45%)
1.25 (-
9%)
L6 3.72 7.81 1.30
3.8
(+2%)
7.86
(+1%)
1.26 (-
3%)
Predictors of overall
satisfaction

16. 15/05/2013 slide 16
EBEAM

17. 15/05/2013 slide 17
Learning Analytics
Department Level
Cath Ellis
University of Huddersfield

18. Social networks
Retention

19. Institutionalised
Operationalisation

20. Social networks
Retention
Institutionalised
Operationalisation

21. Assessment
All students are assessed and all tutors
mark student work
Ubiquitous
Routine Normative
Widely
understood
Expected
Everyone
‘gets’ it
Important
It’s what
students pay
for
It’s what
tutors are
paid to do
Focussed on
achievement
not just
retention

22. Academic Analytics
Business
intelligence
in HE
Learning Analytics
Focused on
retention
and (to a
lesser
extent)
achievement
Assessment Analytics
Assessment data used within a learning analytics
strategy

23. Degree
Classification
GPA at
progression
Module
Results
Assessment
task results
Granularity

24. Degree
Classification
GPA at
progression
Module Results
Assessment
task results
Granularity
Common
errors
Assessment
criteria

25. Ipsative
Individual
improvement
over time
Improvement
between
cohorts

26. Department level
Coordinated approach
Shared comments sets
Linked to other support
materials
Same ‘song sheet’
Pooled data
Student
Module
Course
Fed through to dashboard
Student facing
Tutor facing
Institution/PSRB facing

29. 15/05/2013 slide 29
ITEAM

30. 15/05/2013 slide 30

31. 15/05/2013 slide 31
Course monitoring and the Student Dashboard
• A development which pulls together information
about VLE (module level) engagement and
assessment performance and displays it in a
student and staff view within the VLE
• To help students with the self-regulation of their
studies.
• To help staff identify where support for individuals
or groups of students might be needed

32. 15/05/2013 slide 32
Filter
students
Set threshold

33. 15/05/2013 slide 33

34. 15/05/2013 slide 34
Student input
• ..wanted to know where they were in relation to others as this
would be a motivator to do better
• ..felt it would not de-motivate them to know they were at the
lower end of the league table.
• ..said the currency of data is important if dashboard is to be
meaningful to them
• ..would like a predictor or calculator for working out final
classification
• ..wanted re-assurance that their information would only be
seen by ‘those needing it’
• ..were interested in what else could go in there e.g. book
loans, module pass/fail rates

35. 15/05/2013 slide 35
Design considerations
General, Access, Display, Content, Support
Access
• Levels of staff access need to be agreed & processes developed to
gate-keep access
• It should be clear to staff and students what levels of access exist
• Entry to the SPD should be quick and easy e.g. icon click on
homepage
• Effective filters should be employed to allow staff to access
relevant information quickly
• Links to other University information systems should be
embedded in the SPD e.g. University Policies and Regulations on
classification

36. 15/05/2013 slide 36
Design considerations
Display
• Page content should be signposted clearly to aid navigation within
the dashboard
• Data should be presented in a simple, unambiguous manner for
clarity of meaning and to promote engagement
• Colour schemes should be simple and consistent to ensure clarity of
presentation and to promote engagement
Support
• There must be education for student and teachers about using the
SPD
• Education should target staff and students and include how to
access and functionality, limitations and data interpretation.

37. Average Grade for Module Hits

38. Potential
What other data would be useful to influence
learning and teaching?