Presenter: Ross Ward Organisation: The University of Edinburgh Description: Learning analytics is primarily focused on staff accessing student data to gauge performance, predict student outcomes, and make interventions where necessary. This session will provide an overview of the work that the University has done in researching and developing a bespoke learning analytics building block called, ‘Data for Students’ that allows students to access information about their course - using click counts and comparison of grades against the cohort. The session will report on the outputs of our initial pilot studies and feedback from staff and students who have used the tool.

1. Data for Students
A student-centred approach to analytics in
Learn

2. Ross Ward
Introduction
2
Information Services
Learning, Teaching & Web Services
@rosswoss
@UoE_LTW

3. Learn
Introduction
3
5500 Courses per year (and growing)
Over 70% Course Coverage
Primary platform for on-campus courses

4. Learning Analytics
Introduction
4
Rapidly Developing & Evolving
Powerful tools & reports
Support & Advice
Generally not student facing!

5. Big Data | Little Data
5
Big Data Little Data
• Tutor Facing
• Interventions
• Course Evaluation
• Student Retention
• Student Facing
• Self Monitoring
• Personal Motivation
• Student Attainment
Focus of industry Largely ignored

6. Aim of our project
• Awareness of learning analytics developments at national level
– JISC | Code of Practice for learning analytics
– Prof Dragan Gasevic
• Student Retention not an aim or outcome
• Student focussed & Student facing
• Increasing Awareness
– Existing tools and reports
– Support and guidance
– Keeping in-touch with academic ideas and practices
• Develop a student-facing building block
– What data is suitable?
– What do students want?
– Would students use it?
6

7. Tutor Facing Analytics
7
What are the expectations of our tutors?
User Stories
• Current practice
• Awareness of tools
• Risks and fears

8. Tutor Facing Analytics
8
What are tutors allowed to do?
Learning
Analytics
Steering Group
• University Policies
• Legal Requirements
• Ethical Considerations
• JISC Code of Practice

9. Tutor Facing Analytics
9
How can we support our users?
Self-enrol
Support Course
• 92% of approaches were
available
• “hand-cranked” data analysis
common
• Examples of usage
• Highlight areas for consideration

10. That’s all good and well, but…
10
What about the students?

11. Exposing data to students
11
What about the students?
Some initial research
Worked closely with Student Association
• Not really that bothered
• Mixed opinions about data privacy
• Grades! Grades! Grades!

12. Requirements for the Building Block
12
What about the students?
• Simple as possible
• Easy to interpret
• Anonymous
• Manageable by Instructors
• Opt-in by course
Clicks
&
Grades

13. Click Data
13
• Heatmap of student activity
• Personal activity only
• “Quantified Self”
• Data presented from data
centre – not live
• Course opt-in

14. Grade Data
14
• Tutor controls which grade
columns are included
• Simple to understand
• Cannot be used on small
cohort sizes

15. Pilot Courses
15
What about the students?
UG Physics
 Large cohort
 Driven & competitive students
 Tutor engaged with students about
tools
Online Distance Prof Development
 Very large cohort
 No grade data
 Ad-hoc usage of click data
(minimal)
PG Foreign Language
 Little data to this point
3
Pilot
Courses
(first semester)

16. Feedback from course
16
What about the students?
UG Physics
 Large cohort
 Driven & competitive students
 Tutor engaged with students about tools
Click Count
Focus Group
&
Survey
89%Easy to understand at first glance
69%Found the information useful
61%Would check information weekly
12%Would never check click count data

17. Feedback from course
17
What about the students?
UG Physics
 Large cohort
 Driven & competitive students
 Tutor engaged with students about tools
Click Count
Focus Group
&
Survey
allows me to plan my week
see when I am most productive
little relevance to my performance
can track my activity in the course
Visualise how often I am actually on the website
I've used this to build a
'timetable' of when i should do
work pretty pointless information
I can't think of a way in which I could utilise
the information.
The information is set out in an easy to
read way and tabulated well.
The amount of clicks per day does not
correlate with how much work you are
doing
It shows you how much (or little) you are
doing and gives an incentive to maybe
visit the learn page more often.
I have not yet found a use for it. I just find
it interesting.

18. Feedback from course
18
What about the students?
UG Physics
 Large cohort
 Driven & competitive students
 Tutor engaged with students about tools
Grade Averages
Focus Group
&
Survey
97%Easy to understand at first glance
100%Found the information useful
78%Would check information weekly
4%Would never check click count data

19. Feedback from course
19
What about the students?
UG Physics
 Large cohort
 Driven & competitive students
 Tutor engaged with students about tools
Grade Averages
Focus Group
&
Survey
Allows me to see how well I'm doing
compared to the rest of the class
Shows you if other people are struggling
with the course like you or not
allows to compare own results to the
average to give show what results I should
be getting
It gives me a good overview of how I'm
doing and which topics I struggled with
I find it very useful (and sometimes
reassuring) to compare my grades with my
peers
It's a motivating tool if I do well or not
compared to others.
Column chart is better then a table
especially for comparison
Provides me with an estimation of how
well I am coping with the course work.
It gives me assurance that i am at
the same level as the class average
in the assessments and that i am
doing ok in the course work.

20. Feedback from course
20
What about the students?
UG Physics
 Large cohort
 Driven & competitive students
 Tutor engaged with students about tools
Have you used this information to inform decisions on your
study and work patterns?
Focus Group
&
Survey
Yes
No
60.3%
39.7%

21. Feedback from course
21
What about the students?
UG Physics
 Large cohort
 Driven & competitive students
 Tutor engaged with students about tools
TheTutors Perspective
Focus Group
&
Survey

22. Summary
22
Analytics is not a
“one size fits all”
solution

23. Summary
23
Data must be easy to interpret
Fit in with existing support
and guidance offered
to students

24. Summary
24
Used at student’s
own discretion

25. Summary
25
Provide support and
advice to all

26. Summary
26
Just one tool of many
available

27. What next…
• Courses still piloting the tool
• Not currently developing building block further
– Will be making it available on Oscelot over the summer
• Keeping in-touch with other Student Facing developments
– JISCApp of interest
• Continue to engage with Student association
• Continually shifting attitudes towards analytics
27

28. Thank you
28
Thanks.
Questions?
http://www.ed.ac.uk/information-services/learning-technology