1. BETT 2013, London — LearnLive HigherEd
Learning Analytics:
Unlocking student data for
21st century learning?
Simon Buckingham Shum
Knowledge Media Institute
The Open University UK
simon.buckinghamshum.net
@sbskmi #LearningAnalytics

2. 70-strong lab prototyping next generation
learning / sensemaking / social web media
linked data / semantic web services 2

3. learning objective:
walk out with
better questions
than you can ask right now
3

4. Why are seeing this?
4

5. Why are seeing this?
5

6. Why are seeing this?
6

7. edX: “this is big data, giving us the chance
to ask big questions about learning”
https://www.edx.org/about 7

8. A recent analytics product review…
8

9. A recent analytics product review…
“Some have tried to argue that
this technology doesn't work out
cost effectively when compared to
conventional tests... but this
misses a huge point. More often
than not, we test after the event
and discover the problem — but
this is too late..”
9

10. Aquarium Analytics!
10

11. 11

12. How is your aquatic ecosystem?
“This means that the keeper can be notified before water
conditions directly harm the fish—an assured outcome of
predictive software that lets you know if it looks like the
pH is due to drop, or the temperature is on its way up.
This way, it’s a real fish saver, as
opposed to a forensic examiner,
post-wipeout.”
(From a review of Seneye, in a hobbyist magazine)
12

13. How is your learning ecosystem?
This means that the teacher can be notified before
learning conditions directly harm the students — an
assured outcome of predictive software that lets you
know if it looks like engagement is due to drop, or
distraction is on its way up.
This way, it’s a real student saver,
as opposed to a forensic
examiner, post-wipeout.
13

14. but you still need to know
what good looks like…
and what to do when it drops… 14

15. 15

16. fish
learners?
16

17. Purdue University Signals: real time traffic-
lights for students based on predictive model
17

18. Purdue University Signals: real time traffic-
lights for students based on predictive model
MODEL:
• ACT or SAT score
• Overall grade-point average
• CMS usage composite
• CMS assessment composite
• CMS assignment composite
• CMS calendar composite
Predicted 66%-80%
of struggling
students who
needed help
Campbell et al (2007). Academic Analytics: A New Tool for a New
Era, EDUCAUSE Review, vol. 42, no. 4 (July/August 2007): 40– 18
57. http://bit.ly/lmxG2x

19. Purdue University Signals: real time traffic-
lights for students based on predictive model
“Results thus far show that
students who have engaged with
Course Signals have higher
average grades and seek out help
resources at a higher rate than
other students.”
Pistilli, M. D., Arnold, K. and Bethune, M., Signals: Using Academic
Analytics to Promote Student Success. EDUCAUSE Review
Online, July/Aug., (2012).
http://www.educause.edu/ero/article/signals-using-academic- 19
analytics-promote-student-success

20. Enabling staff to
monitor courses
View profiles
and student showing predictions
academic of academic success
success in relation to success
predictions factors and cohort
Chris Ballard, Tribal Labs / @chrisaballard / www.triballabs.net

21. Predictive model relates predictions to student
success factors to help staff identify interventions
Understand patterns of student
activity and engagement with
university services
Chris Ballard, Tribal Labs / @chrisaballard / www.triballabs.net

22. predictive models
are exciting
but there are many other
kinds of analytics
22

23. Analytics in your VLE:
Blackboard: feedback to students
http://www.blackboard.com/Platforms/Analytics/Products/Blackboard-Analytics-for-Learn.aspx
23

24. Adaptive platforms generate fine-grained
analytics on curriculum mastery
https://grockit.com/research
24

25. a data-centric culture
doesn’t have to involve
advanced technology
25

26. Emerging interest in learning analytics
Professor Mark Stubbs | m.stubbs@mmu.ac.uk
• Why? Make better decisions MMU
Example: Choosing a new VLE: exploring
since 2010 …
VLE usage
Learner patterns
demographics Exam
Entry results … planning
wide
qualifications institution-
2013
support for
• Seek to correlate variables with final success/failure
• Triangulate with extensive survey and focus groups
• Result: Critical Success Factors inform
requirements for new VLE

27. analytics for lifelong,
lifewide learning?
27

28. Why do dispositions matter?
“Knowledge of methods alone
will not suffice: there must be
the desire, the will, to employ
them. This desire is an affair
of personal disposition.”
John Dewey
Dewey, J. How We Think: A Restatement of the Relation of Reflective Thinking
to the Educative Process. Heath and Co, Boston, 1933 28

29. Validated as loading onto
7 dimensions of “Learning Power”
Being Stuck & Static Changing & Learning
Data Accumulation Meaning Making
Passivity Critical Curiosity
Being Rule Bound Creativity
Isolation & Dependence Learning Relationships
Being Robotic Strategic Awareness
Fragility & Dependence Resilience
Univ. Bristol and Vital Partnerships provides practitioner resources and
tools to support their application in schools, HEIs and the workplace 29

30. ELLI: Effective Lifelong Learning Inventory
Web questionnaire 72 items (children and adult versions: used
in schools, universities and workplace)
30

31. Analytics for lifelong/lifewide
learning dispositions: ELLI
Buckingham Shum, S. and Deakin Crick, R. (2012). Learning Dispositions and Transferable Competencies: Pedagogy, Modelling and
Learning Analytics. Proc. 2nd Int. Conf. Learning Analytics & Knowledge. (29 Apr-2 May, Vancouver). Eprint: http://oro.open.ac.uk/32823

32. ELLI generates cohort data for each
dimension
32

33. EnquiryBlogger:
Tuning Wordpress as an ELLI-based learning journal
Piloting from Yr 5, to secondary, to Masters level
Standard Wordpress
editor
http://learningemergence.net/tools/enquiryblogger 33

34. EnquiryBlogger:
Tuning Wordpress as an ELLI-based learning journal
Piloting from Yr 5, to secondary, to Masters level
Categories from
ELLI
http://learningemergence.net/tools/enquiryblogger 34

35. EnquiryBlogger:
Tuning Wordpress as an ELLI-based learning journal
Piloting from Yr 5, to secondary, to Masters level
Plugin visualizes blog
categories, mirroring
the ELLI spider. Direct
navigation to blog
posts from here
35

36. EnquiryBlogger
dashboard – direct
navigation to learner’s
blogs from the visual
analytic

37. LearningEmergence.net
more on analytics for learning to learn, authentic
enquiry, leadership and complex learning systems
37

38. unpacking deeper learning
example:
online student discourse
analytics that go beyond
“number of forum posts”
+ “trending topics”
38

39. Social Network Analysis (SNAPP)
What’s going on
in these discussion forums?
Bakharia, A. and Dawson, S., SNAPP: a bird's-eye view of temporal participant interaction. In: Proceedings of the 1st 39
International Conference on Learning Analytics and Knowledge (Banff, Alberta, Canada, 2011). ACM. pp.168-173

40. Social Network Analysis (SNAPP)
40
http://www.slideshare.net/aneeshabakharia/snapp-20minute-presentation

41. Social Network Analysis (SNAPP)
2 learners connect
otherwise separate
clusters
tutor only engaging
with active students,
ignoring disengaged
ones on the edge
41
http://www.slideshare.net/aneeshabakharia/snapp-20minute-presentation

42. Social Learning Analytics about to appear in
products…
http://www.desire2learn.com/products/analytics (this is from a beta demo)
42

43. Discourse analytics: what intellectual
contribution does this learner make?
Rebecca is playing
the role of broker,
connecting peers’
contributions in
meaningful ways
De Liddo, A., Buckingham Shum, S., Quinto, I., Bachler, M. and Cannavacciuolo, L. Discourse-centric learning analytics. 1st International
Conference on Learning Analytics & Knowledge (Banff, 27 Mar-1 Apr, 2011), ACM: New York. pp.22-33 http://oro.open.ac.uk/25829

44. Semantic Social Network Analytics:
shows if users agree or disagree
De Liddo, A., Buckingham Shum, S., Quinto, I., Bachler, M. and Cannavacciuolo, L. Discourse-centric learning analytics. 1st International
Conference on Learning Analytics & Knowledge (Banff, 27 Mar-1 Apr, 2011), ACM: New York. pp.22-33 http://oro.open.ac.uk/25829

45. Discourse analytics on webinar
textchat
Can we spot the
quality learning
conversations in
a 2.5 hr webinar?
Ferguson, R. and Buckingham Shum, S., Learning analytics to identify exploratory dialogue within synchronous text chat. In: 1st
International Conference on Learning Analytics and Knowledge (Banff, Canada, 2011). ACM

46. Discourse analytics on webinar
textchat
Given a 2.5 hour webinar, where in the live
textchat were the most effective learning
conversations?
Not at the start and end of a webinar…
Sheffield, UK not as sunny See you!
as yesterday - still warm
bye for now!
Greetings from Hong Kong
bye, and thank you
Morning from Wiltshire,
80
sunny here! Bye all for now
60
40
20
0
9:28
9:32
10:13
11:48
12:00
12:05
12:04
9:36
9:40
9:41
9:46
9:50
9:53
9:56
10:00
10:05
10:07
10:07
10:09
10:17
10:23
10:27
10:31
10:35
10:40
10:45
10:52
10:55
11:04
11:08
11:11
11:17
11:20
11:24
11:26
11:28
11:31
11:32
11:35
11:36
11:38
11:39
11:41
11:44
11:46
11:52
11:54
12:03
-20
-40
Average Exploratory
-60

47. Discourse analytics on webinar
textchat
Given a 2.5 hour webinar, where in the live
textchat were the most effective learning
conversations?
Not at the start and end of a webinar
but if we zoom in on a peak…
80
60
40
20
0
9:28
9:32
10:13
11:48
12:00
12:05
12:04
9:36
9:40
9:41
9:46
9:50
9:53
9:56
10:00
10:05
10:07
10:07
10:09
10:17
10:23
10:27
10:31
10:35
10:40
10:45
10:52
10:55
11:04
11:08
11:11
11:17
11:20
11:24
11:26
11:28
11:31
11:32
11:35
11:36
11:38
11:39
11:41
11:44
11:46
11:52
11:54
12:03
-20
-40
Average Exploratory
-60

48. Discourse analytics on webinar
textchat
Given a 2.5 hour webinar, where in the live
textchat were the most effective learning
conversations?
Not at the start and end of a webinar
but if we zoom in on a peak…
Classified as
“exploratory
talk”
(more
substantive
100 for learning)
50
0
9:28
“non-
9:40
9:50
10:00
10:07
10:17
10:31
10:45
11:04
11:17
11:26
11:32
11:38
11:44
11:52
12:03
-50 exploratory”
Averag
-100

49. “Rhetorical parsing” to identify constructions
signifying scholarly writing
OPEN QUESTION:
“… little is known …”
“… role … has been elusive”
“Current data is insufficient …”
CONTRASTING IDEAS:
“… unorthodox view resolves …”
“In contrast with previous
SURPRISE: hypotheses ...”
“We have recently observed ... “... inconsistent with past
surprisingly” findings ...”
“We have identified ... unusual”
“The recent discovery ... suggests
intriguing roles”
http://technologies.kmi.open.ac.uk/cohere/2012/01/09/cohere-plus-automated-rhetorical-annotation
De Liddo, A., Sándor, Á. and Buckingham Shum, S., Contested Collective Intelligence: Rationale, Technologies, and a Human-Machine Annotation
Study. Computer Supported Cooperative Work, 21, 4-5, (2012), 417-448. http://oro.open.ac.uk/31052

50. “What are the key contributions of this text?
Human analyst Computational analyst
http://technologies.kmi.open.ac.uk/cohere/2012/01/09/cohere-plus-automated-rhetorical-annotation
De Liddo, A., Sándor, Á. and Buckingham Shum, S., Contested Collective Intelligence: Rationale, Technologies, and a Human-Machine Annotation
Study. Computer Supported Cooperative Work, 21, 4-5, (2012), 417-448. http://oro.open.ac.uk/31052

51. learning objective
– how are we doing?
walk out with
better questions
than you could ask 30mins ago
51

52. How will my org. evolve from a digital
exoskeleton to a nervous system?
Ed Dumbill: http://strata.oreilly.com/2012/08/digital-nervous-system-big-data.html 52

53. The Wal-Martification of education?
“What counts as
data, how do you get
it, and what does it
actually mean?”
“The basic question is not
what can we measure?
The basic question is
“data narrowness” what does a good
“instrumental learning” education look like?
“students with no curiosity” Big questions.
http://chronicle.com/blogs/techtherapy/2012/05/02/episode-95-learning-analytics-could-lead-to-wal-martification-of-college 53
http://lak12.wikispaces.com/Recordings

54. Analytics provide maps
= systematic ways of distorting reality
in order to reduce complexity
“A marker of the health of the
learning analytics field will be
the quality of debate around
what the technology renders
visible and leaves invisible.”
Buckingham Shum, S. and Deakin Crick, R. (2012). Learning Dispositions and Transferable Competencies:
Pedagogy, Modelling and Learning Analytics. Proc. 2nd Int. Conf. Learning Analytics & Knowledge. (29
Apr-2 May, 2012, Vancouver, BC). ACM: New York. Eprint: http://oro.open.ac.uk/32823

55. Will your staff know how to
read and write analytics?
This will become a key literacy.
55

56. What if you engaged your
learners in the co-design of
the analytics which will track
them?
Think about the conversations
you’d need to have…
56

57. Are you ready for
your performance indicators
to be computed from analytics?
57

58. Our analytics are our
pedagogy
They promote assessment regimes
— which drive (and strangle)
educational innovation
58

59. Join the community…
SoLAResearch.org / @SoLAResearch
LAKconference.org / @LAKconf
59

60. Learning Analytics Policy Brief
Exec Summary for UNESCO IITE
http://bit.ly/LearningAnalytics 60

61. BETT 2013, London — LearnLive HigherEd
Learning Analytics:
Unlocking student data for
21st century learning?
Simon Buckingham Shum
Knowledge Media Institute
The Open University UK
simon.buckinghamshum.net
@sbskmi #LearningAnalytics