Keynote, Library Data Carpentry 2016, Sydney http://librarydatacarpentry.github.io

1. Who are you and makes you special?
Simon Buckingham Shum
Professor of Learning Informatics
Director, UTS Connected Intelligence Centre
@sbuckshum • Simon.BuckinghamShum.net
utscic.edu.au
Keynote, Library Data Carpentry 2016, Sydney
http://librarydatacarpentry.github.io

2. Learning
Technology
KMi,
Open
U.
AI
&
Argumenta<on
Learning
Disposi<ons
Human-­‐Centred
Informa<cs
Learning
Analy<cs
Seman<c
Scholarly
Publishing
Dialogue
/
Issue
/
Argument
Visualisa<on
Introducing my
quantified
background
(at least, in Nov. 2013
courtesy LinkedIn Labs)

3. OUR CONTEXT
3

4. OUR CONTEXT
4
Large scale data and analytics
are pervading societal life
Data and Algorithms have deep societal
implications – good and bad – demanding
informed debate
Implications for the future workforce…
How universities teach, research, operate —
and are assessed…
How to equip graduates for
“the age of complexity” (Stephen Hawking)

5. 2011
2011
Envisioning “the Data
Intensive University”
UTS-wide Forum to consider
the profound implications of
the data revolution
Followed by UTS-wide
consultation, strategy devpt,
and launch of CIC
UTS STRATEGIC CONVERSATION AROUND ANALYTICS

6. UTS CONNECTED INTELLIGENCE CENTRE
6
CIC catalyses the use of data and
analytics among UTS students,
educators, researchers and leaders
We teach human-centred data science •
design analytics tools for UTS • evaluate
these • disseminate internally and
globally
We aim to shape critical debate on big
data in education, and societal learning

7. PARTNERING
ACROSS UTS
7
Faculties
&
Institutes
Student
Support
Units
CIC
Business
Units

8. “LibrAIrian”a University Library staff member who advises
students, educators and researchers
on the uses and abuses of
AI, Data Science and Human-Centered Computing
for learning, knowledge and innovation

10. What’s the difference between
a LibrAIrian and a data technician?

11. Panel debate,
LAK 2013
With thanks to
John Behrens
(Pearson)
hIp://simon.buckinghamshum.net/2013/03/lak13-­‐edu-­‐data-­‐scien<sts-­‐scarce-­‐breed

12. Things that make me crazy

13. “Looks cool. What does it
mean?”
“I don’t know”

14. “Looks great at a high level,
how have you explored it
and tested the
assumptions?”
“What assumptions?”

15. “What are the relevant
background issues in this
area?”
“Ask the SME, I’m just the
data person”

16. “Does it really make sense
to get that result?”
“I don’t know, it just came
out that way”

17. Be a philosopher

18. critical perspective
algorithmic + scholarly + creative
intelligence
18

19. http://www.uts.edu.au/future-students/analytics-and-data-science

20. A DISTINCTIVE APPROACH TO DATA SCIENCE
utscic.edu.au
machine learning • statistics • data curation • ethics • user experience • information science
visualization • narrative • social computing • learning analytics • project management
project-based learning • authentic assessment
regular, meaningful employer engagement

21. What’s the difference between
a LibrAIrian and a learning analytics
technician?

22. critical perspective
learning analytics
means many things to many people
learning analytics are not neutral
22

23. It’s out of the labs and into products: every learning tool
now has an “analytics dashboard” (a Google image search)
23

24. https://guides.instructure.com/m/4152/l/66789-what-are-course-analytics
Summary statistics in the LMS (Canvas)
24

25. https://grockit.com/research
Written skills mastery for a SAT
25

26. Intelligent tutoring for skills mastery (CMU)
Lovett M, Meyer O and Thille C. (2008) The Open Learning Initiative: Measuring the effectiveness of the OLI statistics course in accelerating student
learning. Journal of Interactive Media in Education 14. http://jime.open.ac.uk/article/2008-14/352
“In this study, results showed that
OLI-Statistics students [blended
learning] learned a full semester’s
worth of material in half as much
time and performed as well or
better than students learning from
traditional instruction over a full
semester.”
26

27. Purdue University Signals: real time traffic-lights for
students based on predictive model
27
Campbell et al (2007). Academic Analytics: A New Tool for a New Era, EDUCAUSE
Review, vol. 42, no. 4 (July/August 2007): 40–57. http://bit.ly/lmxG2x
Validate a statistical model from:
• ACT or SAT score
• Overall grade-point average
• CMS usage composite
• CMS assessment composite
• CMS assignment composite
• CMS calendar composite
Predicted 66%-80% of struggling
students who needed help

28. Spatial clustering algorithm to provoke reflection
28Eric Coopey, R. Benjamin Shapiro, and Ethan Danahy. 2014. Collaborative spatial classification. In Proceedings of the 4th International Conference on
Learning Analytics & Knowledge (LAK '14). ACM, New York, NY, USA, 138-142. DOI= http://dx.doi.org/10.1145/2567574.2567611

29. Co-located collaboration spaces
Analyse the students’ activity
traces for significant patterns
Timely feedback for personal and
team reflection

30. Co-located collaboration spaces
…can now be instrumented with sensors
Voice
Gesture
Pen
Touch

31. Co-location activity dashboards
Multimodal data fusion and analysis… …to deliver visual analytics for reflection
e.g. this dashboard shows team member
participation on different modalities
Applications for researchers working on high
performance teams; group dynamics?
R. Martinez, K. Yacef, J. Kay, and B. Schwendimann.
An interactive teacher’s dashboard for monitoring multiple groups
in a multi-tabletop learning environment. Proceedings of
Intelligent Tutoring Systems, pages 482-492. Springer, 2012.
Voice
Gesture
Pen
Touch

32. Visual analytics of f-f teamwork
R. Martinez, K. Yacef, J. Kay, and B. Schwendimann.
An interactive teacher’s dashboard for monitoring
multiple groups in a multi-tabletop learning
environment. Proceedings of Intelligent Tutoring
Systems, pages 482-492. Springer, 2012.

33. A field exercise…
33

34. Posture analysis of fieldwork students
34
Masaya Okada and Masahiro Tada. 2014. Formative assessment method of real-world learning by integrating heterogeneous elements of
behavior, knowledge, and the environment. Proceedings 4th International Conference on Learning Analytics and Knowledge (LAK '14).
ACM, New York, NY, USA, 1-10. DOI= http://dx.doi.org/10.1145/2567574.2567579

35. 1st International Workshop on
Discourse-Centric Learning Analytics
analytics that look beneath
the surface, and quantify
linguistic proxies for ‘deeper
learning’
Beyond number / size / frequency
of posts; ‘hottest thread’
http://www.glennsasscer.com/wordpress/wp-content/uploads/2011/10/iceberg.jpg
http://solaresearch.org/events/lak/lak13/dcla13

36. Highlighted sentences are colour-
coded according to their broad type
Sentences have Function Keys signalling where
an academic rhetorical move has been
recognised (e.g. a claim of Novelty )
AWA: Academic Writing Analytics
ANALYTICAL writing
https://utscic.edu.au/tools/awa

37. Reflective writing (Nursing)
Applications for researchers working with text
corpora, e.g. interview transcripts; literature
analysis; scenario planning?
Buckingham Shum, S., Ágnes Sándor, Rosalie Goldsmith, Xiaolong Wang, Randall Bass and Mindy McWilliams (2016, In Press). Reflecting on Reflective Writing
Analytics: Assessment Challenges and Iterative Evaluation of a Prototype Tool. 6th International Learning Analytics & Knowledge Conference (LAK16). Edinburgh, UK.
ACM Press. http://dx.doi.org/10.1145/2883851.2883955 Preprint: http://bit.ly/LAK16paper

38. Educa<onal
worldview
38
epistemology
pedagogyassessment
Knight, S., Buckingham Shum, S. and Littleton, K. (2014). Epistemology, Assessment, Pedagogy: Where Learning Meets Analytics in the Middle Space.
Journal of Learning Analytics, 1, (2), pp.23-47. http://epress.lib.uts.edu.au/journals/index.php/JLA/article/download/3538/4156
Knight, S. and Buckingham Shum, S. (In Press). Theory & Learning Analytics. Handbook of Learning Analytics & Educational Data Mining.
the
middle
space of
learning analytics
What epistemological
assumptions are shaping the
assessment regime, and hence
the pedagogy?
What questions are analytics used
to help answer?

39. To go deeper into analytics for “21st century competencies”
39
hIp://simon.buckinghamshum.net/2015/05/cfp-­‐learning-­‐analy<cs-­‐for-­‐c21-­‐competencies
Contributions are invited to this special issue:
• Analytics for higher order competencies such as critical thinking,
curiosity, resilience, creativity, collaboration, sensemaking, self-
regulation, reflection/meta-cognition, transdisciplinary thinking, or
skilful improvisation
• Theoretical arguments around the opportunities, or indeed the limits, for
analytics in illuminating particular competencies
• Principles and methodologies for combining complementary analytical
approaches, including reflections on conventional educational
assessment instruments, and computational approaches
• Methodologies for validating analytics
• Analytics for learning dispositions/mindsets/“non-cognitive” factors
known to shape readiness to engage in learning
• Analytics for different kinds of authentic assessment and inquiry-based
learning
• Technological challenges and opportunities for lifelong, life-wide
learning analytics extending beyond formal educational contexts
• Arguments regarding whether analytics could effect a shift in the
assessment regimes, and associated pedagogies and epistemologies,
promoted by conventional education policy
• Analysis of the systemic organisational adoption issues for such
analytics
• Visualisation design for different user groups, in particular, to promote
increasing learner self-awareness and capacity to take responsibility for
one’s learning
Next
Special
Issue
(due
July
2016)

40. What’s the difference between
a LibrAIrian and a knowledge
infrastructure scholar?

41. critical perspective
knowledge infrastructures
embody values and assign power
41

42. Framing future knowledge
infrastructures
http://knowledgeinfrastructures.org

43. Framing future knowledge
infrastructures
http://knowledgeinfrastructures.org
This too, however, is not a neutral
feature. As knowledge
infrastructures shape, generate
and distribute knowledge, they do
so differentially, often in ways that
encode and reinforce existing
interests and relations of power.
[…] At scale, the effect of these
choices may be an aggregate
imbalance in the structure and
distribution of our knowledge.

44. Framing future knowledge
infrastructures
http://knowledgeinfrastructures.org
“Transformative infrastructures cannot be merely
technical; they must engage
fundamental changes in our
social institutions, practices,
norms and beliefs as well. For that
reason, many scholars have dropped the dualistic
vocabulary of “technical” and “social” altogether as
anything other than a first order approximation,
replacing those terms with concepts such as
collectives (Latour 2005),
assemblages (Ong & Collier 2005), or
configurations (Suchman 2007…”

45. Accounting tools are not neutral
Du Gay, P. and Pryke, M. (2002) Cultural Economy: Cultural Analysis and Commercial Life. Sage, London. pp. 12-13
“accounting tools...do not simply
aid the measurement of economic
activity, they shape the
reality they measure”
45

46. Bowker, G. C. and Star, L. S. (1999). Sorting Things Out: Classification and Its Consequences. MIT Press, Cambridge, MA, pp. 277, 278, 281
“Classification systems provide both a
warrant and a tool for forgetting
[...] what to forget and how to forget it
[...] The argument comes down to asking
not only what gets coded in but what gets
coded out of a given scheme.”
46

47. Selwyn, N. (2014). Data entry: towards the critical study of digital data and education. Learning, Media and Technology.
http://dx.doi.org/10.1080/17439884.2014.921628
“observing, measuring, describing,
categorising, classifying, sorting, ordering
and ranking). […] these processes of meaning-making are never
wholly neutral, objective and ‘automated’ but are fraught with
problems and compromises, biases and
omissions.
47

48. To learn more…
https://youtu.be/RVgXvmeSnUk

49. http://governingalgorithms.org
In
an
increasingly
algorithmic
world
[…]
What,
then,
do
we
talk
about
when
we
talk
about
“governing
algorithms”?
49

50. To learn more…
50
hIp://simon.buckinghamshum.net/2016/03/algorithmic-­‐accountability-­‐for-­‐learning-­‐analy<cs

51. “LibrAIrian”a University Library staff member who advises
students, educators and researchers
on the uses and abuses of
AI, Data Science and Human-Centered Computing
for learning, knowledge and innovation
51

52. Thank You!
Discussion…
@sbuckshum