Presentation given at Learning Analytics Summer Institute 2013. Theories of learning postulate multiple agencies (individual, small group, and collective) and epistemologies e.g., acquisition, intersubjective meaning making, participation). Though we may research these separately, learners experience all of these at once, so learning is a complex phenomenon. Need to connect levels of analysis. Also need to bring in multiple "voices" or theoretical and research traditions, and learn how to manage productive multivocality among them. Two efforts towards this end are briefly described. If it takes on these challenges, Learning Analytics can help by enabling us to manage multiple levels of analysis.

1. Supported by the National Science Foundation
Challenges for
Learning Analytics
Dan Suthers
University of Hawaii
Learning as a
Complex
Phenomenon:

2. Learning in Socio-Technical Networks
Agency
Who or what is the agent
that learns?
 Individual
 Small groups
 Networks (communities,
cultures, societies)
Epistemologies
What is the process of
learning?
 Acquisition
 Intersubjective meaning-
making
 Change in Participation
The correspondence is not strict, and analysis can be
applied from local to network levels
How do socio-technical settings foster learning?
Based on  Suthers (ijCSCL 2006)

3. Examples
 Individual Epistemologies
Learning as acquisition of information, knowledge or skills
– Local: contribution theory, equilibration, given/new
(epistemological gradients in explanation), practice, etc.
– Network: strength of weak ties, diffusion theories (contagion
theory, diffusion of innovations)
 Intersubjective epistemologies
Learning as intersubjective meaning-making
– Local: argumentation, co-construction, group cognition
– Network: Knowledge building, communities of scientists
 Participatory epistemologies
Learning as changes in social participation and identity
– Local: apprenticeship, mentoring ...
– Network: Legitimate Peripheral Participation in a CoP

4. Challenges for Learning Analytics
Claim: individuals participate in the foregoing forms of
learning simultaneously
 Challenge to rise above one-dimensional analytics:
How does learning (enhancements of knowledge,
skills, and cultural capital) take place through the
interplay between individual and collective agency in
socio-technical networks?
 Demands analyses that connect learning activity in
specific times and places with the larger socio-
technical network contexts in which they take place
 Will require coordinating multiple analytic methods
(and their traditions)

5. Traces* Analytic Hierarchy
 Activity is distributed across media:
– Traces of activity are fragmented across multiple logs,
breaking up participants’ singular experience
– Reunite traces of interaction into a unified analytic artifact
 Logs may record activity in the wrong ontology:
– Abstract event data to other appropriate levels of description
 Behavior is contingent on the resources of the setting
in diverse ways, and setting may be non-local in time
and space
– Sequential interaction analysis and aggregate network
analysis are complementary
– Enable mapping between these descriptions both ways
 Suthers (HICSS 2011),  Suthers & Rosen (LAK 2011),
* “Traces” NSF VOSS project

6. Contingencies
Mediated Associations
Uptake Ties
Interaction Affiliations

7. Productive Multi-Vocality Project
Learning sciences are diverse: how to bring multiple
analytic “voices” into productive dialogue to provide
some coherence?
 Sharing/comparing approaches to analyzing
collaborative learning
– 5+ years, 37+ researchers, 5 corpora, 1 book!
– Shift from technical focus (shared tools) to social/dialogical
focus (productive multivocality) between epistemologies as
well as theories
 PMV ≠ mixed methods:
– Multiple voices (agency)
– Productive tensions in addition to harmonious use
 Suthers, Lund, Rosé, Dyke, Law, Teplovs, et al. (CSCL 2011)

8. Strategies for Productive Multivocality
 Dialogue about the same data, from different
perspectives
– Issues in agreement on what data is worth considering
 Share an analytic objective (e.g., “pivotal moments”)
– Vague, so interpretable by each tradition (boundary object)
 Bring analytic representations into alignment with
each other and the original data
– Good tools help
 Eliminate inconsequential differences and Iterate
– Focus on more essential differences and convergences
 Push the boundaries of traditions without betraying
 Issues: appropriateness of data; extensions of concepts
 Reflect on Practice: dialogue about methods as
object-constituting, evidence-producing and argument-
sustaining tools

9. Summary:
Challenges for Learning Analytics
Learning in socio-technical settings involves
multiple agencies and processes
➠ Requires analysis across 'local' and 'network'
'levels' (Traces project)
➠ Requires coordination of diverse disciplinary
traditions (Productive Multivocality Project)
Learning Analytics can help us understand and
manage learning in its full complexity
Thank You / Mahalo / Danke / Merci / Domo Arigato / Xie Xie / G’Day
Dan Suthers, Suthers@hawaii.edu
Supported by the National Science Foundation and many colleagues