Presentación de Davinia Hernández-Leo, profesora de la Universitat Pompeu Fabra (UPF), Barcelona: «Apoyo al profesorado con analíticas de comunidad, diseño y aprendizaje»
10_05_2019 Seminario eMadrid sobre «Tecnologías de la educación dentro y fuera del aula» / UAM
1. Apoyo al profesorado con analíticas de comunidad, diseño y aprendizaje
Supporting teachers with community, design and learning analytics
eMadrid, UAM, 10 Mayo 2019
Davinia Hernández-Leo
TIDE, Universitat Pompeu Fabra, Barcelona
twitter: @TIDE_UPF
website : http://www.upf.edu/web/tide
3. Support teachers in designing the “best possible”
conditions for their students to learn, documenting them,
making their design ideas explicit and sharable.
(Conole, 2012; Agostinho, 2011; Laurillard, 2013;
Mor, Craft & Hernández-Leo, 2013)
Learning Design
3
Teaching as a design science
5. The case of CSCL macro-scripting
Free collaboration does not necessarily lead to fruitful
learning. An option is to structure the collaborative learning
flow (activity sequence, roles, group formation, resource
allocation) to trigger potentially effective social interactions.
(Dillenbourg & Tchounikine, 2007; Kobbe et al., 2007)
Learning Design in Collaborative Learning
5
6. Learning design authoring and enactment
— Pyramid collaborative learning flow pattern. Scalable approach. Flow
creation and control mechanisms for elasticity and dynamism
6
Learning Design in CSCL: example of PyramidApp
https://www.upf.edu/web/tide/tools
Manathunga, K., & Hernández‐Leo, D. (2018). Authoring and enactment of mobile pyramid‐based
collaborative learning activities. British Journal of Educational Technology, 49(2), 262-275.
7. Design layers metaphor
(by spaces)
Systematically co-designed
with teachers
Emphasis in design for
blended learning
(PBL, flipped classroom, …)
7
Learning Design in blended learning: edCrumble
https://www.upf.edu/web/tide/tools
Albo L, Hernández-Leo D. edCrumble: designing for learning with data analytics. EC-TEL 2018. Best Demo Award.
8. 8
Learning design tools
Tools for conceptualization, authoring, implementation
Generic vs. specific tools wrt. pedagogical approach
Sharing, co-creating learning designs
9. 9
Integrated Learning Design Environment
Integrated solution to support the full lifecycle of (co-)designing,
sharing, and deploying ICT-based learning activities
Hernández-Leo, D., et al. (2018). An Integrated Environment for Learning Design. Frontiers in ICT, 5, 9
11. ILDE, previous results
Used in several different educational contexts
(higher education, vocational education, adult education)
— Well accepted in all its functions
— Fits the needs of different types of communities
— No difference in acceptance
— Flexible design processes
— Limited social awareness and inquiry process
Data used to evaluate and understand processes…
11
12. Learning design and data
Data used to evaluate and understand processes…
How can data be used to support learning design and learning
design communities?
12
13. Data Analytics in teaching & learning
Learning Analytics (LA): “the measurement, collection, analysis, reporting of
data about learners and their contexts, for porpuses of understanding and
optimizing learning and the environments in which it occurs” SoLAR (Ferguson,
2012)
Social Learning Analytics (Buckingham-Shum & Ferguson 2012) & Community
analytics and visualizations (Klamma, 2013; Vasileva and Sun, 2007), including Social
Network Analysis (SNA) (de Laat & Schreurs, 2013)
13
14. ANALYTICS LAYERS FOR LEARNING DESIGN FRAMEWORK
Supporting learning design with analytics
14
Community of practitioners
and related stakeholders
Learning Design (LD) tools
Learners experiencing
learning designs
Community Analytics
metrics and patterns of design activity
Design Analytics
metrics of pedagogical decisions
Learning Analytics
metrics of learners’ engagement,
achievement... aligned with design intent
Hernández-Leo, D., et al. (2019) Analytics for learning design: A layered framework and tools, British Journal of Educational
Technology. https://doi.org/10.1111/bjet.12645
16. Available data
ILDE2, extension with data analytics features
— https://www.upf.edu/web/tide/tools/ilde2
CoT project, 2 schools
RESET, Smartlet, Illuminated:
— Master program for teacher training
— MOOC for teachers
— Teacher training workshops
Supporting learning design with analytics
16
17. Analytics for learning design
Community of practitioners
and related stakeholders
Community Analytics
metrics and patterns of design activity
Understanding community behaviour
Triggering orientation and inspiration
(e.g. awareness, provides a context for your own activity)
Data includes:
— what tools are being used,
— what types of designs are created, tags,
— who are the (co-)authors, edits, views,
— versioning of designs,
— social ratings …
Michos, K., Hernández-Leo, D., (2016) Understanding
Collective Behavior of Learning Design Communities.
EC-TEL 2016.
Garreta-Domingo, M., Sloep, P., Hernández-Leo, D., Mor, Y.,
(2018) Design for collective intelligence: Pop-up communities in
MOOCs. Journal AI & Society 33(1), 91-100.
18. Analytics for learning design
Community of practitioners
and related stakeholders
Community Analytics
metrics and patterns of design activity
Understanding community behaviour
Triggering orientation and inspiration
Data includes:
— what tools are being used,
— what types of designs are created, tags,
— who are the (co-)authors, edits, views,
— versioning of designs,
— social ratings …
Duplicates ….
19. Analytics for learning design
Community of practitioners
and related stakeholders
Community Analytics
metrics and patterns of design activity
Understanding community behaviour
Triggering orientation and inspiration
Data includes:
— what tools are being used,
— what types of designs are created, tags,
— who are the (co-)authors, edits, views,
— versioning of designs,
— social ratings …
Michos, K., Hernández-Leo, D., (2018) Supporting awareness in communities of
learning design practice, Computers in Human Behavior, 85, 255-270.
21. Blending the dashboard with learning design tasks
With/Without dashboard use
Differences in community members´ participation and perceptions
Community dashboard evaluation
21
22. Dashboard use increased social interactions by means of comments and profile views
Positive user experience (UMUX questionnaire) for understanding community participation
Alignment with tasks for learning design
Community dashboard evaluation
22
Variables Group N Ranks U Z p r
profiles views High dashboard use 25 31.26 138.50 -3.238 .001 -0.46
Low dashboard use 24 18.27
comments High dashboard use 25 30.70 157.50 -2.877 .004 -0.41
Low dashboard use 24 19.06
created designs High dashboard use 25 30.12 172.00 -2.570 .010 -0.36
Low dashboard use 24 19.67
re-used designs High dashboard use 25 30.08 173.00 -2.547 .011 -0.36
Low dashboard use 24 19.71
edits High dashboard use 25 29.60 185.00 -2.335 .020 -0.33
Low dashboard use 24 20.21
23. Learning Design (LD) tools
Design Analytics
metrics of pedagogical decisions
Categorise learning designs
Support awareness and reflection
about design decisions
Data includes:
— learning objectives,
— type of tasks tasks,
— the social planes (individual, collaborative),
— resources and tools associated with the task,
— expected length of time for tasks, …
Analytics for learning design
Albo L, Hernández-Leo D. edCrumble: designing for learning with data
analytics. EC-TEL 2018. Best Demo Award.
24. Learning Design (LD) tools
Design Analytics
metrics of pedagogical decisions
Categorise learning designs
Support awareness and reflection
about design decisions
Interaction with the
Community Analytics layer
Analytics for learning design
Albo L, Hernández-Leo D. edCrumble: designing for learning with
data analytics. EC-TEL 2018. Best Demo Award.
25. Learners experiencing
learning designs
Learning Analytics
metrics of learners’ engagement,
achievement... aligned with design intent
Accumulated evidence of design’s impact
Support awareness and reflection
about effects of designs, redesign
Data includes:
— snapshot data capturing relevant milestones
(e.g., access to resources or completion of tasks)
— data about the process (e.g., participation in
discussions)
— performance (e.g., grades)
— students’ and teachers’ opinions…
Analytics for learning design
Michos, K., Manathunga, K., Hernández-Leo, D. (2016). Connecting pattern-based
learning designs with analytics: The case of the PyramidApp., CLAD - EC-TEL 2016
26. Learners experiencing
learning designs
Learning Analytics
metrics of learners’ engagement,
achievement... aligned with design intent
Accumulated evidence of design’s impact
Support awareness and reflection
about effects of designs, redesign
Data includes:
— snapshot data capturing relevant milestones
(e.g., access to resources or completion of tasks)
— data about the process (e.g., participation in
discussions)
— performance (e.g., grades)
— students’ and teachers’ opinions…
Analytics for learning design
Michos, K., Fernández, A., Hernández-Leo, D., & Calvo, R.
(2017). Ld-Feedback App: Connecting Learning Designs with
Students’ and Teachers’ Perceived Experiences. EC-TEL.
27. Learners experiencing
learning designs
Learning Analytics
metrics of learners’ engagement,
achievement... aligned with design intent
Accumulated evidence of design’s impact
Support awareness and reflection
about effects of designs, redesign
Interaction with Design Analytics
and Community Analytics
— Teacher inquiry, Community inquiry
Analytics for learning design
Michos, K., & Hernández-Leo, D., Albó, L. (2018). Teacher-led
inquiry in technology-supported school communities. British
Journal of Educational Technology 49(6), 1077-1095.
29. Teacher produced artifacts & Learning Analytics
29
Learning Analytics reports
Teacher produced artifacts
Reflections on the enactment
30. Some results
Awareness and regulation of design activity (before, during, and after enactment)
Connection between initial design and data-informed reflections
Documentation for later review
Sharing learning design problems with solutions, with contextual information
Teacher roles as designers is guided by practicality and time constraints
In-depth discussions about teaching and learning strategies with the use of student generated data
and TILE
30
31. Conclusions
31
Data driven learning design support in teacher communities
Support to teachers
Three analytics layers: Community, Design and Learning analytics
And interplays!
Awareness, inspiration, reuse, teacher (and community) inquiry, learning redesign
Future work:
— New communities (e.g., Maker activities)
— Motivations, incentives
— Socio-emotional aspects and learning design (Spotlighters project)
— Actionable dashboard for PyramidApp
Concerned about ethical aspects, e.g.
— Beardsley, M., Santos, P., Hernández-Leo, D., Michos, K. (2019). Ethics in educational technology research:
informing participants in data sharing risks. British Journal of Educational Technology. 10.1111/bjet.12781