Presentation of my research work to PAWS research group, during my visit to the School of Information Sciences of the University of Pittsburgh. 26th February, 2019.

1. Supporting educators as designers of complex blended learning
scenarios: visualization model and analytics
Laia Albó
TIDE, Universitat Pompeu Fabra, Barcelona
Twitter: @TIDE_UPF @LaiaAlbo
Website: http://www.upf.edu/web/tide
Research visit - University of Pittsburgh February 26th, 2019

2. Contents
1. Universitat Pompeu Fabra (UPF)
2. TIDE research group
3. My research
4. edCrumble Demo
5. Brainstorming
2

3. 1. Universitat Pompeu Fabra (UPF)
Public, Young, Urban
1st Spanish university in scientific productivity
1st Spanish university in THE
12th in the World (under 50 years)
https://www.youtube.com/watch?v=8dgvWkICCr8
https://www.youtube.com/watch?v=FjAuggflTVE
3
ICT Department
Engineering degrees &
Interdisciplinary Master's Programs
45 permanent positions, 9 ERCs, 5 spin-off companies, 888
undergraduate students, 180 Master and PhD students

4. 2. TIDE research group
4
https://www.upf.edu/web/tide/people

5. 2. TIDE research group
Learning Design
Support teachers in designing the “best possible” conditions for their students to
learn, documenting them, making their design ideas explicit and shareable.
(Conole, 2012; Agostinho, 2011; Laurillard, 2013; Mor, Craft & Hernández-Leo, 2013)
5

6. 2. TIDE research group - ILDE
6
https://www.upf.edu/web/tide/people
Hernández-Leo, D., Asensio-Pérez et al. (2018). An Integrated Environment for Learning Design. Frontiers in ICT, 5, 9
Integrated solution to support the full lifecycle of (co-)designing, sharing, and deploying ICT-
based learning activities

7. 2. TIDE research group - Analytics layers
7
Analytics layers for learning design framework (AL4LD framework)
Community Analytics
metrics and patterns of design activity
Design Analytics
metrics of pedagogical decisions
Learning Analytics
metrics of learners’ engagement,
achievement... aligned with design intent
Hernández-Leo, D., Martinez-Maldonado, R., Pardo, A., Muñoz-Cristóbal, J. A., & Rodríguez-Triana, M. J. et al. (available online)
Analytics for learning design: A layered framework and tools, British Journal of Educational Technology. https://doi.org/10.1111/bjet.12645

8. 2. TIDE research group - Analytics layers
8
https://www.upf.edu/web/tide/people

9. 2. TIDE research group - Main focus
9
● Data-driven understanding and support for learning design
● Three analytics layers: Community, Design and Learning analytics. And interplays!
● Teacher inquiry, learning redesign, awareness, inspiration, community inquiry
● Currently:
○ Extending experimentation
○ Physiological data, multimodal analytics (Milica Vujovic)
○ “Across-spaces” data, adaptive scripting (Ishari Amarasinghe)
○ Neuroscience and learning design (Marc Beardsley)
○ Understanding ethical implications in procedures

10. Contents
1. Universitat Pompeu Fabra (UPF)
2. TIDE research group
3. My research
4. edCrumble Demo
5. Brainstorming
10

11. Research goal
Study how technology can support educators in the design of blended learning
approaches with MOOC and MOOC resources in campus.
11

12. Why Blended MOOCs?
● The use of MOOCs as part of the face-to-face (f2f) regular university courses has emerged as a new
form of blended learning (BL).
● MOOC-based BL designs have spread taking different forms and combinations
(Delgado-Kloos, Muñoz-merino, Alario-hoyos, Ayres, & Fernández-Panadero, 2015; Pérez-Sanagustín et al., 2017;
Rayyan et al., 2016)
● BL has a positive impact on the teaching and learning effectiveness
(Garrison & Kanuka, 2004; López-Pérez, Pérez-López, & Rodríguez-Ariza, 2011; Means, Murphy, & Baki, 2013; Moskal,
Dziuban, & Hartman, 2013; Sharpe, Benfield, Roberts, & Francis, 2006),
● The studies of effectiveness lack consistency in what constitutes BL environments, and what
outcomes are being compared
(Siemens, Gašević, & Dawson, 2015)
12

13. Why Blended MOOCs?
● Limited evidence on which pedagogy or technology influence learning outcomes in BL scenarios
(Arbaugh, 2014; Littlejohn & Pegler, 2007; Torrisi-steele & Drew, 2013)
● Very little is currently known about the best ways of designing effective MOOC-based BL
(Bralić & Divjak, 2018)
13

14. Why Blended MOOCs?
The use of a systematic way of representing BL designs:
1. would facilitate their comprehension,
2. would allow the sharing and comparison between the outcomes of the different blended LDs to
study their effectiveness in a more accurate way
3. to finally lead to improve these types of practices.
● Which can be the main elements of a MOOC-based BL design representation?
● How these elements can be articulated in a visual representation?
● How analytics can support the design of BL?
● ...
14

15. Contributions
1. Analytical studies of the state of the problem
○ A survey exploring the teachers’ barriers on adopting hybrid MOOCs.
○ Analysis of the use of MOOCs by the higher education students.
1. Models, challenges and implications for design of MOOC-based hybrid pedagogies
○ Experimenting new approaches of hybrid courses using video-based learning in a traditional campus.
○ Participation on two hybrid MOOCs cases.
○ Performance of a quasi-experiment exploring how the use of different mobile devices influences the students’
engagement, behaviour and experience watching academic online videos in a collaborative classroom.
1. Learning design tool and analytics to guide professors in developing blended learning scenarios
(with MOOCs and MOOCs’ resources).
○ Visual representation of hybrid learning designs.
○ Authoring tool development.
15

16. Research goal
Study how technology can support teachers in the design of complex blended
learning approaches with MOOC and MOOC resources in campus.
16

17. Methodology: design-based research
17

18. Methodology: design-based research
18

19. Phase 1: problem indentification (I)
[Pub 1] Albó, L., Hernández-leo, D., & Oliver, M. (2015). Blended MOOCs: university teachers’ perspective.
In HybridEd Workshop, EC-TEL 2015 (pp. 11–15). Toledo, Spain, September 18, 2015: CEUR proceedings.
→ A survey exploring the teachers’ barriers on adopting hybrid MOOCs.
19
Main identified barriers:
● Technological
● Institutional
● Pedagogical

20. Phase 1: problem indentification (II)
[Pub 2] Albó, L., Hernández-leo, D., & Oliver, M. (2016). Are higher education students registering and
participating in MOOCs? The case of MiríadaX. Proceedings of the European MOOC Stakeholder Summit
2016, 1–12.
→ Analysis of the use of MOOCs by the higher education students.
● Age 18-24 (32% of all participants)
● Interested in technology subjects
● Register for 3.56 MOOCs
● Complete 0.55 MOOCs
● Completion rate of 12.87%
20

21. Methodology: design-based research
21

22. Phase 2: Identification of Tentative Products & Design Principes (I)
[Pub 3] Albó, L., Hernández-Leo, D. (2018). Smartphones or laptops in the collaborative classroom? A
study of video-based learning in higher education. Behaviour & Information Technology.
22
Objective
Exploring how the use of different mobile devices
influences the students’ engagement, behaviour
and experience watching academic online videos
Results
● Laptops → better results: student’s engagement with the videos, their collaborative behaviour
and satisfaction with the device.

23. Phase 2: Identification of Tentative Products & Design Principes (II)
[Pub 4] Albó, L., Hernández-Leo, D., Barcelo, J., & Sanabria, L. (2015). Video-Based Learning in Higher
Education: the Flipped or the Hands-on Classroom? In EDEN Annual Conference (pp. 400–408).
Barcelona, Spain.
[Pub 4] Albó, L., Hernández-Leo, D., Barceló, L., & Sanabria, L. (2016). Video-based Learning in Higher
Education: The Flipped or the Hands-On Classroom? Special Issue of the European Journal of Open,
Distance and E-Learning (Best of EDEN 2015), 50–61.
Objective
→ Experimenting new approaches of hybrid courses using video-based learning in a traditional campus.
23

24. Phase 2: Identification of Tentative Products & Design Principes (II)
Case study: Video-Based Learning (VBL) + Project-Based Learning (PBL)
Focus: Study the behaviour and satisfaction of the students using the videos, their utility as well as the position of the
professors.
24

25. Phase 2: Identification of Tentative Products & Design Principes (II)
Video-based learning…
may not only converge in the use of flipped
classroom methodology.
Videos in a hands-on class...
as a support tool that encourages a more
autonomous, flexible and significant learning.
The application of a flipped or a hands-on classroom
approach...
depends on diverse aspects:
1.The nature of the course
2.The behaviour emerging from the students
3.The design of the activities
25

26. Phase 2: Identification of Tentative Products & Design Principes (II)
Results
26

27. Phase 2: Identification of Tentative Products & Design Principes (III)
[Pub 5] Albó, L., & Gelpí, C. (2017). From a FutureLearn MOOC to a blended SPOC : the experience of a
Catalan Sign Language course. In HybridEd Workshop, EMOOCs 2017.
[Pub 6] Albó, L., & Hernández-leo, D. (2017). Breaking the walls of a campus summer course for high
school students with two MOOCs. In HybridEd Workshop, EMOOCs 2017.
→ Participation on two hybrid MOOCs cases
Experiencing the challenges of designing and implementing these type of courses!
27

28. Phase 2: Identification of Tentative Products & Design Principes (IV)
[Pub 7] Albó, L., & Hernández-Leo, D. (2016). Blended learning with MOOCs: towards supporting the
learning design process. In G. Ubachs & L. Konings (Eds.), The Online, Open and Flexible Higher
Education Conference 2016 (pp. 578–588). Rome (Italy): EADTU, October 2016.
Objectives
→ Exploring emerging variations of blended learning
with MOOCs
→ Exploring which design elements are necessary to help
professors during the blended learning design process.
28

29. Phase 2: Identification of Tentative Products & Design Principes (IV)
29

30. Phase 2: Identification of Tentative Products & Design Principes (IV)
[Pub 7] Albó, L., & Hernández-Leo, D. (2016). Blended learning with MOOCs: towards supporting the
learning design process. In G. Ubachs & L. Konings (Eds.), The Online, Open and Flexible Higher
Education Conference 2016 (pp. 578–588). Rome (Italy): EADTU, October 2016.
Results
Providing information regarding the ongoing design can:
● Support the process of learning design → Design Thinking
● Contribute to expand the knowledge of this type
of learning to professors.
30

31. Methodology: design-based research
31

32. Tentative Products & Theories
[Pub 8] Albó, L., Hernández-Leo, D. Conceptualizing a visual representation model for MOOC-based
blended learning designs (submitted to a journal).
32

33. Tentative Products & Theories
33

34. Tentative Products & Theories
34

35. Tentative Products & Theories
35

36. Tentative Products & Theories
36

37. Tentative Products & Theories
37
Visual representation model for blended learning designs

38. Methodology: design-based research
38

39. [Pub 9] Albó, L., & Hernández-Leo, D. (2018). Co-creation process and challenges in the
conceptualization and development of the edCrumble learning design tool. In A. Piotrkowicz, R. Dent-
Spargo, S. Dennerlein, I. Koren, P. Antoniou, P. Bailey, … C. Pahl (Eds.), Joint Proceedings of the CC-TEL
2018 and TACKLE 2018 Workshops. Leeds, United Kingdom: CEUR-WS.
[Pub 10] Albó, L., Hernández-Leo, D. (2019). Developing a web-based visual representation model for
blended learning designs: edCrumble learning design tool (to be submitted to a journal soon).
39
Phase 3: Prototyping of preliminary products and theories

40. 40
Phase 3: Prototyping of preliminary products and theories

41. 41
Phase 3: Prototyping of preliminary products and theories

42. 42

43. 43

44. Methodology: design-based research
44

45. 45
Phase 3: Assessment of preliminary products and theories (IV)

46. Methodology: design-based research
46

47. [Pub 11] Albó, L., & Hernández-leo, D. (2018). Identifying design principles for learning design tools: the
case of edCrumble. In V. Pammer-Schindler, M. Pérez-Sanagustín, H. Drachsler, R. Elferink, & M. Scheffel
(Eds.), Lifelong Technology-Enhanced Learning. EC-TEL 2018. Lecture Notes in Computer Science (Vol.
11082, pp. 406–411). Leeds, UK: Springer, Cham.
[Pub 12] Albó, L., & Hernández-Leo, D. (2018). edCrumble: designing for learning with data analytics. In V.
Pammer-Schindler, M. Pérez-Sanagustín, H. Drachsler, R. Elferink, & M. Scheffel (Eds.), Lifelong
Technology-Enhanced Learning. EC-TEL 2018. Lecture Notes in Computer Science, vol 11082. (pp. 605–
608). Leeds, UK: Springer, Cham.
47
Phase 3: Prototyping of preliminary products and theories (IV)

48. Analytics for learning design
48
Design Analytics
metrics of pedagogical decisions
Support awareness and reflection about design
decisions.
Data includes:
● Learning objectives,
● Type of tasks tasks,
● The social planes (individual, collaborative),
● Resources and tools associated with the task,
● Expected length of time for tasks, …

49. Analytics for learning design
49
Interaction with the community layers
● Agregated meta-data from several designs.

50. Analytics for learning design
50
Understanding community behaviour.
Triggering orientation and inspiration.
Data includes:
● what tools are being used,
● what types of designs are created, tags,
● who are the (co-)authors, edits, views,
● versioning of designs,
● social ratings …
Community Analytics
metrics and patterns of design activity
Michos, K., Hernández-Leo, D., (2018) Supporting awareness in communities of
learning design practice, Computers in Human Behavior, 85, 255-270.

51. Thank you!
Laia Albó
Research stay supported by the SEBAP Research Mobility Award:
Research visit - University of Pittsburgh February 26th, 2019