The document summarizes the work of the Metis project, which aims to create an Integrated Learning Design Environment (ILDE) that meshes existing learning design authoring tools and enables sharing and enactment of learning designs. The methodology involves gathering feedback from stakeholders, iterative development and testing. The initial ILDE prototype prioritizes integration of tools like WebCollage and OpenGLM as well as conceptualization tools. Feedback indicated interest in a variety of design tools, co-creation capabilities, and support for the full design lifecycle. The project refined its terminology and considered extending the main tools.

1. CoCo Seminar, University of Sydney May 13, 2015
Design for Learning:
Communities and Flexible Design Processes

2. ! Public, Young, Urban
! 1st Spanish university in
scientific productivity,
1st Spanish university in THE
12th in the World (under 50 years)
! ~10000 undergraduate,
~4000 graduate students,
~
600 FT academics, ~340 permanent professors
! Associate Professor, ICT Department
! Coordinating the Learning Technologies section (since 2008)
of the Interactive Technologies Group (GTI)
An urban university in the heart of Barcelona
Ciutadella campus:
social sciences and humanities
Mar campus:
health and life sciences
Poblenou campus:
communication and
information technologies

3. Interactive Technologies Group
! Research sections:
— Digital inclusion
— Data visualization and interaction
— Graphics
— Learning Technologies
http://gti.upf.edu/
Former
Current
A human side of technology

4. Teaching Quality and Innovation Unit
UPF Engineering School
http://www.usquidesup.upf.edu/en
Four main axis of work:
• Studies
• Innovation projects
• Specific actions in courses
• Dissemination of studies,
innovations and results
- E.g., Albó, L., Hernández-Leo, D., Barceló, L., Sanabria, L. (2015) Video-Based Learning in Higher Education: The Flipped or the
Hands-On Classroom? EDEN Annual Conference, Barcelona, Spain.

5. technologies
for learning design
computer-supported
collaborative learning
blended
learning

6. PhD Thesis and continuation, at / with University of Valladolid
Support design, computational representation, CSCL, embedded assessment
- Hernández-Leo, D., Asensio-Pérez, J.I., Dimitriadis, Y. & Villasclaras, E.D. (2010) Pattern languages for generating CSCL scripts: from
a conceptual model to the design of a real situation. In: Goodyear P.; Retalis, S. (eds.). E-learning, design patterns and pattern languages,
Sense Publishers, pp. 49-64.
- Hernández-Leo, D., Asensio-Pérez, J.I., Dimitriadis, Y. (2005) Computational Representation of Collaborative Learning Flow Patterns
Using IMS Learning Design. Educational Technology & Society, 8(4): 75-89
- Hernández-Leo, D., Villasclaras-Fernández, E.D., et al (2006) Collage, a Collaborative Learning Design Editor Based on Patterns.
Educational Technology & Society, 9(1), 58-71.
- Villasclaras, E., Hernández-Leo, D. et al. (2013) Web Collage: an implementation of support for assessment design in CSCL macro-
scripts, Computers & Education, 67(September 2013), 79–97.
• Design patterns, design process
• IMS Learning design, authoring tool

7. - Santos, P., Hernández-Leo, D., Blat, J., (2014) To be or not to be in situ outdoors, and other implications for the design and implementation,
in geolocated mobile learning, Pervasive and Mobile Computing, 14(Oct.2014), 17-30.
- Santos, P., Pérez-Sanagustín, M., Hernández-Leo, D., Blat, J. (2011) QuesTInSitu: From Tests To Routes For Assessment In Situ Activities,
Computers & Education, 57(4), 2517-2534.
- Pérez-Sanagustín, M., Santos, P., Hernández-Leo, D., Blat, J. (2012) 4SPPIces: A case study of factors in a scripted collaborative-learning
blended course across spatial locations, International Journal of Computer-Supported Collaborative Learning, 7(3), 443-465.
- Melero, J., Hernández-Leo, D., Sun, J., Santos, P., Blat, J. (2015) How was the activity? A visualization support for a case of location-based
learning design, British Journal of Educational Technology, 46(2), 317-329.
• Location-based learning/assessment
• Game-based learning
• Blended / seamless learning
• Design by teachers
• Interoperability IMS LD, IMS QTI
• GPS, NFC
• Authoring
• Data visualization (self-diagnosis, redesign)
• Effect of roles (carrying the mobile device) within
groups
UPF Location, design for learning, technologies

8. - Hernández-Leo, D., Nieves, R., Arroyo, E. et al. (2012) SOS: Orchestrating Collaborative Activities Across Digital and Physical Spaces
Using Wearable Signaling Devices, Journal of Universal Computer Science, 18(15), 2165-2186.
- Balestrini, M., Hernández-Leo, D., Nieves, R., Blat, J., (2014) Technology-Supported Orchestration Matters: Outperforming Paper-Based
Scripting in a Jigsaw Classroom, IEEE Transactions on Learning Technologies, 7(1), 17-30.
• Signal Orchestration System
• Manager + Fixed and Wearable devices
• Adding digital information to physical spaces:
visual, haptic and auditory ”signals” indicating
group formation, distribution of resources,
assignment of work areas, change of activity…
• Low-cost, ambient orchestration awareness
• Evaluation of diverse prototypes
(dynamism, attention, workload …)
• Seamless learning, across spaces
UPF Technologies for orchestration in physical space

9. Learning design sharing and co-edition
Social network, editors, repository
3
http://ldshake.upf.edu
- Hernández-Leo, D., Lauren, R., Carralero, M.A., Chacón, J., et al. (2011) LdShake: Learning design solutions sharing and co-edition,
Computers & Education, 57(4), pp. 2249-2260.
- Hernández-Leo, D., Moreno, P., Carrió, M., Chacón, J., Blat, J., (2015) LdShake and the "Biologia en Context" teachers' community across
high schools. In: Mor, Y., Craft, B., Maina, M., (eds.). The Art and Science of Learning Design. Sense Publisher, Series “Technology Enhanced
Learning”, pp. 195-210.
!

10. http://ilde.upf.edu/about/
Design for Learning:
Communities and Flexible Design Processes

11. 1 Metis project, vision for an
Integrated Learning Design Environment (ILDE)
2 Methodology
3 ILDE
4 Use of ILDE
5 Conclusions
⌘ Index

12. Learning design / Design for learning
Metis project: the problem
! Supporting teachers in defining the best possible
conditions for students’ to learn
(representations, sharing, interpretation for software systems)
— Set of methodologies, languages, tools…
— But transfer into practice lagging behind expectations
• No integrated solution to support the full lifecycle of
designing and deploying ICT-based learning activities
• Need of adequate professional development
Metis: Meeting teachers co-design needs by means of integrated learning environments

13. ! Mesh available learning design authoring tools,
and to enable the sharing and enactment of
learning designs (Ld), within an Integrated
Learning Design Environment (ILDE)
! Design professional development actions
(workshop packages) to promote learning
design practice, to be used in conjunction with
the ILDE
End-user testbeds from different kinds of
educational contexts (3 user groups: vocational
training, adult education, higher education)
Metis project
Objectives

14. The
METIS
project
www.me2s-­‐project.org
Is2tuto
Tecnologie
Dida=che
3
contexts
HE
Voca2onal
Training
Non-­‐formal
training
Learning
design
exper2se

15. Vision
Teacher
community
system,
co-­‐
create
and
share
learning
designs
Integra2on
of
exis2ng
learning
design
authoring
tools
Automa2c
deployment
of
learning
designs
in
mainstream
VLEs

16. 1 Metis project, vision for an
Integrated Learning Design Environment (ILDE)
2 Methodology
3 ILDE
4 Use of ILDE
5 Conclusions
⌘ Index

17. – Vision
+
feedback
with
stakeholders
• Project
user
groups:
collec2on
of
requirements,
discussions
in
mee2ngs
(context,
scenarios)
• Broader
survey
– Itera2ve
development
in
cycles
• Four
rounds
of
tes2ng
• Two
round
of
pilots
(design
and
enactment
workshops)
– ILDE
About
devoted
page
(made
available
early,
con2nuously
updated,
used
by
third-­‐par2es)
• Demo
server,
open
to
any
interested
party
-­‐
person
• Video
demoing
the
ILDE
• Links
to
source
code
and
documenta2on
• Links
to
ILDE
installa2ons
(also
beyond
the
project
user
groups)
Methodology

18. http://ilde.upf.edu/about

19. Priori2es:
Moodle
as
VLE
and
of
a
set
of
authoring
tools
(WebCollage,
OpenGLM)
but
also
conceptualiza2on
tools
(Course
Map,
Persona
Card)
Working
use
cases:
(1)
Choose
a
learning
design
tool
among
several
available
(2)
Produce
a
learning
design
(3)
Co-­‐produce
a
learning
design
(4)
Share
a
learning
design
(5)
Instan4ate
and
(6)
Deploy
a
learning
design
into
chosen
VLEs
(7)
Provide
feedback
and
reflec2ons
on
learning
designs
(8)
Explore
designs,
implementa2ons
and
feedback
Learnt in the process
- Hernández-Leo, D., Chacón, J., Prieto, J.P., Asensio-Pérez, J.I., Derntl, M., (2013) Towards an Integrated Learning Design Environment.
Proceedings of 8th European Conference on Technology Enhanced Learning, EC-TEL 2013, Paphos, Cyprus, LNCS 8095, pp. 448–453.

20. ! (3) interested in using a variety of learning design tools supporting
different granularities (from modules/courses to activities) and design
phases
! (3) highly interested in being able to (co-)create designs by reusing
existing ones
! (3) recognized as an essential feature allowing diverse actors to
annotate the designs
! (3) support for the whole lifecycle will simplify processes in the institution
! Relevance of co-production depends on the teacher collaboration
culture of the institution/sector (collaboration, sharing and reuse critical
for AE and HE, applicability in VT very limited)
! Exploration of design and comments, especially valuable in AE and HE
- Hernández-Leo, D., Chacón, J., Prieto, J.P., Asensio-Pérez, J.I., Derntl, M., (2013) Towards an Integrated Learning Design Environment.
Proceedings of 8th European Conference on Technology Enhanced Learning, EC-TEL 2013, Paphos, Cyprus, LNCS 8095, pp. 448–453.
First feedback from user groups

21. (Co-)author
Implement
(instantiate +
deploy)
Explore /Investigate /
(Co-)conceptualize
Evaluate
Produce
a
prototype:
a
detailed,
formal
and
reusable
defini2on
of
a
learning
design
Apply
an
authored
learning
design
using
a
specific
VLE,
a
par2cular
group
of
students
and
set
of
tools
Work
on/with
ideas
for
design,
e.g.
crea2on
of
representa2ons
of
design
elements
and
their
interconnec2ons,
understanding
of
the
context
and
expected
users
Learnt in the process
Conceptualize before authoring
Refinement of terminology

22. Learnt in the process
Extension of main tools to consider
- Hernández-Leo, D., Asensio-Pérez, J.I., Derntl, M., Prieto, L.P., Chacón, J., (2014) ILDE: Community Environment for Conceptualizing,
Authoring and Deploying Learning Activities. In: Proceedings of 9th European Conference on Technology Enhanced Learning, EC-TEL 2014,
Graz, Austria, pp. 490-493.

23. Learnt in the process
Refinement of use cases / scenarios
Enable
the
duplica2on
of
LDs
(and
visualiza2on
of
versioning)
to
support
reuse
- Chacón, J., Hernández-Leo, D., Mor, Y., Asensio-Pérez, J. I., (forthcoming)
User-centered design: supporting learning designs’ versioning in a
community platform. In: Kinshuk, Gross, B., Maina, M., (eds.) The architecture
of ubiquitous learning: learning designs for emerging pedagogies, Springer,
Series “Lecture Notes in Educational Technology”.

24. • Objec4ve:
Understanding
needs
beyond
the
project
user
groups
(diverse
educa2onal
level/sectors)
• Total
89
Par2cipants
• Mostly
higher
and
secondary
educa2on
• Research
ques4ons
:
– Are
there
gaps
between
ILDE
scenarios
happening
vs.
desirability?
– What’s
the
poten4al
usefulness
of
the
ILDE
to
ins2tu2ons?
(Average
of
3.63,
no
significant
difference
between
sectors)
– Are
educators
willing
to
try
/
know
more
about
the
ILDE?
(77%
Yes
or
probably)
Learnt in the process
Interest beyond project user groups

25. Learnt in the process
(after evaluation of first round of workshops, 41 respondents)
Extension of requirements
• Integra2on
of
addi2onal
LD
tools,
support
to
addi2onal
versions
of
VLEs
• Enable
“LD
projects”
(compliant
to
LD
workflows
/
pathways
through
a
selec2on
of
tools)
• Contextual
help,
Crea2ve
Commons
licenses,
…
• A
number
of
revisions
in
tooling
(e.g.,
improved
naviga2on,
sync.
edi2on)
• Improved
genera2on
of
data
tracked
for
evalua2on
purposes

26. 1 Metis project, vision for an
Integrated Learning Design Environment (ILDE)
2 Methodology
3 ILDE
4 Use of ILDE
5 Conclusions
⌘ Index

27. Schematic of architecture
G
L
U
E!
External
learning
tools
Design
projects
Versioning
…
(extended)
(extended)
(extended)
• SyncMetaLD

28. A physical architecture (example)
8/12/14

29. 3- (Co-)produce the designs
3.1- Create from scratch
Portal
Registry
LD tools
Repository
LDs
Registry
VLEs
2- Generate a project integrating the required LD tools
1- Plan/select a learning design workflow
3.2- Explore existing designs, replicate and adapt
3.3- Share the designs
5- Access tasks,
resources, tools …
4.4- Run
4.3- Bind tools
4.2- Bind learners
4- Implement
4.1- Select (add) VLE
R.
tools
Tool1 Tool2
Tool3 Tool4
Learning Design (LD) tool providers
Tools for learning
VLEs
A path of use
Flexible
6- Evaluate, reflect…

30. Scenarios
S.1
Before
crea2ng
learning
designs,
educators
analyse
the
contexts
in
which
the
designs
will
be
applied
(audience
of
the
design,
se=ng,
constraints,
pre-­‐requisites,
etc.)
S.2
Educators
conceptualise
macro-­‐designs
of
courses
(think
of
learning
goals,
iden2fy
main
blocks
of
ac2vi2es,
etc…)
13
conceptualiza2on
tools
+
5
tools
for
“open”
conceptualiza2ons
CONCEPTUALIZE,
AUTHOR,
IMPLEMENT

31. Use
cases
/
scenarios
S.3
Educators
author
detailed
designs
of
learning
ac2vi2es,
including
suppor2ng
resources
and
tools
S.4
Educators
duplicate
exis2ng
designs
to
reuse
them
in
their
context
(probably
previous
refinement/adapta2on)
WebCollage,
OpenGLM,
eXeLearning
CONCEPTUALIZE,
AUTHOR,
IMPLEMENT

32.
S.5
Educators
organise
several
conceptualised/authored/
implemented
learning
design
artefacts
related
to
the
same
learning
design
scenario
(e.g.
in
design
projects
or
folders)
-­‐
Define/edit
a
workflow
for
a
project
-­‐
Create
a
project
Use
cases
/
scenarios
S.6
Educators
implement
learning
designs
in
VLEs
Several
versions
of
Moodle,
MediaWiki,
…
CONCEPTUALIZE,
AUTHOR,
IMPLEMENT

33. Use
cases
/
scenarios
S.7
Educators
share
their
learning
designs
so
that
other
educators
in
the
ins2tu2on
can
be
aware
of
them
S.8
Educators
share
their
learning
designs
so
that
other
educators
in
the
ins2tu2on
can
reuse
them
S.9
Educators
share
their
learning
designs
so
that
other
educators
in
the
ins2tu2on
can
par4cipate
in
their
co-­‐crea4on
SHARE
/
CO-­‐CREATION

34. Use
cases
/
scenarios
S.10
Educators
document
reflec4ons
aqer
applying
implemented
designs
with
students
S.11
Educators
provide
forma4ve
comments
to
designs
(created
by
others
or
oneself)
EVALUATE

35. Use
cases
/
scenarios
S.12
Educators
explore
exis2ng
designs
in
the
ins2tu2on
to
be
aware
of
other
educators’
ac2vity
S.13
Educators
explore,
when
available,
exis2ng
duplica4ons
of
a
design
to
see
how
the
design
has
been
reused
and
refined
for
different
contexts
S.14
Educators
explore
exis2ng
implementa4ons
of
designs
with
students
to
be
aware
of
other
educators’
ac2vity
(within
the
ins2tu2on)
S.15
Educators
are
interested
in
exploring
learning
designs
from
other
ins4tu4ons
(see
collabora2on
with
ODS)
EXPLORE

36. 1 Metis project, vision for an
Integrated Learning Design Environment (ILDE)
2 Methodology
3 ILDE
4 Use of ILDE
5 Conclusions
⌘ Index

37. • Several installations
• Communities of different sizes
• With f2f workshops
or completely online
Use of ILDE

38. • Use of multiple pedagogies and tools
o The Open University (UK): CourseMap,
Storyboard, Compendium LD, OpenGLM,
Heursitic Evaluation…
o Agora (ES): Persona Card, Factors and
Concerns, Design Patterns, WebCollage…
o KEK Eurotraining (GR): Design
Narrative,
Persona Card, KEKLessonPlan,
WebCollage…
o MSF (International): combination with
tools already used in the institution
o HandsOn MOOCs: Dream, Persona Card
Flexible (and extensible) design processes
Use of different combination of tools in each community

39. 1st edition
(>300 active participants, >1400 designs, >600 comments)
2nd edition
(>2600 registered, 120 countries, 30 facilitators, 7 languages)
(>390 active participants, >2200 designs, >870 comments)
Use of ILDE in HandsOn MOOC
ILDE pages created by language:
• Bulgarian 629
• Catalan 444
• English 406
• Greek 321
• Spanish 237
• French 153
• Slovenian 23
ILDE pages created by activity:
• Dream: 328
• Persona card: 387
• Factors and concerns: 297
• Objectives: 305
• Heuristics: 321
• Scenario: 302
• Design Studio Report: 287
• WebCollage: 32
- Garreta, M., Hernández-Leo, D., Mor, Y., Sloep, P. Teachers’ perceptions about the HANDSON MOOC: A Learning Design Studio case,
submitted.

40. MOOC 1st ed: (weekly survey, participation varied every week
73 - 148)
How comfortable do you feel with the system?
Use of ILDE in HandsOn MOOC

41. MOOC 2nd ed: (weekly
survey, participation
varied every week
53 - 129)
How comfortable do
you feel with the
system?
Use of ILDE in HandsOn MOOC

42. Evaluation 2nd round of Metis workshops
HE, AE, VT
• ILDE evaluation, based on the TAM model (and subs.)
Perceived ease of use of the system functions
o Perceived usefulness of the system functions
o Social norms.
• Instruments
o A questionnaire – at the end of each workshop
o A follow up bespoke interview - after the enactment phase
o Data tracked by the ILDE - ‘actions’ performed on the platform by
participants
o Data observed by observers
(Pozzi et al., METIS project deliverables)
http://metis-project.org/index.php/download-material

43. Total number of respondents: 107
Results
Perceptions about project function

44. Results
Perceptions about all functions

45. " ILDE is able to support the whole learning design lifecycle from
Conceptualization down to Authoring up to Implementation
" Working with peers and having the possibility to share ideas with
others are highly appreciated features
" LD in ILDE is possible even independently on the pedagogical approach
" The single functions have proven to be stable, pretty ease to be used
and useful
" Need to improve ease of use of Implementation functions
" Support for projects appreciated
" Allow smoother passage among phases, (esp. from Conceptualize to
Authoring)
" There are no significant differences in the acceptance of the ILDE in
the 3 contexts (vocational training, adult education, higher education)
" The evolution and feedback of enactors of especial interest
Results

46. 1 Metis project, vision for an
Integrated Learning Design Environment (ILDE)
2 Methodology
3 ILDE
4 Use of ILDE
5 Conclusions
⌘ Index

47. Conclusions
• Integrated Learning Design Environment (ILDE)
• Community environment for conceptualization, authoring,
implementation of designs for learning
• Well accepted in all its functions
• Fits the needs of different types of communities
• No difference in acceptance
• Flexible design processes
• Training workshops are very important,
focus on the pedagogical aspects
its design can be adapted to context, as ILDE supported design processes

48. ! www.metis-project.org
! ILDE (software and documentation, six languages),
http://ilde.upf.edu/about
! Metis workshop package (customizable meta-design, six languages)
http://ilde.upf.edu/v/ijf
Metis outcomes

49. davinia.hernandez@upf.edu