This document discusses ensuring the integrity and interoperability of educational usage and social data through a framework to support competency assessment. It proposes SCALA, an extensible web-platform that integrates usage and social data from different learning tools using the IMS Caliper Measurement Framework to provide enriched rubrics for competency assessment. SCALA aims to make competency assessment more objective, scalable, and able to uncover latent skills by analyzing integrated educational and context data.

1. Ensuring tthhee iinntteeggrriittyy aanndd
iinntteerrooppeerraabbiilliittyy ooff eedduuccaattiioonnaall uussaaggee
aanndd ssoocciiaall ddaattaa tthhrroouugghh CCaalliippeerr
ffffrrrraaaammmmeeeewwwwoooorrrrkkkk ttttoooo ssssuuuuppppppppoooorrrrtttt ccccoooommmmppppeeeetttteeeennnnccccyyyy
aasssseessssmmeenntt
A. Rayon, Mariluz Guenaga and A. Núñez

2. CCoonntteexxtt
1 • HHiigghheerr EEdduuccaattiioonn
2
• TTeecchhnnoollooggyy-rriicchh eennvviirroonnmmeennttss
3 • CCoommppeetteennccyy-bbaasseedd lleeaarrnniinngg

3. Assessment ooff ccoommppeetteenncciieess
Subjective
Difficult to scale
Hard to find latent skills
ICT in education
Challenge
Usage and social data
Great promise to improve
learning and assessment
Learning
Analytics
Enriched
Rubrics

4. Challenge: ddaattaa iinntteerrooppeerraabbiilliittyy
Models and
Methods
The scope of LA
interoperability
Data for
Analytical
It may be useful to define
interoperability in levels:
• Domain-specific concepts
•High-level concepts
•Format
Results
Analysis
Q: how can we get
data out of the
operational system
Data takes many forms:
• Attributes and actions
• Resources and context
• Reference data
From: “Learning Analytics Interoperability – The Big Picture in Brief”. Adam Cooper,
Cetis, University of Bolton, UK. March 2014. @laceproject

5. OOuurr pprrooppoossaall:: SSCCAALLAA
upport ompetency ssessment through
approach
Extensible web-platform
Integrates usage and social data to provide enriched
rruubbrriiccss
◦ Uses IMS Caliper Measurement Framework
Oriented to non-expert users: teachers and students

6. Learning
data
Context
data
UDLM:
University of Deusto Learning Model
University
Faculty
Curriculum
Subject
Learning outcomes
Method
Competency
Indicator
Rubric
…
Intelligent
data
SCALA data integration
SCALA
Model
SCALA
Process
SCALA
DB

7. Learning
data
Context
data
University
Faculty
Curriculum
Subject
Learning outcomes
Method
Competency
Indicator
Rubric
…
Intelligent
data
SCALA data integration
SCALA
Model
SCALA
Process

8. University of Deusto’s
University
Faculty
Learning Model
Teacher Student
Learning Event
Action
Activity
Context
Learning
Context
SSCCAALLAA MMooddeell
Curriculum
Teaching
method
Subject Syllabus
Workload
Generic
competencies
Learning
outcomes
Indicator
Assessment
Learning
Activities
Activity
metrics
Foundational
metrics
Engagement Performance
Quiz
Reading
Lecture …
IMS Metric
Profile

9. SSCCAALLAA DDaattaa IInntteeggrraattiioonn
Service bus
◦ Data from the source to SCALA-DB
◦ Extensible data adapters
SSCCAALLAA pprroocceessss
◦ Extract-Transform-Load
◦ We use Kettle, Pentaho’s Data Integration
Module
◦ In parallel for each data adapter

10. EExxppeerriimmeennttaattiioonn
Context
◦ 2013-14
◦ Engineering Faculty
◦ Entrepreneurship subject
◦ Teamwork competency
Learning tools
◦ Moodle, Google Docs, MediaWiki, Youtube,
Twitter, Google Forms
Result: SCALA integrates heterogeneus data
into normalized and exchangeable data
interoperable

11. CCoonncclluussiioonnss
Data interoperability is a key aspect of LA
Involves benefits: efficiency, adaptability,
durability of data, innovation, aggregation,
sshhaarriinngg……
We are ready for exploitation!

12. SSppaanniisshh NNeettwwoorrkk OOff LLeeaarrnniinngg
AAnnaallyyttiiccss
Founded in 2013
Network of researchers and developers
interested on the field
CCrreeaattiinngg ssyynneerrggiieess aanndd pprroommoottiinngg rreellaatteedd
initiatives
New members are welcome!
http://snola.deusto.es
snola@deusto.es
@snolaresearch

13. TThhaannkk yyoouu!!
Mariluz Guenaga
mlguenaga@deusto.es
@mlguenaga

14. IIMMSS CCaalliippeerr vvss TTiinn CCaann AAPPII
Balance between completely
open mechanisms and rigid
schema
Is more encompassing:
◦ Takes a learning environment view
◦ Includes LTI as a important
Widely used
◦ More experience
Only defines the statement
pattern
It does not especify verbs
◦ Advantage: open to a wide range
component
◦ Incorporates existing standards
(LIS, QTI…)
Both are interoperable
◦ Caliper xAPI
◦ xAPI mapping Caliper
of users
◦ Disadv: adopters need to define
these missing parts