Architecture from IMPRESS Erasmus+ European Project to integrate different gamified approaches into a single integrated dashboard

1. From heterogeneous
activities to unified analytics
dashboards
Iván Martínez-Ortiz, Iván Pérez-Colado, Dan Cristian Rotaru,
Manuel Freire, Baltasar Fernández-Manjón

2. Teachers and gamified activities in the class
● Teachers: increasing number of gamified activities that they can use.
● A typical analytics dashboard feeds from a single system
○ complex to integrate different independent activities into a single dashboard
● Multiple activities integrated together?
○ Instead of switching dashboards: teachers expect both:
■ class overview dashboards that integrate all class activities
■ per-student dashboards with all activities by a single student
○ Activities are often grouped together in a hierarchy -> aggregation.
● Tools not originally designed with analytics? -> offline activities

3. Improve students engagement in software engineering
Integrating four types of activities into unified dashboards:
● Generic results from offline activities (in this case, Kahoot! [3]) (Offline
activity), reported as CSV files with a simple structure.
● Interactive quizzes (via ARSNOVA.click) (NodeJS), where students
compete for quick and correct answers.
● A formal-specification game (FormalZ) (Java), where students build
specifications that correctly reject invalid inputs and, for valid inputs, correctly
describe their outputs.
● A unit-test writing game (Code Defenders) (Java), where students
compete to build tests that detect all bugs, and introduce bugs not detected
by current tests.
Code Defenders FormalZ

4. Game Learning Analytics
Import CSV
Tracker
TrackerTracker Code Defenders
FormalZ
Offline Activity
Realtime Long Term

5. DESIGN AND PROOF OF CONCEPT
● Reuse Open-source pre-existing analytics platform -> EU H2020 RAGE
○ xAPI standard -> trackers
○ Two credentials:
■ The user who plays
■ The activity being performed
● Different issues:
○ Unifying User-IDs
○ Managing Activity Trees
○ Offline Activities
○ Updating Activity Trees
○ Informing Activities Context of Use
○ Estimating Implementation Effort
Session of gameplay

6. Unifying User-IDs in IMPRESS
● Common user-IDs -> Traces belonging to the same player
● Several of the participating institutions do not have institutional LMSs
● Use of single-sign-on (SSO) highly important -> Simple SAML.
Code
Defenders
FormalZ
SSO

7. IMPRESS: Managing Activity Trees
● IMPRESS require to structure different game activities into a larger activity
● When building dashboards:
○ Add and remove activities.
○ Weights of sub-activities.
● Activity Rollup: Second iteration from Beaconing H2020 Project
1. Storage: Formula and parameters to be used during rollup + actual tree
structure.
2. Evaluation: Child activities completion -> Progress and success tracking
using formula
3. Display of results
● Activity manager -> dynamically modify the activity tree.

8. Managing Activity Trees
Config
Result
parent.variable1 = (Child1.variable1 * 0.5) + (Child2.variable1 * 2)

9. Integrating Offline Activities
● Purpose: generate similar traces to tracker-enabled activities.
● Upload the activity results (CSV files) -> public and well-documented
API.
○ Each line corresponds to an xAPI-SG trace.

10. Activity Trees
Diagram of the rollup process, where root activities update their variables
based on the values present in their child activities.

11. Updating Activity Trees
● If the activity tree + variables are modified -> Inconsistent results
● “Reload all traces” button -> Discards all results + regenerate using new
config
Results
Raw
tracesAnalytics Framework
AnalysisQueue
Traces
Clears
Reload all traces

12. Updating Activity Trees: How rollup works?
● All traces have a target activity -> Unique Activity Identifiers
● Same type activity to be used multiple times
● Traces are re-enqueued changing the activity target with parent identifier
● Traces are rolled-up until the root node
● Weighted variables change values in a child activity
↪ synthetic rollup trace is added to the queue

13. Updating Activity Trees: How rollup works?

14. Informing Activities of their Context of Use
● Different versions of an activity -> different parts of a given activity tree
○ Same quiz developing tool -> multiple quizzes
● Example: Two URLs from the same game, different activity:
○ https://quiz.server/q?activityid=1234&quiz=unit-test
○ https://quiz.server/q?activityid=5678&quiz=formal-spec
● Require additional support from activity-server side
○ Parse the activity-ID -> include it in all analytics traces generated.
Quiz Server
URL Parser formal-spec Traces
Analytics Fram
AnQueue
Activity
5678

15. CONCLUSIONS
● Addressed integration of multiple heterogeneous activities into a
unified analytics dashboard to improve a software engineering
course.
○ Feasible but still complex
● We consider it to be easily generalizable to many other cases and
integrations
○ Integrate tracker components or uploading of their activity logs
○ Configure the server to indicate
■ Who participates
■ What the activity hierarchy looks like
■ How it should be presented and aggregated for analytics
purposes

16. FUTURE WORK
● To validate the activity manager and the overall integration of the
component activities
○ performing several experiments with actual students from the
participating institutions
● Improve and simplify the interface to simplify operation and increase
understanding
● Deal with different non technical issues in different educational
settings (e.g. GDPR)
● Partial simplified integration
○ Full integration with Formal Z
○ Improving dashboard provided by Code Defender

17. Baltasar Fernandez-Manjon
@baltaFM
balta@fdi.ucm.es
Code: https://github.com/e-ucm
https://www.slideshare.net/BaltasarFernandezManjon/
www.e-ucm.es
Trank you!