How higher education learning and teaching can learn from serious game developers. Keynote at the 5th annual SeGAH conference concurrent with WWW 2017 held in Perth, Western Australia

1. Curtin University is a trademark of Curtin University of Technology
CRICOS Provider Code 00301J
SeGAH 2017
Analytic and Strategic
Challenges of Serious Games

2. Greater Curtin @ Perth

3. Learning Futures
 Develops strategic innovations that advance the
mission of the university
 Builds human and technological capacity
 Leads and manages early stage innovation
projects :
formal and informal learning innovations,
pathways & partnerships, UniReady, learning analytics
 Promotes faculty-based research and
continuous improvement using learning
analytics.

4. Curtin Institute for Computation

5. Education Theme:
Build computational
skills across the
university

6. UNESCO Chair of
Data Science in Higher Education
Learning & Teaching

7. Aim of the Chair
To advance global knowledge, practice and policy
concerning the application of data science in the
transformation of higher education learning and teaching
toward improved personalization, access and effectiveness
of education for all.

8. Objectives of the UNESCO Chair
 Data Science
Collaboration
 Professional Learning
 Expand Open Education
 Ethical and Social
Impacts
 Multicultural &
Interdisciplinary
Research
 Open Assessment
Resources

9. Today’s Agenda
 What do Serious Games teach?
 Data challenges of new psychometrics
 Learning analytics
 Educational research with big data
 Methods of educational data science
 What higher education can learn

10. What do Serious Games teach?

11. A Series of Interesting Decisions Wrapped
with Fun and Competition
Immersive affordances
 Research-based learning progressions
 Epistemic challenges and experiences
 Unobtrusive assessment and immediate feedback
Which can teach-train-reinforce
 Embodied Intelligence – Heuristics - Strategic thinking
 Social Interaction Skills – Communication & Collaboration

12. Today’s Agenda
 What do Serious Games teach?
 Challenges of new psychometrics
 Learning analytics
 Educational research with big data
 Methods of educational data science
 What higher education can learn

13. About that data…

14. Why New Psychometrics
There is a need for new frameworks,
concepts and methods for measuring
what someone knows and can do
based on game interactions and
artifacts created during serious play
Why? Ubiquitous, unobtrusive,
interactive big data created by people
working in digital media performance
spaces

15. New Psychometric Landscape
 A “do over” for performance
assessment
 New ways of performing & new
methods of data capture, analysis and
display
 Complex tasks create evidence:
higher order thinking (e.g. decision sequences)
physical performances demonstrating skills
emotional responses

16. New Space for Performance
 Unfold in time
 Cover a multivariate space of possible actions
 Assets contain both intangible (e.g. value, meaning,
sensory qualities, and emotions) and tangible components
(e.g. media, materials, time and space)
NOTE: Asset utilization during performance provides
evidence of what a user knows and can do

17. Performance Space Features
 Unconstrained complex multidimensional stimuli and
responses
 Dynamic adaptation of items to user, which entails
interactivity and dependency
 Nonlinear behaviors with both temporal and spatial
components
NOTE: Higher order and creative thinking is supported in
such a space

18. New Analysis Perspectives
 Hypothetico-deductive methods cannot induce or discover
a new hypothesis or rule to explain particular observations
 Thus the need for data mining, network analysis, and
probability-based methods to augment IRT
 Themes: Data-driven Science & Complexity
GOAL: To characterize and understand high-resolution
multidimensional time-based data

19. Types of Evidence
 Intentional (e.g. constructed responses, whole documents,
short answers, writing, speaking, tool utilization, action
sequences, utilization of help or scaffolds)
 Unintentional (e.g. gestures, utterances, eye movements,
affective states, time taken to respond)

20. Analysis Concepts
 Segmentation of time into events, slices, episodes, activity
segments or n-grams, which are atomistic points for
aggregation
 Segments must be recognizable in relationship to some
structure of meaning – attributes in some frame
 Components of event structure are identified as situations
eliciting action-product learning trajectories made by users.

21. Adaptive Performance Assessments
 Assessments can now be fully embedded and
synonymous with adaptive changes in the digital learning
environment
 There is a thin line between formative and summative,
primarily differentiated by the purposes and audiences of
assessment

22. Analysis Challenges of a Multidimensional
Landscape
 Time (e.g. historical preconditions, longitudinal
data, recurring patterns and autocorrelations)
 Space (e.g. brain use patterns, neighbor
effects, socioeconomic topologies)
 Scale (e.g. neurons to social communities)
 Dynamics (e.g. unique behavioral profiles even
under highly similar conditions)

23. Today’s Agenda
 What do Serious Games teach?
 Data challenges of new psychometrics
 Learning analytics
 Educational research with big data
 Methods of educational data science
 What higher education can learn

24. Learning Theory Framework
for Software Agents
Content
that adapts
to group
and
individual
profiles
Agents for
selecting,
personalizin
g, organizing
& reusing
Agents for
translating,
reformatting,
time shifting,
monitoring,
summarizing
Agents for
critiquing,
“just in time”
feedback &
adaptive
testing
Community
AssessmentLearner
Knowledge

25. Learning analytics
 Educational data mining (EDM) refers to the process of
extracting useful information out of a large collection of
complex educational datasets (Romero, Ventura, Pechenizkiy, &
Baker, 2011)
 Academic analytics (AA) is the identification of meaningful
patterns in educational data in order to inform academic
issues (e.g., retention, success rates) and produce
actionable strategies (e.g., budgeting, human resources)
(Campbell, DeBlois, & Oblinger, 2010)

26. Learning analytics
 Learning analytics use static and dynamic information
about learners and learning environments - assessing,
eliciting and analysing it - for real-time modelling,
prediction, and optimisation of learning processes and
learning environments (Ifenthaler, 2015)

27. Types of Analytics
The different types of analytics can be thought of as a continuum
with increasing value and difficulty. The type of analytics chosen
will be dependent on the business value of the problem.
Value
Difficulty
Descriptive
Analytics
Diagnostic
Analytics
Predictive
Analytics
Prescriptive
Analytics
What happened?
Why did it happen?
What will happen?
What should
I do?

28. Today’s Agenda
 What do Serious Games teach?
 Data challenges of new psychometrics
 Learning analytics
 Educational research with big data
 Methods of educational data science
 What higher education can learn

29. The Vees (there are a few more but these will do)

31. Another view of the process

32. What is Complexity?
 A characteristic of an agent or system
(complexity is not a thing, it exists via relationships)
 Complicated, yes but more…
 Surprising, yes but could be via simple rules…
 Hovers (not sits) between chaos and order

33. Today’s Agenda
 What do Serious Games teach?
 Data challenges of new psychometrics
 Learning analytics
 Educational research with big data
 Methods of educational data science
 What higher education can learn

34. 34
Blackboard
StudentOne
Online Library
Cluster Method of Data Integration for Insight
Condense, classify and map
hypotheses
Surveys
• eValuate
• CASS
• CEQ
• School
Classifications
(constructed)
Conduct analysis &
interactively validate results
Model Construction
(Student Discovery Model)
Analytical Data Set
10 Sources
300GB Data
12 billion data
elements
51,181
Students
1273 Attributes
8 Clusters
Voice of Business, Voice of
Students and Voice of Data
External Data Sets
• Census
• SES indexes
• Geocoding

35. • A unit of one – the student
• Micro-segmentation is appropriate
• Techinique is Kohonen or Self Organising
Map (SOM) through the Viscovery tool
• SOM is a neural network
• 1273 attributes viewable on the map
• for each of the 51,181 Students in scope
• Map built from 274 simultaneously
considered attributes
Attrition StudentsUnderlying statisticsClusters Map International Students
Self-Organizing Map Model
eValuate Sentiment
+ 1270 more attributes
Blackboard Logins

36. Student A Semantic Network
Analysis

37. Student C
Semantic Network
Analysis

38. Semantics
Structure
Surface Matching
Structural
Matching
Concept Matching
Propositional
Matching
Balanced
Semantic Matching
)( ief
 
2
!22
!



n
n
n
)( iv
),,( kji vev
 
)(),((
),,(),,,(
.1.1,
.2.2.2.1.1.1,
mlBA
kjikjiBA
ees
vevvevs



  
vi
iu
uv
V
i V



2
,, 
Gamma Matching
Graphical Matching
  ji
ji
eKSpTd ,
,
(max
(Ifenthaler, 2014; Ifenthaler & Pirnay-Dummer, 2014;
Pirnay-Dummer, Ifenthaler, & Spector, 2010)

39. Today’s Agenda
 What do Serious Games teach?
 Data challenges of new psychometrics
 Learning analytics
 Educational research with big data
 Methods of educational data science
 What higher education can learn

40. Learning for Tomorrow
Technology
Project SummariesCurtin University is a trademark of
Curtin University of Technology
CRICOS Provider Code 00301J
Page 9
The vision is for analytics to provide insight in domains that span the student experience:
Leveraging internal
knowledge and external
datasets together to provide
insight to economic,
population and industry
trends.
Generating an understanding
of students, their behaviours
and experiences to better
target, tailor and engage with
them.
Transformation of teaching
& learning content in
response to changes in
student behaviour, desires
and external factors.
Measurement and rapid
reaction to student
interactions, leading to a
dynamic adoption of best
practice teaching and
assessment techniques.
Leverage of rich university
datasets and analytical
skillsets to promote depth
and breadth of research,
innovation and knowledge
advancement.
Current LearnersCommunity of Future Students Community of Advocates
Market Analytics Curriculum Analytics Teaching Analytics Graduate AnalyticsLearner Analytics
Education Analytics Domains
learning experience; executed at a global scale.
Track and Assist the Learner’s Journey
The university needs to build staff leadership &
capacity in data-driven decision-making across
all of its delivery domains to promote game
based (challenge based) learning.

41. Build a Distributed Learning Analytics
Capacity
(Ifenthaler, 2015)

42. THANK YOU
david.c.gibson@curtin.edu.au
Learning Futures @ Curtin University
Inspiring and supporting
innovation, excellence and impact
in learning and teaching