Authoring System in TARGET

TUG-KMI

Authoring System in TARGET
www.reachyourtarget.org

¨
Georg Ottl

Knowledge Management Institute
Cognitive Science Section

April 29, 2010

¨
Georg Ottl April 29, 2010 Page 1/37

TUG-KMI

Outline

Research environment

Competence performance assessment

Experts competence structure modeler

Probabilistic graphical models
Factor graphs

¨

TUG-KMI

Outline




Factor graphs

¨

TUG-KMI

Transformative, Adaptive, Responsive and enGaging
EnvironmenT (TARGET)

Serious game based learning
environment
Enterprise Competence
Development
Improve competences in the
project management and
innovation domain

¨

TUG-KMI

Five key concepts of TARGET

¨

TUG-KMI

TARGET Learning Process

¨

TUG-KMI

Role of TUG-KMI in TARGET

TUG-KMI responsible for TARGET learning process
TUG-KMI responsible for workpackage competence
development
Competence performance assessment component
Story adaptation/interventions
Integration competence development/TARGET learning
process

¨

TUG-KMI

Competence performance assessment mockup

¨

TUG-KMI

Outline




Factor graphs

¨

TUG-KMI


Competence
Problems Assessment
Competences

¨

TUG-KMI

TARGET competence performance assessment
Interpret observable performance in game experiences in
regards to a competence state1 .
Include motivational state emotional state in interpretation
Competence assessment as basis for macro and
microadaptive2 interventions and adaptations.
Computational model to automatically assess competence
state.
1
Klaus Korossy. “Modeling Knowledge as Competence and Performance”.
In: Knowledge Spaces: Theories, Empirical Research, Applications. Ed. by
Dietrich Albert and Josef Lukas. Mahwah, NJ: Lawrence Erlbaum Associates,
1999, pp. 103–132.
2
Dietrich Albert et al. “Microadaptivity within Complex Learning Situations
- a Personalized Approach based on Competence Structures and Problem
Spaces”. In: Proceedings of the international Conference on Computers in
Education (ICCE 2007). 2007.
¨

TUG-KMI

TARGET competence performance assessment model
authoring

Interpretation of the game experiences in terms of
competences can be done by a social community through
inspection.
Creation of a model by using the social community
observations input (cold start problem)
Experts create a model to automatically interpret performance

¨

TUG-KMI

TARGET competence performance assessment
requirements

Knowledge model/competence state exchange with HRM
Systems such as SAP.
Assessment in realtime3 to enable microadaptive interventions.

3
O. Conlan et al. Realtime Knowledge Space Skill Assessment for
Personalized Digital Educational Games. IEEE, 2009, pp. 538–542.
¨

TUG-KMI

Basic principle of probabilistic assessment of the
competence state

1. If the learner has the competence ci , than increase the
likelihood of all competence states γci containing ci and
decrease the likelihood of all competence states γ ci .
2. If the learner does not have the competence ci , than decrease
the likelihood of all competence states γci containing ci and
increase the likelihood of all competence states γ ci .

¨

TUG-KMI

Assessment calculation complexity reduction

No structure. Possibly 2n states to be updated on every
performance observation
Deﬁnition of a partial order relation on competences
exploiting the properties of the “PrerequesiteOf” relation type
reduces amount of possible competence states to be taken
into consideration.
Can experts or the community directly create a competence
assessment model??
Authoring tools can help to create a model for competence
assessment.

¨

TUG-KMI

Mathematical and computational model for competence
assessment
Nondeterministic assessment4
Traditional, multiplicative update rule56
Belief propagation networks such as Bayesian Networks7
4
C. Hockemeyer. “A Comparison of non-deterministic procedures for the
adaptive assessment of knowledge”. In: Psychologische Beitrage 44.4 (2002),
pp. 495–503.
5
Jean-Claude Falmagne and Jean-Paul Doignon. “A class of stochastic
procedures for the assessment of knowledge”. In: British Journal of
Mathematical and Statistical Psychology 41 (1988), pp. 1–23.
6
Jean-Claude Falmagne and Jean-Paul Doignon. “A markovian procedure
for assessing the state of a system”. In: Journal of Mathematical Psychology
32.3 (1988), pp. 232–258.
7
M. Villano. “Probabilistic Student Models: Bayesian Belief Networks and
Knowledge Space Theory”. In: Proceedings of the Second International
Conference on Intelligent Tutoring Systems. Springer, 1992, 491–498.
¨

TUG-KMI

Outline




Factor graphs

¨

TUG-KMI

Current state Competence Modeler

Support to create competence assessment models
Using the well studied PrerequesiteOf relation8
Support experts (psychologists) to create knowledge
structures.

8
Dietrich Albert et al. Knowledge Structures. Ed. by Dietrich Albert. New
York: Springer Verlag, 1994.
¨

TUG-KMI

Hasse diagram visualization

Figure: Popular knowledge space visualizations

¨

TUG-KMI

Problems solved and Related Problems

Computer supported Hasse diagramm creation
Visualizations done with the Java Universal Network/Graph
Framework (JUNG) framework9

9
J. Madadhain et al. “Analysis and visualization of network data using
JUNG”. In: Journal of Statistical Software 10 (2005), pp. 1–35.
¨

TUG-KMI

Hasse diagram visualization

→
Figure: Version 0.13 and 0.16

¨

TUG-KMI

¨

TUG-KMI

Problems solved and Related Problems

Creation of a Hasse diagram reduced to the problem of
calculating the minimal transitive reduction of a graph which
was shown to have the same complexity as calculation of the
transitive closure of a graph10 .
Eﬀective calculation and detection of cycles by maintaining
the online topological order of the graph11
Visualizations done with the JUNG12
10
A. V. Aho, M. R. Garey, and J. D. Ullman. “The Transitive Reduction of a
Directed Graph”. In: SIAM Journal on Computing 1.2 (1972), pp. 131–137.
11
David J. Pearce and Paul H. J. Kelly. “A dynamic topological sort
algorithm for directed acyclic graphs”. In: J. Exp. Algorithmics 11 (2006),
p. 1.7.
12
J. Madadhain et al. “Analysis and visualization of network data using
JUNG”. In: Journal of Statistical Software 10 (2005), pp. 1–35.
¨

TUG-KMI

Current State Competence Modeler
On-Line Demo Afternoon

https://dev-css.tu-graz.ac.at/

¨

TUG-KMI

Outline




Factor graphs

¨

TUG-KMI

Probabilistic Graphical Models
Whatfor?

Simple way to visualize the structure of a probabilistic model
Graphical representation allows insights into the properties of
the model
Insights into conditional independence properties
Complex computations can be expressed in terms of graphical
representations; use of graph based inference algorithms that
exploit graph properties for calculation.

¨

TUG-KMI

Graph Terminology

A graph comprises vertices
V = (a, b, c, d) connected
by edges

Definition
A graph G is a pair G = (V , E ), where V is a (finite) set of
vertices and E ⊆ V × V is a (finite) set of edges.

¨

TUG-KMI

Definition
A graph G is called undirected iff

∀A, B ∈ V : (A, B) ∈ E ⇒ (B, A) ∈ E (1)

Two ordered pairs (A, B) and (B, A) are identified and represented
by only one undirected edge.

Definition
A graph G is called directed iff

∀A, B ∈ V : (A, B) ∈ E ⇒ (B, A) ∈ E (2)

An edge (A, B) considered to be a directed edge from A towards
B

¨

TUG-KMI

Visual Representation Graph Models

Figure: Directed Graph
Figure: Undirected Graph

¨

TUG-KMI

Probabilistic Graphical Models (1/2)
Whatfor?

In a probabilistic graph model every vertice represents a
random variable
The edges express probabilistic relationships between the
variables
Directed Graphical probabilistic Models
Bayesian Networks
Undirected Graphical Probabilistic Models
Markov Random Fields
Loose coupling between statistical variables.

¨

TUG-KMI

Graphical probabilistic models

Question: Can directed probabilistic models such as as
Bayesian networks be used for assessment. How does believe
propagation relate to the classical update rule?
Question: How is the relation between directed and
undirected probabilistic graphical models?
Use of directed and undirected graphical probabilistic models
to assess the players state
Eﬃcient sum and dot product calculation.

¨

TUG-KMI

A ﬂexible probabilistic graphical model, the Factor Graph

Factor Graphs13 as a single representation for directed and
undirected graphical probabilistic models
Factor Graphs were successfully applied for Bayesian Networks
and Markovian Models
Multiple applications in artiﬁcial intelligence and signal
processing based on Factor Graphs

13
Frank Kschischang et al. “Factor Graphs and the Sum-Product Algorithm”.
In: IEEE Transactions on Information Theory 47 (2001), pp. 498–519.
¨

TUG-KMI

Factor graph conversion (1/3)

Example 1:(A simple probabilistic graph) S1 S2
Let f (S1, S2, S3) be a function of three
variables, and suppose that f can be
expressed as a product
f (S1, S2, S3) = p(S1)p(S2)p(S3|S1, S2) S3

¨

TUG-KMI


S1 S2

Example 1:(A factor graph)
Let f (S1, S2, S3) be a function of three
variables, and suppose that f can be f
expressed as a product
f (S1, S2, S3) = p(S1)p(S2)p(S3|S1, S2)

S3

¨

TUG-KMI


Eﬃcient algorithms available to calculate probabilities
(Sum-Product algorithm)
Makes extensive use of “conditional independent” properties
Parallelization possible
Approximative algorithms
Eﬃcient marginalization

¨

TUG-KMI

Thank You!

¨

TUG-KMI

[1] A. V. Aho, M. R. Garey, and J. D. Ullman. “The Transitive
Reduction of a Directed Graph”. In: SIAM Journal on
Computing 1.2 (1972), pp. 131–137.
[2] Dietrich Albert et al. Knowledge Structures. Ed. by
Dietrich Albert. New York: Springer Verlag, 1994.
[3] Dietrich Albert et al. “Microadaptivity within Complex
Learning Situations - a Personalized Approach based on
Competence Structures and Problem Spaces”. In:
Proceedings of the international Conference on Computers in
Education (ICCE 2007). 2007.
[4] O. Conlan et al. Realtime Knowledge Space Skill Assessment
for Personalized Digital Educational Games. IEEE, 2009,
pp. 538–542.
[5] Jean-Claude Falmagne and Jean-Paul Doignon. “A class of
stochastic procedures for the assessment of knowledge”. In:
¨

TUG-KMI

British Journal of Mathematical and Statistical Psychology
41 (1988), pp. 1–23.
[6] Jean-Claude Falmagne and Jean-Paul Doignon. “A
markovian procedure for assessing the state of a system”. In:
Journal of Mathematical Psychology 32.3 (1988),
pp. 232–258.
[7] C. Hockemeyer. “A Comparison of non-deterministic
procedures for the adaptive assessment of knowledge”. In:
Psychologische Beitrage 44.4 (2002), pp. 495–503.
[8] Klaus Korossy. “Modeling Knowledge as Competence and
Performance”. In: Knowledge Spaces: Theories, Empirical
Research, Applications. Ed. by Dietrich Albert and
Josef Lukas. Mahwah, NJ: Lawrence Erlbaum Associates,
1999, pp. 103–132.

¨

TUG-KMI

[9] Frank Kschischang et al. “Factor Graphs and the
Sum-Product Algorithm”. In: IEEE Transactions on
Information Theory 47 (2001), pp. 498–519.

[10] J. Madadhain et al. “Analysis and visualization of network
data using JUNG”. In: Journal of Statistical Software 10
(2005), pp. 1–35.

[11] David J. Pearce and Paul H. J. Kelly. “A dynamic
topological sort algorithm for directed acyclic graphs”. In: J.
Exp. Algorithmics 11 (2006), p. 1.7.

[12] M. Villano. “Probabilistic Student Models: Bayesian Belief
Networks and Knowledge Space Theory”. In: Proceedings of
the Second International Conference on Intelligent Tutoring
Systems. Springer, 1992, 491–498.
¨

TUG-KMI

Acronyms

JUNG Java Universal Network/Graph Framework

¨

Authoring System in TARGET

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a Authoring System in TARGET

Semelhante a Authoring System in TARGET (20)

Authoring System in TARGET