Ralf Klamma Kolloquium Universität Klagenfurt,  29. Januar, 2010

1. TeLLNet
Virtual Campfire –
A Hero’s Journey
into Multimedia Storytelling
Ralf Klamma
Kolloquium Universität Klagenfurt,
29. Januar, 2010
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke With thanks to Joseph Campbell: The Hero with a Thousand Faces. Novato, California: New World Library, 2008
I5-RK-0909-1

2. Agenda
WALL-E: A Hero's Journey.
TeLLNet
Conclusions and Outlook
UMIC at RWTH Aachen
Virtual Campfire
Storytelling
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke WALL·E – Der Letzte räumt die Erde auf
I5-RK-0909-2

3. RWTH Aachen University
• 260 institutes in 9 faculties as Europe’s
leading institutions for science and research
TeLLNet • Currently around 31,400 students are enrolled
in over 100 academic programs
• Over 5,000 of them are international students
hailing from 120 different countries
• 1,250 spin-off businesses have created
around 30,000 jobs in the greater Aachen
region over the past 20 years.
• IDEA League
• Germany’s Excellence Initiative:
3 clusters of excellence, a graduate school
Lehrstuhl Informatik 5 and the institutional strategy “RWTH
Aachen 2020: Meeting Global Challenges”
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-3

4. Ultra high-speed Mobile Information
and Communication (UMIC)
 Excellence Cluster
TeLLNet
funded by DFG (2007-2011)
 More than 15 institutes from
Electrical Engineering and
Computer Science
 New professorships and
research groups
 Own new building
http://www.umic.rwth-aachen.de
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-4

5. Community Information Systems
Research Group
TeLLNet
 Established at DBIS chair, RWTH Aachen University
 9 Phd students & researchers
 10-15 paid student workers & thesis workers
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-5

6. Storytelling
TeLLNet Ancient concept to
stimulate
knowledge sharing
and learning
How can storytelling
• Different roles
be utilized for
knowledge sharing • Different contexts
in mobile
professional
communities?
• Different
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
semantics
I5-RK-0909-6

7. Why Virtual Campfire?
 Media is witness of Knowledge may get lost, if no
community / individual is telling
TeLLNet
cultural heritage stories any more
Communities of practice are
groups of people who share a
concern or a passion for
something they do and who
interact regularly to learn how to
Reconstruction? do it better (Wenger, 1998)
How?
– Heterogeneous communities
– Media becomes only research
source when the physical object as
Lehrstuhl Informatik 5
(Informationssysteme) primary source gets lost
Prof. Dr. M. Jarke
I5-RK-0909-7

8. Non-linear
Multimedia Storytelling
Creation of non-linear multimedia stories
TeLLNet Definition of story paths compliant to Movement
Oriented Design
(Victoria University)
Linkage of story
elements with
multimedia
contents
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-8

9. UMIC Subproject Virtual Campfire
TeLLNet
Context-aware Semantic Multimedia
Services for Mobile Communities
2.
1.
A bridge
Context
between 5.
awareness of
multimedia 3. 4. Social
mobile
semantics Monitoring of Mobile network
communities
and mobile multimedia analysis of
via location
multimedia communities webservices multimedia
based
context via interaction
multimedia
metadata
services
Lehrstuhl Informatik 5
management
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-9

10. TeLLNet
Context awareness of
mobile communities
via location based multimedia
services
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-10

11. Car Race Scenario
Context-Aware Multimedia Sharing
 Communities produce and consume media at different places, times, etc. via LBS
 Semantic mapping of context data on semantic base types needed
TeLLNet – GPS data -> Tribunes, Curves, etc. (Places)
– Time data -> Lap, Start, Finish, Box stop, etc. (Events)
– Racing car data –> Ferrari, McLaren-Mercedes, BMW-Sauber Community, etc. (Agents)
 Multimedia queries are dependent on context!
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke IJCSSE, May 2010
I5-RK-0909-11

12. Multimedia Semantics and
Multimedia Context
 Context in computer science: Multimedia Multimedia
situational or environmental semantics context
TeLLNet
information [Abowd et al., 1999 ]
Computing
Context awareness Meaning
 context
– For multimedia input
– For multimedia search User
Richness
context
– For multimedia retrieval
 Information richness Physical
[Daft & Lengel, 1986] Precision
context
 Problem: multimedia
semantics and multimedia Machine Time/
readability location
context are not well combined
for mobile multimedia Community
Lehrstuhl Informatik 5
(Informationssysteme) communities Context
Prof. Dr. M. Jarke
I5-RK-0909-12

13. The Multimedia Community
Processing Model
SPARQL
TeLLNet
Multimedia
resources
MPEG-7
Ontology A
(Commsonomy)
Ontology B Context
Dublin Core
OWL
More precise
RDFS Domain multimedia
TV-Anytime Information query results
RDF
EXIF
Community of
...
Practice (CoP)
Lehrstuhl Informatik 5
(Informationssysteme) Yiwei Cao, Ralf Klamma, Dejan Kovachev: Multimedia Processing on Multimedia Semantics and Multimedia Context ,
Prof. Dr. M. Jarke in Proc. SeMuDaTe'09,, Graz, Austria, December, 2009
I5-RK-0909-13

14. TeLLNet
A bridge between multimedia
semantics and multimedia context
via metadata management
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-14

15. Cross-Media Annotation Services
Collaborative indexing based on:
TeLLNet
– Free text annotations (a Agent
lá Flickr) Event
– Semantic MPEG-7 Concept
Basetypes annotations Object
– Image annotations Place
– Community aware Time
annotations: State
folksonomies
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-15

16. Concepts of CA3M
 Community-Aware Static Multimedia
TeLLNet Adaptation based on Clustering
Algorithms [KSCa06] Uncertainty Management
 Context-Aware Adaptation based on Context Reasoner
Semantic Einrichment of Multimedia [CKHJ08, MDM‘08]
Information [CKHJ08]
MPEG-7ToRDF
Context-Aware
Adaptation
 How to support MPEG-7/MPEG-21 System
Media Adaptation
Semantic Enrichment for further System
(OWL)
Semantic Processing? (MPEG-7/21) [CKHJ08, MDM‘08]
 How to systematically handle [KSCa06, ECTEL‘06]
Uncertain Information in further
Lehrstuhl Informatik 5
(Informationssysteme)
Semantic Processing?
Prof. Dr. M. Jarke
I5-RK-0909-16

17. Context Reasoner
Reasoning on context information
Uncertainty Management

Context Reasoner [CKHJ08]
TeLLNet
 SPARQL
MPEG-7ToRDF Context-
– RDF Query Language Aware
Adaptation
– Ontology Reasoning Language Media
Adaptation
System
(OWL)
System
– Implementation using the Jena Framework (MPEG-7/21) [CKHJ08]
[KSCa06]
PREFIX mpeg7: <http://manet.informatik.rwth-aachen.de/~khodaei/mpeg7-v2.owl#>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
SELECT ?agent ?place
WHERE {
?agent mpeg7:name “FirstName LastName"
?agent mpeg7:hasGeographicPosition ?loc1
?loc1 mpeg7:longitude ?lon1
?loc1 mpeg7:latitude ?lat1
?place mpeg7:hasGeographicPosition ?loc2
?place mpeg7:definition “curve-n”
?loc2 mpeg7:longitude ?lon2
?loc2 mpeg7:latitude ?lat2
Lehrstuhl Informatik 5 FILTER Distance(?loc1, ?loc2) <= distance
(Informationssysteme)
Prof. Dr. M. Jarke
}
I5-RK-0909-17

18. Uncertainty Management
 Clustering Algorithms based on User Preferences
– Calculation Euclidean Distance of Communities Uncertainty Management
TeLLNet
– Clusters are recalculated when new users come Context Reasoner [CKHJ08]
 Rating/Ranking as User Query Feedback MPEG-7ToRDF Context-
Aware
Adaptation
Media System
Adaptation (OWL)
System
(MPEG-7/21) [CKHJ08]
[KSCa06]
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-18

19. Virtual Campfire Extentions
An interactive
TeLLNet semantic video
annotation platform
Metadata standard
based video tagging
on various levels
MPEG-7 based
GIS enabled
Granulation from
frames to videos
(Semi-) professional
communities
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-19

20. TeLLNet
Monitoring of mobile communities
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-20

21. How Virtual Campfire Works –
Lightweight Application Server LAS
(Mobile) Context-Aware Services
TeLLNet Interfaces Data Sources
Storytelling MPEG-7 Map Context SNA
Media Service Services Services Services Services
Creation
Databases
Connectors: HTTP, SOAP, XMPP
Media
Tagging
Media Multimedia Processor
Search Metadata
Multimedia Multimedia Multimedia
Semantic Adaptation Annotation Extractor
Browsing
Multimedia
Multimedia
Multimedia Multimedia Multimedia
Repository
Mashups Repository
Player Input Repository
Automatic
Discovery &
Configuration
WWW
Session User Object Data
Manager Manager Manager Access
Community Engine
Invoking services Data flows
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-21

22. Mobile Multimedia Community
Service Testbed
TeLLNet
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
Dominik Renzel, Ralf Klamma :Semantic Monitoring and Analyzing Context-aware Collaborative Multimedia Services
in Proceedings of the 2009 IEEE International Conference on Semantic Computing, 14-16 September 2009 / Berkeley, CA, USA
I5-RK-0909-22

23. youTell: A Web 2.0 Service for
Collaborative Storytelling
TeLLNet
Lehrstuhl Informatik 5
(Informationssysteme) Ralf Klamma, Yiwei Cao, Matthias Jarke: Storytelling on the Web 2.0 as a New Means of Creating Arts
Prof. Dr. M. Jarke in Borko Furht (Ed.): Handbook of Multimedia for Digital Entertainment and Arts, Springer, pp. 623-650, 2009
I5-RK-0909-23

24. Story-tellling Expert Finding
Knowledge Value of Community sorted by keywords
TeLLNet
# Recommendations
Amateur
Expert values Expert Keywords
Lehrstuhl Informatik 5
Measuring Development of Knowledge Value over Time
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-24

25. TeLLNet
Mobile multimedia web services
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-25

26. Mobile Interfaces to
Multimedia Web Services
TeLLNet
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-26

27. Alternative System Architecture
 LAS access with Mobile Web
TeLLNet Services
 LAS access via
Enterprise Service
Bus (ESB)
Lehrstuhl Informatik 5
(Informationssysteme) Yiwei Cao, Matthias Jarke, Ralf Klamma, O. Mendoza, Satish Srirama: Mobile Access to MPEG-7 Based Multimedia Services
Prof. Dr. M. Jarke in Proc. of the Tenth International Conference on Mobile Data Management, May 18-20, 2009, Taipei, Taiwan, pp. 102-111
I5-RK-0909-27

28. Evaluation Results
 Multimedia files are retrieved on search request on mobile devices
TeLLNet
 MPEG-7 based multimedia services are monitored by MobSOS
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-28

29. Lessons learnt for
Mobile Multimedia Webservices
 Design and implementation of alternative architecture
Factors Mobile Web Services ESB as middleware
TeLLNet
Performance Response time is proportional to multi- Response time increases
media file size and increases exponentially linearly at a low rate
Scalability Not scalable, only one LAS server is Scalable, distributed access
connected to LAS services
Development Details must be known by clients WSDL is used to publish and
cost find Web services on the ESB
Reliability Long response time by Web Services Short response time
overload
 Interoperability and scalability are crucial for mobile multimedia
services across various mobile platforms
 Load balancing can reduce wireless data communication cost
and improve mobile multimedia service quality
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-29
 Quality of services can be traced by service monitoring

30. TeLLNet
Social network analysis of
multimedia interaction
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-30

31. Reflective Information Systems as
an Architectural Foundation
Operational Support Community Reflective Support
Self-Modeling
TeLLNet Can we support CoPs with the Can CoPs continuously elicitate and
collaborative creation of complex implement requirements? How much
multimedia objects? computer science support is needed?
Community
Can CoPs make use of metadata over Can CoPs learn meaningful social
the frontiers of media and standards? Self-Observation interaction and make use of
disturbances?
Can we support CoPs by personalized How can CoPs record their complex their
knowledge management and networking complex media learning traces and
strategies in Social Software? how they can deal with them?
How do adaptive, mobile web-based Can CoPs maintain or even improve their
interfaces for CoPs look like? agency (Learning, Researching, Working)
in the Web 2.0?
Actor-
Agent-oriented
Network
RE
Community Theory
Information Systems Social
Participatory Community Game
Network
Design IS Design Theory
Analysis
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-31
Communities of Practice Media Networks

32. Solution idea for Reflective Support:
Cross-Media Social Network Analysis
 Interdisciplinary multidimensional model of digital networks
TeLLNet
– Social network analysis (SNA) is defining measures for social
relations
– Actor network theory (ANT) is connecting human and media agents
– I* framework is defining strategic goals and dependencies
– Theory of media transcriptions is studying cross-media knowledge
social software Media Networks network of artifacts
Wiki, Blog, Podcast, IM, Chat, Microcontent, Blog entry, Message, Burst, Thread,
Email, Newsgroup, Chat … Comment, Conversation, Feedback (Rating)
i*-Dependencies
(Structural, Cross-media)
network of members
Lehrstuhl Informatik 5
Members
(Social Network Analysis: Centrality,
(Informationssysteme)
Prof. Dr. M. Jarke
Efficiency)
Communities of practice
I5-RK-0909-32

33. MediaBase
 Collection of Social Software
artifacts with parameterized
TeLLNet
PERL scripts
– Mailing lists
– Newsletter
– Web sites
– RSS Feeds
– Blogs
 Database support by IBM DB2,
eXist, Oracle, ...
 Web Interface based on
Plone/Zope, LAS, ...
 Strategies of visualization
– Tree maps
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
– Cross-media graphs
I5-RK-0909-33 Klamma et al.: Pattern-Based Cross Media Social Network Analysis for Technology Enhanced Learning in Europe, EC-TEL 2006

34. Evolution of a
Database Research Community
TeLLNet
VLDB 1990 VLDB 1995
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke VLDB 2000 VLDB 2006
I5-RK-0909-34

35. Conclusions & Outlook
Mobile Multimedia Communities
TeLLNet
Community Use and
Facilitation Creation in
International Community
Mobile Server Community Non-linear stories
Standard-conform
MPEG-7 & MPEG-21
Web Services
Multimedia Adaptation
Further protocols
Context Reasoning
P2P data management GPS-enabled Context through
Media production storytelling
Media Structure
Organisation Media
Settings
Mobile Media
Management
Lehrstuhl Informatik 5
(Informationssysteme)
Prof. Dr. M. Jarke
I5-RK-0909-35