Semantic Annotation Semantic Communication Semantic Search Semantic Integration Semantic Personalization Semantic Proactivity Semantic Visualization Semantic Games

1. SSeemmaannttiicc WWeebb:: AApppplliiccaattiioonnss
Ch Aatif Hussain
Warraich

2. CCoonntteenntt
1. Semantic Annotation
2. Semantic Communication
3. Semantic Search
4. Semantic Integration
5. Semantic Personalization
6. Semantic Proactivity
7. Semantic Visualization
8. Semantic Games

3. Technology RRooaaddmmaapp ffoorr AApppplliiccaattiioonnss
Semantic
Search
2
P2P Agent Technology Web Services
Semantic Web (SW)
Semantic
Integration
Semantic
Games
Semantic
Proactivity
Semantic
Personalization
Machine Learning
Semantic
Communication
Semantic
Annotation 1
3
4
5
6
7

4. 11.. SSeemmaannttiicc aannnnoottaattiioonn

5. OOnnttoollooggyy--bbaasseedd UUsseerr IInntteerrffaaccee
SSiimmppllee uusseerr ddaattaa oonnttoollooggyy ffoorr mmoobbiillee pphhoonneess
Model of user’s data and other resources:
- Contacts (phone numbers, names etc.)
- Notes (some pieces of text)
- Calendar (with some events assigned)
Auto-generated form for data
Data to store in every instance of
defined information model

6. UUssiinngg ggeenneerraatteedd iinntteerrffaaccee
For described data model
forms are generated
Data view is described as an ontology which contains all needed information about data
structure.
User interface is built dynamically from ontology:
• Fields for data
• Form layout, types of controls (e.g. picture, checkboxes etc.)
• Rules for data that can check some constraints, invoke actions, perform calculations –
whatever!

7. Access yyoouurr ddaattaa qquuiicckkllyy aanndd eeaassiillyy……
Contact data
Event data
Possibilities to build
flexible, easily customizable
data management
applications are great.
select to open
another form
Every piece of data is somehow
described in user’s terms from data-view
ontology.
Links between data make it easy to find
any needed information
Contact data
List of
contacts

8. Browsing tthhee aannnnoottaatteedd ddaattaa

9. Using image mmeettaaddaattaa ffoorr bbrroowwssiinngg
aanndd lliinnkkiinngg ttoo ootthheerr ddaattaa
WWoorrkksshhoopp 1122//0044//22000033
Oleksiy VVaaggaann – IIOOGG Khriyenko
TTeerrzziiyyaann
&& MMeettssoo
FFiinnllaanndd,, JJyyvväässkkyyllää
IInnffoorrmmaattiioonn:: ……
LLiinnkkeedd ttoo:: <<iimmaaggee:: VVaaggaann TTeerrzziiyyaann>>
WWoorrkksshhoopp – IIOOGG && MMeettssoo
1122//0044//22000033
FFiinnllaanndd,, JJyyvväässkkyyllää
IInnffoorrmmaattiioonn:: ……
1122//0044//220033
FFiinnllaanndd,, JJyyvväässkkyyllää
IInnffoorrmmaattiioonn:: ……
PPaarrtt ooff <<iimmaaggee:: WWoorrkksshhoopp –– IIOOGG && MMeettssoo>>
LLiinnkk ttoo <<OOlleekkssiiyy KKhhrriiyyeennkkoo>>
SSeelleecctt iimmaaggeess bbyy::
1122//0044//220033
FFiinnllaanndd,, JJyyvväässkkyyllää
IInnffoorrmmaattiioonn:: ……
PPaarrtt ooff <<iimmaaggee:: WWoorrkksshhoopp –– IIOOGG && MMeettssoo>>
LLiinnkk ttoo <<VVaaggaann TTeerrzziiyyaann>>
<<iimmaaggee:: JJoouunnii PPyyööttssiiää>>
<<iimmaaggee:: OOlleekkssiiyy KKhhrriiyyeennkkoo>>
<<iimmaaggee:: AAnnddrriiyy ZZhhaarrkkoo>>
<<iimmaaggee:: OOlleekkssaannddrr KKoonnoonneennkkoo>>
NNaammee:: VVaaggaann TTeerrzziiyyaann
SSeexx:: MMaallee
DDaattee ooff BBiirrtthh:: 2277 DDeecceemmbbeerr,, 11995588
CCiittiizzeennsshhiipp:: UUkkrraaiinnee
<<OOlleekkssiiyy KKhhrriiyyeennkkoo>>
PPhhoonnee:: ++335588 1144 226600 33001111
EE--mmaaiill:: vvaaggaann@@iitt..jjyyuu..ffii
UURRLL:: wwwwww..ccss..jjyyuu..ffii//aaii//vvaaggaann
……
-- DDaattee
-- LLiinnkk::
-- PPllaaccee ((llooccaattiioonn))
-- ……
……
……

10. Location bbaasseedd iimmaaggee aannnnoottaattiioonn
Storing of the
Historical Dynamics
of the places (areas)
Hotspots
Location based
Information Service
GPS
system
Request for location
Location
area/coordinate
Request for location based
information (via coordinate/area)
Information about area (description)
Spain, the
memorial off
”XXX” London, Thames bank.
Near the ”Big” bridge.
Date: 27/03/2004
Additional Information:
<for personal infill>

11. Location bbaasseedd PPhhoottoo AAllbbuumm--MMaapp
FFiinnllaanndd
JJyyvväässkkyyllää
Finland
Jyväsky
lä
Agora
FFiinnllaanndd
JJyyvväässkkyyllää
AAggoorraa
1133//0088//22000033
IInnffoorrmmaattiioonn:: ……
MMaakkee aa iimmaaggee ttrriipp mmaapp::
-- ddaayy
-- mmoonntthh
-- yyeeaarr
……
……
NNookkiiaa
1133//0088//22000033

12. CCoommppoossiinngg PPhhoottoo AAllbbuummss
uussiinngg mmeettaaddaattaa
USER 1
USER 2
USER N
Web server
“My Friends” “Wedding” “Workgroup” “Our Holidays”

13. BBAANNKK:: DDaattaa aannnnoottaattiioonn
In order to make miscellaneous data gathered and used later for some processing,
every piece of data needs label assigned, which will denote its semantics in terms of
some ontology. Software that is developed with support of that ontology can
recognize the data and process it correctly in respect to its semantics.
Ontology of gathered data
Web forms and dialogs generated
Annotated data (RDF)
Processing of data by some other
semantic-aware applications

14. 22.. SSeemmaannttiicc CCoommmmuunniiccaattiioonn

15. SSeemmaannttiicc CCaallll
Call to a person, who can satisfy
my needs/requirements.
Needs: Buy
Car
what
model BMW 318i
NO Addresses
1995 - …
age
<= 250000
mileage
<= 7500 e
price
my
location
Finland
my
location
SEMA – semantic profile based
matching service
Call to a person, who can
satisfy my needs/requirements.
Needs: Sell
Car
what
model BMW 318i
age 1998
150000
mileage
7000 e
price
Finland
my
location
Needs: Sell
Car
what
model BMW 318i
age 1998
mileage 150000
7000 e
price
Finland
my
location
Needs: Buy
Car
what
model BMW 318i
1995 - …
age
<= 250000
mileage
<= 7500 e
price
Finland
Semantic Match
of the Profiles
High Level of Privacy.
IDs
Phone Numbers
Interests
Profile
JUST Business

16. SSeemmaannttiicc CCoommmmuunniiccaattiioonn ((hhuummaann--ttoo--hhuummaann))
uusseerr
request for
semantic call
Search agent,
provides “semantic match”
functionality
Shared
ontology
Semantic
annotation
users

17. SSeemmaannttiicc CCoommmmuunniiccaattiioonn ((hhuummaann--ttoo--mmaacchhiinnee))
request for
semantic call
Search agent,
provides “semantic match”
functionality
Shared
ontology
Condition
Monitoring
Expert
Semantic
annotation
Field devices

18. SSeemmaannttiicc CCoommmmuunniiccaattiioonn ((mmaacchhiinnee--ttoo--hhuummaann))
request for
semantic call
Search agent,
provides “semantic match”
functionality
Shared
ontology
SSmmaarrtt ddeevviiccee
Semantic
annotation
Fault diagnostics
experts

19. SSeemmaannttiicc CCoommmmuunniiccaattiioonn ((mmaacchhiinnee--ttoo--mmaacchhiinnee))
request for
semantic call
Search agent,
provides “semantic match”
functionality
Shared
ontology
SSmmaarrtt ddeevviiccee
Semantic
annotation
Field devices

20. SSeemmaannttiicc CCaallll
• Examples:
“Connect me with someone who can sell
me cheep (< 500) rowing boat in
Jyväskylä”
“Connect me with a blond girl (21-25) who
wants to meet a guy (26) tonight to go to
dancing club in Jyväskylä”, etc.

21. Clients
Public merchants,
AArrcchhiitteeccttuurree ffoorr aa MMoobbiillee PP--CCoommmmeerrccee SSeerrvviiccee
public customers, public
information providers
…
…
SMOs
SMRs
Maps
<path network>
Maps
<business points>
Integration,
Analysis,
Learning
Business
Ontology
Server
I
C
I
I
S
I
Negotiation,
Contracting,
Billing
Meta-
Profiles
Profiles
RDF
External
Environment
…
Map Content
Providers
Server
Location
Providers
Server
…
Content
Providers
Server
…
RDF
$ $ $ Banks
Terziyan V., Architecture for Mobile P-Commerce: Multilevel Profiling
Framework, IJCAI-2001 International Workshop on "E-Business and
the Intelligent Web", Seattle, USA, 5 August 2001, 12 pp.

22. 33.. SSeemmaannttiicc SSeeaarrcchh

23. SSeemmaannttiicc WWeebb:: SSeemmaannttiicc SSeeaarrcchh
uusseerr
request for
semantic search
Shared
ontology
Semantic
annotation
Web resources /
services / DBs / etc.
Search agent,
provides “semantic match”
functionality

24. SSeemmaannttiicc SSeeaarrcchh
What to search?
data (images, image
fragments, video,
etc.)
persons
places
services
…whatever that can be
annotated and
accessed.
Where to search?
Standalone phone;
Local server;
Peer-to-
Peer;
shared
environment;
global
Web.

25. SSeemmaannttiicc FFaacciilliittaattoorrss ffoorr WWeebb
IInnffoorrmmaattiioonn RReettrriieevvaall ((22000044))
IInnBBCCTT TTeekkeess PPRROOJJEECCTT CChhaapptteerr 33..11..33 ::
““IInndduussttrriiaall OOnnttoollooggiieess aanndd SSeemmaannttiicc WWeebb”” ((yyeeaarr 22000044))
1. Generic Semantic Search Facilitator concept,
architecture and ideas for future utilization of
semantic wrappers for non-semantic search
systems
2. Implementation of Semantic Search Assistant for
Google with semantic interface and domain
ontology.

26. HHooww ddooeess iitt wwoorrkk??
1. Get request
2. Translate request into series of queries
to the used search engines, databases,
data storages… Taking into account
the semantics of searched data
3. Combine returned results, filter non-relevant
(if keyword search was used)
results
4. Return set of best-try results

27. SSeennssee DDeetteerrmmiinnaattiioonn
• WWoorrddNNeett is an open source ontology, which
contains information about different
meanings of a term, synonyms, antonyms
and other lexical and semantic relations
• Having several words in search query we
can determine in which context (sense) each
of them is used with the help of WordNet:
 by comparing words synsets
 by comparing words textual descriptions and
examples
 by finding common roots going up in WordNet
hierarchy tree for each word
 by asking a user

28. SSeemmaannttiicc SSeeaarrcchh AAssssiissttaanntt

29. HHooww ddooeess iitt wwoorrkk??
1. Gets keyword query
2. Translates original query into series of
queries to Google taking into account the
semantics of keywords
3. Combines returned results

30. OOnnttoollooggyy
PPeerrssoonnaalliizzaattiioonn::
iiss mmeecchhaanniissmm,, wwhhiicchh
aalllloowwss uusseerrss ttoo hhaavvee
oowwnn ccoonncceeppttuuaall vviieeww
aanndd bbee aabbllee ttoo uussee iitt ffoorr
sseemmaannttiicc qquueerryyiinngg ooff
sseeaarrcchh ffaacciilliittiieess..
“Driver”
“Driver”
“Driver”
“Driver”
“Driver”
CCoommmmoonn oonnttoollooggyy
SSeeaarrcchh
OOnnttoollooggyy PPeerrssoonnaalliizzaattiioonn

31. WWoorrddNNeett 22..00 SSeeaarrcchh EExxaammppllee
• Search word: "driver“  The noun "driver" has 5 senses in WordNet.
1. driver -- (the operator of a motor vehicle)
2. driver -- (someone who drives animals that pull a vehicle)
3. driver -- (a golfer who hits the golf ball with a driver)
4. driver, device driver -- ((computer science) a program that determines how
a computer will communicate with a peripheral device)
5. driver, number one wood -- (a golf club (a wood) with a near vertical face
that is used for hitting long shots from the tee)
• Sense 1
driver -- (the operator of a motor vehicle)
=> busman, bus driver -- (someone who drives a bus)
=> chauffeur -- (a man paid to drive a privately owned car)
=> designated driver --(the member of a party who
is designated to refrain from alcohol
and so is sober when it is time to drive home)
=> honker -- (a driver who causes his car's horn to make a loud honking
sound;
"the honker was fined for disturbing the peace")
=> motorist, automobilist -- (someone who drives (or travels in) an automo
bile)
=> owner-driver -- (a motorist who owns the car that he/she drives)
=> racer, race driver, automobile driver -- (someone who drives racing car
s at

32. GGeenneerraattiinngg ooff rreeqquueessttss sseett
• WordNet API
and dictionaries
are used for
generating the
set of requests
• When user enters original request, SSA
switches to the panel, where different
senses of typed word are presented

33. SSeemmaannttiicc SSeeaarrcchh EEnnhhaanncceemmeenntt ::
CCoommmmoonn ((lliinngguuiissttiicc))
oonnttoollooggyy
DDoommaaiinn oonnttoollooggyy
QQuueerryy :: XX XX XX XX XX XX (( XX XX XX )) XX
SSeemmaannttiiccFFiilltteerriinngg
RReessuulltt::
EEnnaabblliinngg tthhee SSeemmaannttiicc SSeeaarrcchh
SSeemmaannttiicc SSeeaarrcchh AAssssiissttaanntt
((FFaacciilliittaattoorr)) uusseess oonnttoollooggiiccaallllyy
((WWoorrddNNeett)) ddeeffiinneedd kknnoowwlleeddggee aabboouutt
wwoorrddss aanndd eemmbbeeddddeedd ssuuppppoorrtt ooff
aaddvvaanncceedd GGooooggllee--sseeaarrcchh qquueerryy ffeeaattuurreess
iinn oorrddeerr ttoo ccoonnssttrruucctt mmoorree eeffffiicciieenntt
qquueerriieess ffrroomm ffoorrmmaall tteexxttuuaall ddeessccrriippttiioonn
ooff sseeaarrcchheedd iinnffoorrmmaattiioonn.. SSeemmaannttiicc
SSeeaarrcchh AAssssiissttaanntt hhiiddeess ffrroomm uusseerrss tthhee
ccoommpplleexxiittyy ooff qquueerryy llaanngguuaaggee ooff
ccoonnccrreettee sseeaarrcchh eennggiinnee aanndd ppeerrffoorrmmss
rroouuttiinnee aaccttiioonnss tthhaatt mmoosstt ooff uusseerrss ddoo iinn
oorrddeerr ttoo aacchhiieevvee bbeetttteerr ppeerrffoorrmmaannccee
aanndd ggeett mmoorree rreelleevvaanntt rreessuullttss..

34. EExxaammppllee
• Initial query:
hotel reservation agency
(1, 7 and 5 senses correspondingly)
• From first 5 results only
3 are relevant
(results with whole sequence of query
words even does not appear in first three
pages)
• Generated query:
("hotel") ("booking" OR
"reserve") (-"qualification")
("bureau" OR "agency")
(-"means")
• From first 5 results
all are relevant
(using synonym “booking”
along with “reservation” was
helpful)

35. SSeemmaannttiicc SSeeaarrcchh ooff PPeeooppllee
Searching persons in a P2P
environment
Preferences: blond single
girl, weight:45-65 kg,
height 160-180 sm.
Blond single
girl, weight:50
kg, height 170
sm. match
People gathered for a meeting can browse shared data
of each other
• Every data object/fragment
has associated semantic
annotation, which makes
possible data filtering
• Data sharing in big crowds
can be performed in the
ad-hoc manner
(chain messages).

36. 44.. SSeemmaannttiicc IInntteeggrraattiioonn

37. SSeemmaannttiicc WWeebb:: SSeemmaannttiicc IInntteeggrraattiioonn
Integrated resource
Shared
ontology
Semantic
annotation
Web resources /
services / DBs / etc.

38. IInntteeggrraatteedd ddooccuummeenntt:: SSmmaarrttMMeessssaaggee
Message for John
Invite you and <text: name
of recipient’s wife> to
celebrate house-warming
<image: my house>
today at 20:00. Our
address is <text: my
address> <image: map of my
address>. With the best
regards, Crawford family
<image: my family> <voice:
welcome>
<tag>
Message for XXX
Invite you and < text : your
wife> to celebrate house-warming
< image : my
house> today at 20:00. Our
addres is < text : my
addres> < image : map of
my addres>. With the best
regards, family YYY <
image : our family> < voice
: welcome>
UURRIIss
Resource description via
personal ontology
… “Mood” of the message
?
recipient – “John” wife
sender house
sender address
Roninmäentie 5T 23
today at 20:00. Our address
is Roninmäentie 5T 23
sender address map
sender family
Semantic Search
Message from Michele
<##URI>
Invite you and <text: name
of recipient’s wife> to
celebrate house-warming
<##URI> today at
20:00. Our address is
<##URI> <##URI> . With the
best regards, Crawford
family <##URI> <##URI>
OObbjjeeccttss
recipient wife name – “July”
Sender - Michele
Sender - Michele
Sender - Michele
Sender - Michele
??
house
address
address map
family
??
??
??
??
Semantic Search
SMS
Request for
objects
MMS
(objects)
Message from Michele
Invite you and
July to celebrate
house-warming
today at 20:00.
Our address is
Roninmäentie 5T
23
With the best
regards, Crawford
family
Message from Michele
Invite you and July to
celebrate house-warming
With the best regards,
Crawford family

39. CCoorrppoorraattee//BBuussiinneessss HHuubb
Publish own resource descriptions
Advertise own services
Lookup for resources with semantic search
Hub ontology
and shared domain ontologies
Companies would be able to create
“Corporate Hubs”, which would be an
excellent cooperative business environment
for their applications.
Software and data reuse
Automated access to enterprise (or partners’)
resources
Seamless integration of services
Partners / Businesses
What parties can do:
What parties achieve:
Ontologies will help to glue such Enterprise-wide / Cooperative Semantic Web of shared resources

40. OOnnttoonnuuttss aass aa ttooooll ffoorr sseemmaannttiicc iinntteeggrraattiioonn

41. Ontonuts: Competence Profile ooff aann AAggeenntt aass aa
sseerrvviiccee pprroovviiddeerr ((““wwhhaatt ccaann II ddoo”” aanndd ““wwhhaatt ccaann II
aannsswweerr””)) aanndd aapppprroopprriiaattee sseerrvviiccee ppllaann ((““hhooww II ddoo ……
oorr aannsswweerr ……””))
You
can
ask me
for …
a) … action
b) …
information
ontonut

42. External vviieeww ttoo oonnttoonnuuttss:: SShhaarreedd
CCoommppeetteennccee SSppeecciiffiiccaattiioonn
You
can
ask me
for …
a) I know everything about
Mary
b) I know everything about
cats
c) I know what time it is
now
d) I know all lovers of John
e) I know grades on
chemistry of all pupils
from 4-B
a) I can open the door
#456
b) I can fly
c) I can use knifes
d) I can build house from
wood
e) I can visualize maps
f) I can grant access to
folder “444”
We consider ONTONUTS to be shared S-APL specifications of these
competences
External
Internal

43. Internal view to oonnttoonnuuttss:: AAccttiioonn oorr QQuueerryy
PPllaannss
You
can
ask me
for …
External
Internal
a) I know everything about
Mary
S-APL plan of querying
either own beliefs or
external database about
Mary
a) I can open the door
#456
S-APL plan of opening
the door #456
We consider ONTONUTS to be also an internal plans to execute competences

44. Possible ggeenneerraall rruullee ooff oonnttoonnuutt aappppeeaarraannccee
You
can
ask me
for …
External
Internal
IF I have the plan how to perform certain complex or
simple action or the plan how to answer complex or
simple query
AND {time-to-time execution of the plan is part of my
duty according to my role (commitment) OR I am
often asked by others to execute action or query
according to this plan}
THEN I will create ONTONUT which will make my
competence on this plan explicit and visible to others

45. EExxaammppllee ((11)):: AAttoommiicc OOnnttoonnuutt ##11
I can answer
any queries on
mental diseases
of citizens of X
City X
Central
Hospital
Relational
Database
Give me the list of women from X
with mental diseases diagnosed after
I know how
appropriate
database is
organized, I have
access rights and I
am able to query it
2006

46. EExxaammppllee ((22)):: AAttoommiicc OOnnttoonnuutt ##22
I can answer
any queries on
loans in Nordea
bank
Nordea
XML
Database
Give me the list of Nordea clients with
loans of more than 100 000 EURO
I know how
appropriate
database is
organized, I have
access rights and I
am able to query it

47. EExxaammppllee ((33)):: CCoommpplleexx OOnnttoonnuutt ##33
I can answer any queries on
mental diseases and loans of
Nordea bank clients from X
I know how to split query to two
components; I know to whom I can
send component queries (I have
contracts with them); and I know
how to integrate outcomes of these
queries
Give me the list of Nordea clients from X with
loans of more than 200 000 EURO and who
has more than 2 mental disorders during last
5 years

48. Industrial RReessoouurrccee LLiiffeeccyyccllee aanndd HHiissttoorryy
Condition
Monitoring
States Symptoms
RRDDFF
Measurement
Data
Warehousing
Predictive
Measureme
nt
Fault
detection,
alarms
Diagnostics
Predictive
Monitorin
g
HHiissttoorryy
e Plan
Warehousin
RRDDFF
Predictive
Diagnostics
Maintenance
Fault localization
isolation
Maintenance
Planning
Conditions
Warehousin
g
Industrial
Resource
Predictive
Maintenanc
g
Diagnoses
Warehousin
g
Fault
identification,
Maintenance Plan Diagnoses

49. 55.. SSeemmaannttiicc PPeerrssoonnaalliizzaattiioonn

50. Multimeetmobile Project (2000-2001)
Academy of Finland
Project (1999):
Dynamic Integration of
Classification Algorithms Mobile LLooccaattiioonn--BBaasseedd SSeerrvviiccee iinn
18
Information Technology
Research Institute
(University of Jyvaskyla):
Customer-oriented research and
development in Information Technology
http://www.titu.jyu.fi/eindex.html
Multimeetmobile (MMM) Project
(2000-2001):
Location-Based Service System and Transaction
Management in Mobile Electronic Commerce
http://www.cs.jyu.fi/~mmm
SSeemmaannttiicc WWeebb
19
M-Commerce LBS system
http://www.cs.jyu.fi/~mmm
In the framework of the Multi Meet Mobile
(MMM) project at the University of Jyväskylä,
a LBS pilot system, MMM Location-based
Service system (MLS), has been developed.
MLS is a general LBS system for mobile
users, offering map and navigation across
multiple geographically distributed services
accompanied with access to location-based
information through the map on terminal’s
screen. MLS is based on Java, XML and uses
dynamic selection of services for customers
based on their profile and location.
Virrantaus K., Veijalainen J., Markkula J.,
Katasonov A., Garmash A., Tirri H., Terziyan V.,
Developing GIS-Supported Location-Based
Services, In: Proceedings of WGIS 2001 - First
International Workshop on Web Geographical
Information Systems, 3-6 December, 2001, Kyoto,
Japan, pp. 423-432.
20
Adaptive interface for MLS client
Only predicted services, for the customer with known profile
and location, will be delivered from MLS and displayed at
the mobile terminal screen as clickable “points of interest”
21
Route-based personalization
Inductive learning of customer
preferences with integration of predictors
Sample Instances
< xr1, xr2 ,..., xrm ® yr >
< xt1, xt2,..., xtm >
Learning Environment
Predictors/Classifiers
P1 P2 ... Pn
yt
Terziyan V., Dynamic Integration of Virtual Predictors, In: L.I. Kuncheva, F.
Steimann, C. Haefke, M. Aladjem, V. Novak (Eds), Proceedings of the International ICSC
Congress on Computational Intelligence: Methods and Applications - CIMA'2001, Bangor,
Wales, UK, June 19 - 22, 2001, ICSC Academic Press, Canada/The Netherlands, pp. 463-469.
Static Perspective Dynamic Perspective 22

51. CCoonntteexxttuuaall aanndd PPrreeddiiccttiivvee AAttttrriibbuutteess

Contextual
attributes
< > im -
im
location features
profile features
i i x x x x
1
1 2 , , ......, ,
yi
Mobile customer
description
Ordered service
Predictive
attributes

52. Simple distance bbeettwweeeenn TTwwoo PPrreeffeerreenncceess wwiitthh
HHeetteerrooggeenneeoouuss AAttttrriibbuutteess ((EExxaammppllee))
å
" Î Î
( , ) w ( , )2
D X Y = ×
d x y
i i i
i , x X ,
y Y
i i
where
:
ì
ï ï
í
ï ï
î
x -
y
0, if
î í ì
=
-
=
i i
i
i i
i i
range
x y
i
d x y
else :
1, otherwise
if th attribute is nominal -
( , )
Wine Preference 1:
I prefer white wine served at 15° C
Wine Preference 2:
I prefer red wine served at 25° C
Importance:
Wine color: ω1 = 0.7
Wine temperature: ω2 = 0.3
d (“white”, “red”) = 1
d (15°, 25°) = 10°/((+30°)-(+10°)) = 0.5
D (Wine_preference_1, Wine_preference_2) = √ (0.7• 1 + 0.3 • 0.5) ≈ 0.922

53. Advanced distance between Two
Preferences with Heterogeneous Attributes
(Example) - 1
64
å
( , ) w ( , )2
D X Y = ×
d x y
i i i
i x X y Y
" Î Î
, ,
i i
where
:
ì
ï ï
C
c
( , ) 1
í
ï ï
î
2
P c x P c y i
- -
å=
[ ( | ) ( | )] ,if th attribute is nominal;
x y
i i
- -
=
| | i i
,if i
th attribute is numerical.
range
d x y
i
i i
P(wine|colour = white) =
= 100 / 500 = 0.2
P(wine|colour = red) =
= 200 / 300 = 0.67
Domain objects: 1000 drinks;
300 red, 500 white, 200 - other
Soft drinks: 600;
100 red, 400 white, 100 - other
Wines: 400;
200 red, 100 white, 100 - other
P(soft_drink|colour = white) =
= 400 / 500 = 0.8
P(soft drink|colour = red) =
= 100 / 300 = 0.33

54. Advanced distance between Two
Preferences with Heterogeneous Attributes
(Example) - 2
65
å
( , ) w ( , )2
D X Y = ×
d x y
i i i
i x X y Y
" Î Î
, ,
i i
where
:
P(wine|colour = white)
=
= 100 / 500 = 0.2
P(wine|colour = red)
=
= 200 / 300 = 0.67
ì
ï ï
( , ) 1
í
ï ï
î
2
P c x P c y i
- -
[ ( | ) ( | )] ,if th attribute is nominal;
x y
- -
=
å=
| i i
| ,if i
th attribute is numerical.
range
d x y
i
C
c
i i
i i
P(soft_drink|colour = white)
=
= 400 / 500 = 0.8
P(soft drink|colour = red)
=
= 100 / 300 = 0.33
d (“white”, “red”) = √ [(P(soft_drink|colour = white) - P(soft drink|colour =
red) )2 +
+ (P(wine|colour = white) - P(wine|colour = red) )2 ] =
D ( = √ [(0.8 – 0.33 )2 + (0.2 – 0.67 )2 ] ≈ 0.665 Wine_preference_1, Wine_preference_2) = √ (0.7• 0.665 + 0.3 • 0.5) ≈
0.784

55. PPrreeddiiccttiioonn ooff CCuussttoommeerr’’ss AAccttiioonnss
d1 d2
d3
d4
d5
here I washed my car
here I had nice wine here I had massage
here I had great pizza
here I made hair
I am here now.
There are my recent preferences:
1. I need to wash my car: 0.1
2. I want to drink some wine: 0.2
3. I need a massage: 0.2
4. I want to eat pizza: 0.8
5. I need to make my hair: 0.6
Make a guess what I will order now and where !

56. SSmmaarrtt aassssiissttaanntt
Preferences
(semantic profile)
Advices based on automatically
Assumtion:
”users like what
they photograph”
Somewhere in the other places
Nearby there is a
wounderful old
castle!
collected preferences
(photographing)
Conclusion:
”users likes old
castles”
location-based
annotations
location
awareness

57. SSmmaarrtt aassssiissttaanntt
Advices based on configured
preferences
I like it
…saving to the history…
food ordered through
mobile phone
clothes with scannable ID
any other objects with
accessible semantic profile
location-based annotations
searching matches
Nearby supermarket
(Kauppakatu 7) has
shirts that you like
so much

58. 66.. SSeemmaannttiicc PPrrooaaccttiivviittyy

59. IInntteelllliiggeenntt aannsswweerriinngg mmaacchhiinnee
Sometimes users
cannot answer the
income call
Away for sports
Visiting important
meeting
Making presentation
or lecturing
Studying
A phone was
lost or stolen
Sleeping

60. IInntteelllliiggeenntt aannsswweerriinngg mmaacchhiinnee
A phone was
lost or stolen …however they can
Sometimes users
cannot answer the
income call
Away for sports
Visiting important
meeting
Making presentation or
lecturing
camera phone location data
Studying
wake up! I need
you today!
schedule data
Sleeping
configure intelligent
answering machine
schedule data location data
The user is currently at the swimming pool,
Ontokatu 12. He/she will be there until 12 a. m.
If boss or parents call, wake up.
location data camera phone
Detalization of the reply can be configured
depending on the calling person:
Sorry, buddy, I’m busy now. I’m at the
university, we have a meeting with
colleagues.
friend
We have a meeting at auditorium 2. It
started at 14-00 and will last until 16.35
p. m. Here is its photo.
colleague
wife
I’m at the university, sunny. We (Vagan,
Sasha, Ljosha) have a meeting at the
auditorium 2. Here is a detailed map
and photo of the event.

61. GGUUNN CCoonncceepptt:: AAllll GGUUNN rreessoouurrcceess ““uunnddeerrssttaanndd”” eeaacchh ootthheerr
Real
World
objects
Real World Object +
+ OntoAdapter +
+ OntoShell =
= GGUUNN RReessoouurrccee
OntoAdapters
GGUUNN
OntoShells
Real World objects
of new generation
(OntoAdapter inside)

62. WWeebb SSeerrvviicceess ffoorr SSmmaarrtt DDeevviicceess
Smart industrial devices can be
also Web Service “users”. Their
embedded agents are able to
monitor the state of appropriate
device, to communicate and
exchange data with another
agents. There is a good reason to
launch special Web Services for
such smart industrial devices to
provide necessary online condition
monitoring, diagnostics,
maintenance support, etc.
OntoServ.Net: “Semantic Web Enabled Network of Maintenance
Services for Smart Devices”, Industrial Ontologies Group, March 2003,

63. Global Network ooff MMaaiinntteennaannccee SSeerrvviicceess
OntoServ.Net: “Semantic Web Enabled Network of Maintenance
Services for Smart Devices”, Industrial Ontologies Group, March 2003,

64. SSmmaarrtt MMaaiinntteennaannccee EEnnvviirroonnmmeenntt
““EExxppeerrttss iinn eennvviirroonnmmeennttaall
On-line learning
““SSttaaffff//ssttuuddeennttss
with monitored
organizational data””
““DDeevviicceess wwiitthh
oonn--lliinnee ddaattaa””
““MMaannaaggeerr//EExxppeerrtt””
““EExxppeerrttss””
exchang
Maintenance data
ee
Maintenance data exchange
mmoonniittoorriinngg””
““SSeerrvviicceess””
““Human/patient with
embedded medical
sensors ””
““DDooccttoorr//EExxppeerrtt””
““MMeeddiiccaall WWeebb
““WWeebb SSeerrvviicceess ffoorr eennvviirroonnmmeeSSnnetetaarrllv viicceess””
ddiiaaggnnoossttiiccss aanndd pprreeddiiccttiioonn””
““Environment
with sensors ””
““WWeebb SSeerrvviicceess iinn
oorrggaanniizzaattiioonnaall ddiiaaggnnoossttiiccss aanndd
mmaannaaggeemmeenntt””

65. WWhhaatt iiss UUBBIIWWAARREE ((iinn sshhoorrtt))
• UUBBIIWWAARREE iiss aa ttooooll ttoo ssuuppppoorrtt::
 ddeessiiggnn aanndd iinnssttaallllaattiioonn ooff……,,
 aauuttoonnoommiicc ooppeerraattiioonn ooff…… aanndd
 iinntteerrooppeerraabbiilliittyy aammoonngg……
• …… ccoommpplleexx,, hheetteerrooggeenneeoouuss,, ooppeenn,, ddyynnaammiicc
aanndd sseellff--ccoonnffiigguurraabbllee ddiissttrriibbuutteedd iinndduussttrriiaall
ssyysstteemmss;;……
• …… aanndd ttoo pprroovviiddee ffoolllloowwiinngg sseerrvviicceess ffoorr
ssyysstteemm ccoommppoonneennttss::
 aaddaappttaattiioonn;;
 aauuttoommaattiioonn;;
 cceennttrraalliizzeedd oorr PP22PP oorrggaanniizzaattiioonn;;
 ccoooorrddiinnaattiioonn,, ccoollllaabboorraattiioonn,, iinntteerrooppeerraabbiilliittyy aanndd nneeggoottiiaattiioonn;;
 sseellff--aawwaarreenneessss,, ccoommmmuunniiccaattiioonn aanndd oobbsseerrvvaattiioonn;;
 ddaattaa aanndd pprroocceessss iinntteeggrraattiioonn;;
 ((sseemmaannttiicc)) ddiissccoovveerryy,, sshhaarriinngg aanndd rreeuussee..

66. CCuurrrreenntt UUBBIIWWAARREE AAggeenntt AArrcchhiitteeccttuurree
SS--AAPPLL –
Semantic
Agent
Programming
Language
(RDF-based)
http://users.jyu.fi/~akataso/sapl.ht
ml

67. KKeeyy CCoommppoonneennttss ooff UUBBIIWWAARREE
SScciieennttiiffiicc IImmppaacctt
33.. LLaanngguuaaggee
11.. UUBBIIWWAARREE::
AApppprrooaacchh aanndd
AArrcchhiitteeccttuurree
22.. EEnnggiinnee
BBuussiinneessss PPrroocceessss
CChhoorreeooggrraapphhyy
44.. OOnnttoonnuuttss
EExxtteerrnnaall
CCaappaabbiilliittiieess
OOrrcchheessttrraattiioonn

68. Semantic Integration in UBIWARE 3.0

69. Presentation Case for UBIWARE 3.0

70. X1 :firstName :Vagan
X1 :lastName :Terziyan
X1 :sex :Male
X1 :birthday :27/12/1958
X1 :email
:vagan@it.jyu.fi
X1 :interest :fishing
X1 :hasPhoto #vagan.jpg
X1 :group :IOG
X1 :group :RuleML
… X1 :education :KNURE
X1 :position :professor
X1 :hasFriend X2
X2 :firstName :Alain
X2 :lastName :Gourdin
… X1 :hasFriend X3
X3 :firstName :Mikko
X3 :lastName :Vapa
…
Linked Data

71. UUBBIIWWAARREE AAggeenntt:: PPoossssiibbllee FFuuttuurree AArrcchhiitteeccttuurree
RRBBEE –
Reusable
Behavior
Engine RR
““LLiiffee”” BBeehhaavviioorr
RR
BB
EE
RR
BB
EE
RR
BB
EE
RR
BB
EE
BBeelliieeffss
((ffaaccttss,, rruulleess,, ppoolliicciieess,, ppllaannss))
SShhaarreedd
MMeettaa--BBeelliieeffss
SShhaarreedd
RRBBEEss
EEnnvviirroonnmmeenntt
SSooffttSSoouull
HHaarrddSSoouull
SSooffttMMiinndd
HHaarrddMMiinndd
SSooffttBBooddyy
HHaarrddBBooddyy
RRAABB –
Reusable
Atomic
Behavior
AA
BB
RR
AA
BB
RR
AA
BB
RR
AA
BB
SShhaarreedd
BBeelliieeffss
SShhaarreedd
RRAABBss
MMeettaa--BBeelliieeffss
((pprreeffeerreenncceess))
CCoonnffiigguurraattiioonn ((GGEENNOOMMEE))
SShhaarreedd
HHaarrddwwaarree
“Visible” to
other agents
through
observation
OOnnttoobbiilliittyy is self-contained,
self-described,
semantically
marked-up proactive
agent capability (agent-driven
ontonut), which
can be “seen”,
discovered, exchanged,
composed and
“executed” (internally or
remotely) across the
agent platform in a task-driven
way and which
can perform social
uGGtieelitnny-oobammseeed ibse phaarvti oorf
semantically marked-up
agent configuration
settings, which can
serve as a tool for
agent evolution:
inheritance crossover
and mutation
May be
an
agent

72. 77.. SSeemmaannttiicc VViissuuaalliizzaattiioonn

73. TThhiiss iiss nnoott ssiimmppllee
Cube (ID1)
Ball (ID2)
Table (ID3)
hasColor (ID1, “Green”)
hasColor (ID2, “Red”)
hasColor (ID3, “Brown”)
isOnTheLeftSideOf (ID2, ID1)
hasTemperatureC (ID1, 30)
hasTemperatureC (ID2, 25)
isOn (ID1, ID3)
isOn (ID2, ID3)
isLarger (ID2, ID1)
30°
25°

74. SSeemmaannttiicc MMaasshh--UUpp eennggiinnee
needs also context-based
relevant
features selection
“In the idea of a semantic mash-up, the mash-up program is a model-driven
architecture. This puts the structure of the mash-up under model control,
rather than program control. It is still necessary to translate each information
source into a semantic structure (i.e., RDF), but once that has been done, the
structure of the mash-up is specified by a model, rather than by
program code” [TopQuadrant Inc, June 2007].
http://jazoon.com/jazoon07/en/conference/presentationdetails.html?type=sid&detail=870

75. 44ii ((““ffoorr eeyyee””)) SSeemmaannttiiccaallllyy eennhhaanncceedd ccoonntteexxtt--bbaasseedd mmuullttiiddiimmeennssiioonnaall
RReessoouurrccee VViissuuaalliizzaattiioonn ((OO.. KKhhrriiyyeennkkoo))
The visualization of a “human heart”
resource in a context of its internal
condition can be introduced in a form of
internal structure of the heart and its
functional parts.
Healthcare
Person-location
based
PPeerrssoonn
Organs’
condition
Employer
based
Location of healthcare
organization
Work-place
location
Work
Members, training
facilities (stadium)
Training teams,
football field, …
Occupation,
profession
FFoooottbbaallll tteeaamm
SSttaaddiiuumm
PPeerrssoonn
HHuummaann hheeaarrtt
PPeerrssoonn
Family
relation
Family relation
Internal
Condition
Consisting of
external systems
Work
Occupation,
profession
Treatment of
cardiovascular disease
Medical center
location
“Human heart”
resource in a context of
its condition in relation
to other human body
systems can be
visualized as a part of
an internal structure of
a human body.
The visualization of a
“person” resource in a
context of healthcare &
condition of one’s
organs can be performed
in a way of human body
diagram (with a view of
the organs).
At the same time,
”person” resource in a
context of healthcare &
location of a healthcare
organization can be
visualized in a form of a
map.
The visualization of a “person” resource in a context of
family relations can be displayed in a form of family tree
visualization.
The occupation/profession-based visualization of a
“person” resource. Visualization of a working area
with the relevant work-related links: duties, area of
interests, professional related resources, contacts,
etc.

76. EExxeeccuuttaabbllee RReeaalliittyy:: EExxeeccuuttaabbllee sseemmaannttiicc mmeesshh--uupp

77. EExxeeccuuttaabbllee RReeaalliittyy:: EExxeeccuuttaabbllee sseemmaannttiicc mmeesshh--uupp
115
Terziyan V., Kaykova O., Towards "Executable Reality”: Business
Intelligence on Top of Linked Data, In: Proceedings of the First International
Conference on Business Intelligence and Technology (BUSTECH-2011),
September 25-30, 2011, Rome, Italy, IEEE CS Press, pp. 26-33.

78. EExxeeccuuttaabbllee RReeaalliittyy:: EExxeeccuuttaabbllee sseemmaannttiicc mmeesshh--uupp
University of Jyvaskyla
On-the-fly
generated
statistics
Executable Focus
Contexts
Contexts for BI services
Terziyan V., Kaykova O., Towards "Executable Reality”: Business
Intelligence on Top of Linked Data, In: Proceedings of the First International
Conference on Business Intelligence and Technology (BUSTECH-2011),
September 25-30, 2011, Rome, Italy, IEEE CS Press, pp. 26-33.

79. 88.. SSeemmaannttiiccss--EEnnaabblleedd GGaammeess

80. MMeettaaGGaammee:: sseemmaannttiiccaallllyy aannnnoottaatteedd eeppiissooddeess
Semantical Games Space
Semantic Match
MMeettaaGGaammee
On-line Semantic
Composition of the Games

81. Unified GGaammee PPrrooffiillee ooff aa PPllaayyeerr
• Saved game data (game state, user level, points, etc.) can be shared
between many heterogeneous games via common annotation of data
with game ontologies
• Changing games does not mean to change to become a new player…
RDFS/RDF data storage
Game Profile ontologies

82. PPeerrssoonnaalliizzaattiioonn ooff ggaammeess
Games can be designed in a way that they have standardized
(semantic) descriptions of the customizable elements
(images,text, settings) that can be manually/automatically
changed to the preferred by user. Semantic annotation helps
better finding matches between what can be customized and
what should be customized
Customized images

83. EEdduuccaattiioonn SSuuppppoorrtt GGaammeess
GGoo ttoo tthhee nneexxtt
Exercise
storage
Game
Assistant
Home Exercise
Home Exercise
Home Exercise
Home Exercise
lleevveell
YYoouu sshhoouulldd
mmaakkee ssoommee
eexxeerrcciisseess
55 ++ 2233 == ??
113311 –– 9944 == ??
22 ** 55 == ??
History
Mathematics
Geography
Biology