Dr. Achim Rettinger from Karlsruhe Institute of Technology presented this today as part of the Cognitive Systems Institute Speaker Series on October 13, 2016

1. KIT – Karlsruhe Institute of Technology
INSTITUTE OF APPLIED INFORMATICS ANDFORMAL DESCRIPTION METHODS (AIFB)
www.kit.edu
Towards Multi-Step Expert Advice for Cognitive Computing
Achim Rettinger (rettinger@kit.edu)
Cognitive Systems Institute Speaker Series, October/13/2016

2. Institute of Applied Informatics and
Formal Description Methods
2
My Research Group
Media Channel
Analytics
Healthcare
Analytics
KIT
• Former University
of Karlsruhe,
Germany
• 24.800 students
• 9.500 employees
AIFB
• Research Group
Web Science and
Knowledge
Managment
• Prof. Studer and
Prof. Sure-Vetter
KSRI
• Industry-on-
campus model
• Prof. Satzger

3. Institute of Applied Informatics and
Formal Description Methods
3
Our Research
Cross-Lingual
Technologies
Cross-Modal
Technologies
Language A Language B
DiCaprio
appeare
d in
Titanic
DiCaprio
spielt in
Titanic
(Mogadala et al. 2015)
instances of modalities present in the documents. To reduce the c
we assume a multi-modal document Di = (T ext, Media) to contai
media item either an image, video or audio embedded with a text desc
collection Cj = {D1, D2...Di...Dn} of these documents in different lang
{LC1 , LC2 ...LCj ...LCm } are spread across web. Formally, our research
to find a cross-modal semantically similar document across language
LCo using unsupervised similarity measures on low-dimension correla
representation. Figure 2 shows broad visualization of the approach.
Fig. 2. Correlated Space Retrieval
(Zhang et al. 2014)

4. Institute of Applied Informatics and
Formal Description Methods
4
Our Research
Semantic Search Entity Summarization
Fig. 1. Automatically annotated excerpt of a Wikipedia article9
and the summaClient
knowledge panel with a summary by LinkSUM.
that can be enabled at the top of each page. Other proprietary solutions include
the Bing Knowledge Widget6
and Ontotext’s Now7
. Most of the proprietary
solutions are highly customized and the annotation and knowledge panel parts
are often strongly connected.
4 Summary
With ELES, we propose loose coupling between automatic entity linking and en-
tity summarization systems via ITS 2.0. We exemplify the lightweight integration
approach with the applications DBpedia Spotlight and the qSUM method of the
SUMMA entity summarization interface.
Filter for
Multiple
Entities
Constan
t Stream
(Zhang et at. 2016) (Thalhammer et al. 2016)

5. Institute of Applied Informatics and
Formal Description Methods
5
Our Innovation Projects
LiMexLiMe – crossLingual crossMedia knowledge extraction
http://xlime.eu
Augment with
related content
from news and
social media
Semantic
Search across
content in
channels
Supported by

6. Institute of Applied Informatics and
Formal Description Methods
6
“Watson Seminar” supported by IBM Academic Initiative
Our Teaching
▪ Create a system that identifies
the relationship between two
randomly given characters
Expectations to final solution

7. Institute of Applied Informatics and
Formal Description Methods
7
TOWARDS
MULTI-STEP EXPERT ADVICE
FOR COGNITIVE COMPUTING
Joint work with Patrick Philipp

8. Institute of Applied Informatics and
Formal Description Methods
8
Many tasks comprise multiple steps …
Step 1 Step 2 Step n…

9. Institute of Applied Informatics and
Formal Description Methods
9
Medical Assistance
Brain
Stripping
Brain
Registration
Robust
Brain
Normalization
Normal
Brain
Normalization
Tumor
Segmentation
Map
Generation
Tumor
Prediction
Tumor Progression Mapping
(Philipp et al. 2015)

10. Institute of Applied Informatics and
Formal Description Methods
10
Natural Language Processing
Named Entity
Recognition
Named Entity
Linking
Entity Disambiguation
WebofDocuments
WebofThings

11. Institute of Applied Informatics and
Formal Description Methods
11
Multiple “experts“ might be available …
Step 1 Step 2 Step n…
Expert 1
Expert 2
Expert
m
Expert 1
Expert 2
Expert
m
Expert 1
Expert 2
Expert
m
…
…
…

12. Institute of Applied Informatics and
Formal Description Methods
12
Natural Language Processing
Named Entity
Recognition
Named Entity
Linking
Entity Disambiguation - Example
FOX
Stanford
Tagger
X-LISA
POS Rules
…
AGDISTIS
AIDA
X-LISA
Disambiguator
…

13. Institute of Applied Informatics and
Formal Description Methods
13
Develop robust approaches given various data distributions
NLP: News articles, social media, blogs, …
Medical Assistance: Patients of different departments, scans taken with different
machines by different people
à Many Machine Learning techniques oversimplify as they assume data
to be independent and identically distributed (i.i.d.)
Multiple interpretation steps render brute force approaches
impractical
Number of possible alternatives grow fast over multiple steps
Potential (continuous-) parameters have to be set
Different kinds of additional constraints might be set
Execution / query budgets: Not all experts can be asked
Time budgets: A solution has to be found in a predefined time frame
à Learn behavior of experts with as few training samples as possible
and transfer knowledge among different training datasets
Various Challenges

14. Institute of Applied Informatics and
Formal Description Methods
14
Natural Language Processing
Can be applied to natural language processing tasks
E.g. named entity recognition and –disambiguation pipeline
Hypothesis generation and evaluation
Score outputs of experts
Adapt weight over time
Dynamic learning
Learn weights for each expert given a specific context
Adapt expert choices given a specific context
Incrementally improves with experience
Connection to
IBM Watson‘s Cognitive Computing Capabilities

15. Institute of Applied Informatics and
Formal Description Methods
15
(Budgeted-) Decision Making with Expert Advice (Cesa-Bianchi et al.
1997, Amin et al. 2015)
Adversarial (non i.i.d.) setting with potential budgets
Best expert / subset of experts need to be found
(Contextual-) Bandits (e.g. Auer et al. 2002)
Approaches for adversarial and i.i.d. settings available
Only one action can be played, no feedback for the rest
A high-dimensional context might be given to generalize
(Contextual-) Markov Decision Processes (Puterman 1996,
Krishnamurthy et al. 2016 ) for Reinforcement Learning
Multi-stage contextual bandit with different context spaces
Only intractable solutions with good theoretical performance guarantees exist
Connection to Decision Making Theory

16. Institute of Applied Informatics and
Formal Description Methods
16
Problem Formalization –
Entity Disambiguation Example
!
"!
!
Michael
Jordan
basket
ball
$!
!
$%
!
!
"!
!
$!
%
$%
%
!
"!
! !
"!
!!
"!
!Michael
Jordan
à
NE
basketball
à
NE
Michael
Jordan
à
NE
basket
ball
à
NIL
!
"!
!Michael
Jordan
à
dbpedia:
Michael_J
ordan
basket
ball
à
NIL
+1
Michael
Jordan
à
NE
basket
ball
à
NIL
basket
ball
à
NIL
Michael
Jordan
à
dbpedia:
Michael_J
ordan

17. Institute of Applied Informatics and
Formal Description Methods
17
Probabilistic Soft Logic (PSL)
PSL (Kimmig et al. 2012) is a template language to instantiate a Hinge
Loss Markov Random Field (HL-MRF) (Bach et al. 2012)
0.3: *+,$-. /, 1 ∧ 345$"64+ 1, 7 ≫ 345$"64+(/, 7),
0.8: "<4="$ /, 1 ∧ 345$"64+ 1, 7 ≫ 345$"64+(/, 7)
Given such PSL rules and observations (data), we can infer the unknown
truth values (atoms)
Our Idea: Certain sequences of experts perform better on certain
decision candidates
Introduce a set of PSL rules that describes the dependencies between
experts and decision candidates in a specific state
Collect observations of executions of the pipeline
Probabilistic inference will give you the weights telling you how to
execute experts in each state

18. Institute of Applied Informatics and
Formal Description Methods
18
PSL Rules for Multi-Step Learning
>!
?@!
>%
?@!
>!
?
>%
?
>A
?
!
B!
?@!
%
B!
?@!
!
B!
?

19. Institute of Applied Informatics and
Formal Description Methods
19
PSL Rules for Multi-Step Learning
>!
?@!
>%
?@!
>!
?
>%
?
>A
?
!
B!
?@!
%
B!
?@!
!
B!
?
Hypothesis / Locality / Weight / Value

20. Institute of Applied Informatics and
Formal Description Methods
20
PSL Rules for Multi-Step Learning
>!
?@!
>%
?@!
>!
?
>%
?
>A
?
!
B!
?@!
%
B!
?@!
%
B!
?
Hypothesis / Locality / Weight / Value
C!.!: D4EFG,5H >, B => K$,Lℎ5(>, B)
C1.2: K$,Lℎ5(>, B!) ∧ PH<45ℎ$"," >, B!, B% => QFG=$(B%)

21. Institute of Applied Informatics and
Formal Description Methods
21
PSL Rules for Multi-Step Learning
>!
?@!
>%
?@!
>!
?
>%
?
>A
?
!
B!
?@!
%
B!
?@!
!
B!
?
Independence

22. Institute of Applied Informatics and
Formal Description Methods
22
PSL Rules for Multi-Step Learning
>!
?@!
>%
?@!
>!
?
>%
?
>A
?
!
B!
?@!
%
B!
?@!
!
B!
?
Independence / Combination
C2: R-.$<$-.$-5 >!, >%, B => K$,Lℎ5(>!, B)

23. Institute of Applied Informatics and
Formal Description Methods
23
PSL Rules for Multi-Step Learning
>!
?@!
>%
?@!
>!
?
>%
?
>A
?
!
B!
?@!
%
B!
?@!
!
B!
?
Robustness / Future Reward

24. Institute of Applied Informatics and
Formal Description Methods
24
PSL Rules for Multi-Step Learning
>!
?@!
>%
?@!
>!
?
>%
?
>A
?
!
B!
?@!
%
B!
?@!
!
B!
?
Robustness / Future Reward
C3: S4T="5 >!, >%, B => K$,Lℎ5(>!, B)

25. Institute of Applied Informatics and
Formal Description Methods
25
Task: Named Entity Recognition + Named Entity Disambiguation
(Entity Linking) for tweets and news articles
Scenario 1 (individual steps): Predict the performance on NER and
NED of experts for
Tweets, left out from training set
Articles, trained on tweets only
Scenario 2 (full pipeline): Given a process for collecting samples (e,s)
(i.e. expert performance on tweet or article), select best outcomes to
improve overall performance
Empirical Evaluation

26. Institute of Applied Informatics and
Formal Description Methods
26
1. NER
1. NED
2.
Preliminary Results

27. Institute of Applied Informatics and
Formal Description Methods
27
Heuristic similarity measures such as text length or number of extra
characters yield good results
The relational learning approach (PSL) seems to allow for knowledge
transfer but further evaluations are needed
PSL scales well for thousands of tweets and articles if meta-
dependencies are precomputed
Lessons learnt

28. Institute of Applied Informatics and
Formal Description Methods
28
PSL approach beats State-of-the-Art for heterogeneous textual data
Our approach needs to be embedded into contextual bandit /
reinforcement learning techniques. No exploration / exploitation
strategy implemented so far.
Conclusion & Future Work

29. Institute of Applied Informatics and
Formal Description Methods
29
(Amin et at. 2015)
(Auer et al. 2002)
(Krishnamurthy et al.
2016)
(Puterman 1994)
(Bach et al. 2012)
(Kimmig et al. 2012)
Amin, K., Kale, S., Tesauro, G., and Turaga, D. S. (2015).
Budgeted prediction with expert advice. In AAAI, pages
2490–2496.
Auer, P., Cesa-Bianchi, N., Freund, Y., and Schapire, R. E.
(2002). The nonstochastic multiarmed bandit problem.
SIAM J. Comput., 32(1):48–77.
Krishnamurthy, A., Agarwal, A., and Langford, J. (2016).
Contextual-mdps for pac-reinforcement learning with rich
observations. CoRR, abs/1602.02722.
Puterman, M.L. (1994). Markov Decision Processes: Discrete
Stochastic Dynamic Programming. WileyInterscience, New York.
Bach, S. H., Broecheler, M., Getoor, L., and O’Leary, D. P.
(2012). Scaling MPE inference for constrained continuous
markov random fields with consensus optimization. In
NIPS, pages 2663–2671.
Kimmig, A., Bach, S., Broecheler, M., Huang, B., and
Getoor, L. (2012). A short introduction to probabilistic soft
logic. In NIPS Workshop on Probabilistic Programming:
Foundations and Applications, pages 1–4.
References

30. Institute of Applied Informatics and
Formal Description Methods
30
(Zhang et al. 2016)
(Thalhammer et al. 2016)
(Philipp et al. 2015)
(Mogadala et al. 2015)
(Zhang et al. 2014)
Lei Zhang, Michael Färber, Achim Rettinger; XKnowSearch! Exploiting Knowledge Bases for
Entity-based Cross-lingual Information Retrieval; The 25th ACM International on Conference
on Information and Knowledge Management (CIKM), ACM, Oktober, 2016
Andreas Thalhammer, Nelia Lasierra, Achim Rettinger; LinkSUM: Using Link Analysis to
Summarize Entity Data; In Bozzon, Alessandro and Cudré-Mauroux, Philippe and Pautasso,
Cesare, Web Engineering, 16th International Conference, ICWE 2016, Lugano, Switzerland,
June 6-9, 2016. Proceedings, Seiten: 244-261, Springer International Publishing, Lecture
Notes in Computer Science, 9671, Cham, Juni, 2016
Patrick Philipp, Maria Maleshkova, Darko Katic, Christian Weber, Michael Goetz, Achim
Rettinger, Stefanie Speidel, Benedikt Kämpgen, Marco Nolden, Anna-Laura Wekerle,
Rüdiger Dillmann, Hannes Kenngott, Beat Müller, Rudi Studer; Toward Cognitive Pipelines of
Medical Assistance Algorithms; International Journal of Computer Assisted Radiology and
Surgery, November, 2015
Aditya Mogadala, Achim Rettinger; Multi-Modal Correlated Centroid Space for Multi-Lingual
Cross-Modal Retrieval; In Hanbury, Allan and Kazai, Gabriella and Rauber, Andreas and Fuhr,
Norbert, Advances in Information Retrieval: 37th European Conference on IR Research
(ECIR), Vienna, Austria., Seiten: http://people.aifb.kit.edu/amo/ecir2015/, Springer
International Publishing, Cham, Germany, April, 2015
Lei Zhang, Achim Rettinger; X-LiSA: Cross-lingual Semantic Annotation; Proceedings of the
VLDB Endowment (PVLDB), the 40th International Conference on Very Large Data Bases
(VLDB), 7, (13), Seiten 1693-1696, September, 2014
Own Publications

31. Institute of Applied Informatics and
Formal Description Methods
31
rettinger@kit.edu
http://www.aifb.kit.edu/web/Achim_Rettinger/en
concerning
Research Discussions
Innovation Ideas
about
Expert Processes
Cross-Lingual Technologies
Cross-Modal Technologies
Semantic Search
Entity Summarization
Thank you & feel free to contact me