Presentation at HIBIT 2019

1. Section for Artificial Intelligence and Decision Support, Medical University of Vienna
Assoc.Prof. Dr. Matthias Samwald
1
Towards Transformative Artificial Intelligence
in Life Science and Health Care
Assoc.Prof. Mag. Dr. Matthias Samwald
Section for Artificial Intelligence and Decision Support
Medical University of Vienna

2. Assoc.Prof. Dr. Matthias Samwald
Section for Artificial Intelligence and Decision Support, Medical University of Vienna
Medical University of Vienna
• One of the largest hospitals in Europe
• 25 university clinics
• 8000 students
2

3. Assoc.Prof. Dr. Matthias Samwald
Section for Artificial Intelligence and Decision Support, Medical University of Vienna
Section for Artificial Intelligence
and Decision Support
• History dating back to 1977
• Multidisciplinary work on AI, medicine,
biology, cognitive science
3

4. „From data to knowledge to action“
Section for Artificial Intelligence and Decision Support, Medical University of Vienna
Assoc.Prof. Dr. Matthias Samwald
4
Diagnosing diseases based on lab values or images
Natural language processing
• Identifying key statements and their semantic relations in biomedical texts
• Deep learning for biomedical text summarization
• Creating high-precision biomedical search engines
• Re-purposing deep NLP models for biological sequences (proteins)
Predicting novel links in large biomedical knowledge bases
Decision support systems for enabling personalized medicine
Strategic considerations for harnessing AI for global progress
Raw data
Knowledge
Action

5. „From data to knowledge to action“
Section for Artificial Intelligence and Decision Support, Medical University of Vienna
Assoc.Prof. Dr. Matthias Samwald
5
Diagnosing diseases based on lab values or images
Natural language processing
• Identifying key statements and their semantic relations in biomedical texts
• Deep learning for biomedical text summarization
• Creating high-precision biomedical search engines
• Re-purposing deep NLP models for biological sequences (proteins)
Predicting novel links in large biomedical knowledge bases
Decision support systems for enabling personalized medicine
Strategic considerations for harnessing AI for global progress
Raw data
Knowledge
Action

6. Predicting novel links in large biomedical
knowledge bases
Section for Artificial Intelligence and Decision Support
Matthias Samwald
6

7. We are using neural models for predicting yet unknown
links in biomedical knowledge bases

8. We are using neural models for predicting yet unknown
links in biomedical knowledge bases
Knowledge graph embeddings
• low dimensional
• vector space
• certain properties can be represented
by notions of distance
similar entities are close together
relationships based on geometric regularities

9. We are using neural models for predicting yet unknown
links in biomedical knowledge bases

10. Gene
Drug
Disease
Anatomy
Phenotype
GO Term
Pathway
Relation
Multiple Relation Types
Negative Relation
Ontology Relation
We are using neural models for predicting yet unknown
links in biomedical knowledge bases
SIDER
OpenBioLink

11. Decision support systems for enabling
personalized medicine
Section for Artificial Intelligence and Decision Support
Matthias Samwald
11

12. The promise of pharmacogenomics (PGx)
 Genetic polymorphisms in small set of
genes impact pharmacokinetics of a wide
variety of medications
 Hope: Genetic patient data could be used
to make pharmacotherapy safer and more
effective
 Reality: PGx is still rarely used in clinical
routine
Section for Artificial Intelligence and Decision Support
Matthias Samwald
12

13. Section for Artificial Intelligence and Decision Support
Matthias Samwald
13
The frequency of actionable pharmacogenes. Over 95% of the population
carry at least one actionable genotype for one of the genes covered by
the DPWG guidelines.

14. We found that patients often receive multiple PGx drugs
over time
Section for Artificial Intelligence and Decision Support
Matthias Samwald
14
Age 40 – 64
(n = 26 561 525)
Age ≥ 65
(n = 5 429 266)
Prescribed one or more PGx
drugs
42,2% 50,6%
Prescribed three or more
PGx drugs
7,5% 13,8%
Expected co-occurence of
drug with high-risk genotype
1,9% 4,1%
Matthias Samwald, Hong Xu, Kathrin Blagec, Philip E. Empey, Daniel C. Malone, Seid Mussa Ahmed, Patrick Ryan, Sebastian Hofer and Richard D. Boyce. “Incidence of
Exposure of Patients in the United States to Multiple Drugs for Which Pharmacogenomic Guidelines Are Available.” PLOS ONE 11, no. 10 (October 20, 2016):
e0164972
Wolfgang Kuch, Christoph Rinner, Walter Gall and Matthias Samwald. “How Many Patients Could Benefit From Pre-Emptive Pharmacogenomic Testing and Decision
Support? A Retrospective Study Based on Nationwide Austrian Claims Data.” Studies in Health Technology and Informatics 223 (2016): 253–58.
Patients in the United States with novel prescriptions of one or more PGx drugs within a
four-year time window

15. How can we create IT-based workflows for facilitating
clinical PGx?
Section for Artificial Intelligence and Decision Support
Matthias Samwald
15
 Genetic patient data needs to be available easily and quickly
 Ideally, a single test would yield data on all important PGx genes
and data could be re-used in all care processes, without requiring
multiple tests (pre-emptive PGx).
 Clinical decision support algorithms based on up-to-date
guidelines to guide medical practitioners
 Ideal IT solutions
need to be available
across a wide
variety of care settings

16. We developed a barrier-free system for PGx data access
and decision support
Section for Artificial Intelligence and Decision Support
Matthias Samwald
16
 Created open-source inference engine for PGx decision support based on raw
genetic test results
 Developed a decentralized, mobile-based system for making PGx data and decision
support available across healthcare settings
 „Medication Safety Code“ system
Matthias Samwald, Klaus-Peter Adlassnig. „Pharmacogenomics in the pocket of every patient? A prototype based on Quick Response (QR) codes“ Journal of the American Medical Informatics Association 20, no. 3
(January 5, 2013): 409–412.
Jose Antonio Miñarro-Giménez, Kathrin Blagec, Richard D. Boyce, Klaus-Peter Adlassnig and Matthias Samwald. „An Ontology-Based, Mobile-Optimized System for Pharmacogenomic Decision Support at the Point-of-
Care.” PLoS ONE 9, no. 5 (May 2, 2014): e93769. doi:10.1371/journal.pone.0093769.

17. We developed a barrier-free, decentralised system for PGx
data access and decision support
Section for Artificial Intelligence and Decision Support
Matthias Samwald
17
The Medication Safety Code system

18. We developed a barrier-free, decentralised system for PGx
data access and decision support
Section for Artificial Intelligence and Decision Support
Matthias Samwald
18

19. We developed a barrier-free, decentralised system for PGx
data access and decision support
Section for Artificial Intelligence and Decision Support
Matthias Samwald
19

20. We conducted a mixed-methods study on system design
Section for Artificial Intelligence and Decision Support
Matthias Samwald
20
• Mixed methods study with PGx experts, clinicians and pharmacists (n
= 114)
• System proved capable of enabling PGx guided decision making
of physicians and pharmacists in fictional patient scenarios
• Majority of participants agreed (57,6) or strongly agreed (9,8%) that
user experience was appealing
• Feedback was used to make further improvements to the system
• System adopted in other research projects (e.g., at Mayo Clinic, US)
Kathrin Blagec, Katrina M. Romagnoli, Richard D. Boyce and Matthias Samwald. “Examining Perceptions of the Usefulness and Usability of a Mobile-Based System for
Pharmacogenomics Clinical Decision Support: A Mixed Methods Study.” PeerJ 4 (February 8, 2016): e1671

21. The Ubiquitous Pharmacogenomics (U-PGx) H2020
project
Section for Artificial Intelligence and Decision Support
Matthias Samwald
21
Implement and evaluate IT-enabled,
pre-emptive PGx in clinical care
(Implementation study)
Five years duration
€ 15 million budget
10 EU countries
“Multicenter, multi-healthcare
system, multigene, multidrug, multi-
ethnic, multilingual”
Kathrin Blagec, Rudolf Koopmann, Mandy Crommentuijn – van Rhenen, Inge Holsappel, Cathelijne H van der Wouden, Lidija Konta, Hong Xu, Daniela Steinberger, Enrico Just, Jesse J Swen, Henk-Jan Guchelaar
and Matthias Samwald „Implementing Pharmacogenomics Decision Support across Seven European Countries: The Ubiquitous Pharmacogenomics (U-PGx) Project“. Journal of the American Medical
Informatics Association, Feb 9 2018.

22. The Ubiquitous Pharmacogenomics (U-PGx) H2020
project
Section for Artificial Intelligence and Decision Support
Matthias Samwald
22
Panel of 13 genes
43 drugs of interest
Both inpatients and outpatients
Across all medical specialties
Kathrin Blagec, Rudolf Koopmann, Mandy Crommentuijn – van Rhenen, Inge Holsappel, Cathelijne H van der Wouden, Lidija Konta, Hong Xu, Daniela Steinberger, Enrico Just, Jesse J Swen, Henk-Jan Guchelaar
and Matthias Samwald „Implementing Pharmacogenomics Decision Support across Seven European Countries: The Ubiquitous Pharmacogenomics (U-PGx) Project“. Journal of the American Medical
Informatics Association, Feb 9 2018.

23. Section for Artificial Intelligence and Decision Support
Matthias Samwald
23
PGx gene Affected drugs
CYP1A2 Clozapine
CYP2B6 Efavirenz
CYP2C19 Warfarin, Citalopram, Escitalopram, Sertraline, Imipramine, Voriconazole
CYP2C9 Phenytoin
CYP2D6 Flecainide, Propafenon, Codeine, Oxycodone, Tramadol, Tamoxifen, Paroxetine, Venlafaxine,
Amitriptyline, Clomipramine, Doxepin, Imipramine, Notriptyline, Metoprolol, Aripiprazole,
Haloperidol, Pimozide, Zuclopenthixol, Atomoxetine
CYP3A5 Tacrolimus
DPD Capecitabine, Fluorouracil, Tegafur
HLA B*5701 Flucloxacillin
SLCO1B1 Atorvastatin, Simvastatin
TPMT Azathioprine, Mercaptopurine, Thioguanine
UGT1A1 Irinotecan
VKORC1 Acenocoumarol, Phenprocoumon, Warfarin
FVL Estrogen containing drugs
Adapted from the guidelines of the Dutch Pharmacogenetics Working group (DPWG). CYP, cytochrome P450; DPD, dihydropyrimidinedehydrogenase;
FVL, factor five Leiden HLA, human leucocyte antigen; SLCO, solute carrier organic anion transporter; TPMT, thiopurine S-methyltransferase; UGT,
UDP-glucuronosyltransferase; VKORC, vitamin K oxide reductase complex.

24. DPWG classification of clinical effect
Section for Artificial Intelligence and Decision Support
Matthias Samwald
24

25. We found health informatics capabilities to differ
strongly between institutions and countries
Section for Artificial Intelligence and Decision Support
Matthias Samwald
25
NL GB IT ES AT SI GR
EHR
inpatient
setting
Yes Yes Partially Yes Partially Partially No
EHR
outpatient
setting
Yes Partially Partially Yes No No No
Active CDS
Yes (for
PGx, DDIs,
contraind.)
Yes (for
allergies
and DDI)
No Partially No No No
Passive
CDS
No Yes Yes Yes Yes No No
Structured
laboratory
results
Yes Yes Yes Yes Yes No No

26. The decision support solutions meet the needs of
clinicians
Section for Artificial Intelligence and Decision Support
Matthias Samwald
26
Strongly
disagree
Disagree Neutral Agree Strongly agree
The CDS tools integrates
well with my work routine
1 2 11 31 6
I feel that I have received
enough training to
confidently use the different
CDS tools.
0 4 13 30 4
Very Dissatisfied Dissatisfied Neutral Satisfied Very Satisfied
Overall satisfaction 1 0 9 34 7
To little Just right Too much
Do the CDS tools provide too much,
too little or just the right amount of
information?
3 39 9

27. Randomization
U-PGx
Standard
Standard
U-PGx
01-01-2017
01-06-2018
01-01-2020
Approx.
8000
patients
in total
Analysis of clinical outcome
Clinical trial measuring outcome still underway

28. The CDSS systems developed in U-PGx are also
being made available outside of the project
Section for Artificial Intelligence and Decision Support
Matthias Samwald
28

29. How can we utilize
Artificial Intelligence
for transformative progress?
Section for Artificial Intelligence and Decision Support, Medical University of Vienna
Assoc.Prof. Dr. Matthias Samwald
29

30. The truth about ‚deep learning‘ is that it makes lots of
people sad
Section for Artificial Intelligence and Decision Support, Medical University of Vienna
Assoc.Prof. Dr. Matthias Samwald
30
In late 2018, Google DeepMind
had it‘s debut at the CASP13
protein structure prediction
competition and won against
established players

31. The truth about ‚deep learning‘ is that it makes lots of
people sad
Section for Artificial Intelligence and Decision Support, Medical University of Vienna
Assoc.Prof. Dr. Matthias Samwald
31

32. AI progress is best steered and measured by public
benchmarks. Progress has been significant in last years.
Section for Artificial Intelligence and Decision Support, Medical University of Vienna
Assoc.Prof. Dr. Matthias Samwald
32

33. Section for Artificial Intelligence and Decision Support, Medical University of Vienna
Assoc.Prof. Dr. Matthias Samwald
33

34. Section for Artificial Intelligence and Decision Support, Medical University of Vienna
Assoc.Prof. Dr. Matthias Samwald
34

35. We observe increasing model generality and algorithmic
progress becoming a public good
Section for Artificial Intelligence and Decision Support, Medical University of Vienna
Assoc.Prof. Dr. Matthias Samwald
35
A small set of models and algorithms seems to be applicable to a wide variety of data
modalities
• Sequence (Text? SMILES string? Processes? DNA? Protein?)
• Graph (Social network? Citation network? Molecular structure? Molecular interaction
network? Ontologies and knowledge bases?)
• 2D/3D data array (Photo? Microscopic image? MRI?)

36. We observe increasing model generality and algorithmic
progress becoming a public good
Section for Artificial Intelligence and Decision Support, Medical University of Vienna
Assoc.Prof. Dr. Matthias Samwald
36
Global progress bottlenecked by availability of good (semi-)public train and test sets and
solid benchmarks
Rather than facing ever-increasing model complexity, we keep finding model structures
that are simpler yet more powerful and more generally applicable (even thought they
might be scaled up a lot)

37. AI is both an enabling ‘General Purpose Technology’
& an ‘Invention of a Method of Invention’
Section for Artificial Intelligence and Decision Support, Medical University of Vienna
Assoc.Prof. Dr. Matthias Samwald
37
NO YES
NO
YES
‚Enabling General Purpose Technology‘?
„Invention of
a Method
of Invention“?
Autonomous vehicles
Combustion engine
Artificial Intelligence
Any specific drug
MRI machine
CRISPR
WWW

38. Unsupervised and multi-task pre-training generates models that
have a general understanding of the phenomena in the world
Section for Artificial Intelligence and Decision Support, Medical University of Vienna
Assoc.Prof. Dr. Matthias Samwald
38
• Unsupervised pre-training = ‚Predict everything from everything else‘
• Multi-task training = ‚Train a single model to do many different things‘
• Very recent example: BERT (2018) and BioBERT (2019) architectures achieving state of
the art across many NLP tasks

39. Unsupervised and multi-task pre-training generates models that
have a general understanding of the phenomena in the world
Section for Artificial Intelligence and Decision Support, Medical University of Vienna
Assoc.Prof. Dr. Matthias Samwald
39
Type 2 ________ ________ is characterized by insulin __________
Type 2 diabetes mellitus is characterized by insulin resistance
Fill in the blanks!

40. Unsupervised and multi-task pre-training generates models that
have a general understanding of the phenomena in the world
Section for Artificial Intelligence and Decision Support, Medical University of Vienna
Assoc.Prof. Dr. Matthias Samwald
40
• Train/test algorithms not only on raw data of single modality, but data aggregated across
different modalities
• ‚Multi-omics‘
• Increase spatial and temporal resolution
• Better Integrate raw data / machine learning and higher-level knowledge / knowledge-based
systems

41. Strategic analysis of meta-layers of progress:
Incremental vs. radical improvements
Section for Artificial Intelligence and Decision Support, Medical University of Vienna
Assoc.Prof. Dr. Matthias Samwald
41
What should be prioritized to see most progress 20 years from now?
Today ??? 2040

42. Section for Artificial Intelligence and Decision Support
Matthias Samwald
42
We should not think only
about AI, but about
hybrid intelligent
systems

43. Section for Artificial Intelligence and Decision Support
Matthias Samwald
43
R&D processes,
organisations,
societies
Human
Knowledge &
Decision Making
Artificial
Intelligence

44. Section for Artificial Intelligence and Decision Support, Medical University of Vienna
Assoc.Prof. Dr. Matthias Samwald
44
 AI to aid doctor’s decision making
 AI for creating better drugs for known targets
 AI for fundamental, basic biological research
 AI for developing completely novel research tools and apparatuses
 AI for general human intelligence augmentation (high-precision information
retrieval, analysis, visualization, sharing insights)
 Autonomous AI (closing the intervention-inference loop w/o humans)
 … AI for AI (recursive self-improvement) …

45. Section for Artificial Intelligence and Decision Support
Matthias Samwald
45
Need to find strategies to apply
AI with
* Greatest leverage
* Greatest versatility
* Lowest risk

46. The AI Strategies project (work in progress)
Section for Artificial Intelligence and Decision Support
Matthias Samwald
46

47. Thanks!
This work has received funding from the European Union’s Horizon 2020 research and Innovation programme under grant
agreement No 668353 and the Austrian Science Fund (FWF; P 25608-N15)
Section for Artificial Intelligence and Decision Support
Matthias Samwald
Project members:
Kathrin Blagec
Hong Xu
Anna Breit
http://samwald.info/