Augmented personalized healthcare uses IoT sensors, AI, and big data to provide personalized and continuous healthcare centered around the patient. This involves collecting 360-degree multimodal data on personal, public, and population levels to provide actionable insights. Key challenges include ensuring sensor data quality and reliability, handling data heterogeneity, and developing contextual interpretations and personalized actions. The presenter's Kno.e.sis research initiative applies these concepts to asthma management using environmental, medical, and lifestyle data, and to bariatric surgery monitoring to improve compliance and outcomes.
OSCamp Kubernetes 2024 | SRE Challenges in Monolith to Microservices Shift at...
Augmented Personalized Healthcare with IoT and AI
1. Augmented Personalized Healthcare
How Smart Data with IoTs and AI is about to change
Healthcare
Invited Talk @ Big Data Integration and IoT for Smart Health Care,
3rd Intl Forum on Research and Technologies for Society and Industry
Modena Italy, 13 September 2017
Prof. Amit Sheth
LexisNexis Ohio Eminent Scholar; Executive Director, Kno.e.sis
Wright State University
Background: http://bit.ly/k-APH, http://bit.ly/kAsthma, http://j.mp/PARCtalk,
2. 2
• Traditional Healthcare
• Healthcare: Then and Now
• Augmented Personalized Health for health care
of the future, and associated technical
challenges
• kHealth: Three ongoing applications
Outline
8. 8
Healthcare: Then and Now
Limited data 360 degree multimodal
● Personal-Public-Population
● Physical-Cyber-Social big data
driven
9. The Patient of the
Future
MIT Technology Review,
2012
http://www.technologyreview.com/featuredstory/426968/the-patient-of-the-future/9
Patients are increasingly taking control of own health
Patient Generated Health Data (PGHD) are “health-related data created, recorded, or gathered by or from
patients (or family members or other caregivers) to help address a health concern. PGHD include, but are not
limited to health history, treatment history, biometric data, symptoms, and lifestyle choices.”
Office of the National Coordinator for Health Information Technology (ONC).
11. AIM: Doctor in a self-driving Car
[Image: courtesy Artefac, https://flipboard.com/@flipboard/-who-needs-a-hospital-when-this-self-dri/f-4204314e11%2Ffastcodesign.com t]
12. 12
Converting big data into smart data through contextual and
personalized processing such that patient and clinician can
make better decisions and take timely actions for Augmented
Personalized Health
Future Health Care
13. 13
Augmented Personalized Healthcare (APH) is expected
to enhance healthcare by personalizing the use of all
relevant
Physical, Cyber, and Social data obtained from wearables,
sensors and Internet of Things, mobile applications,
electronic medical records, web-based information,
and social media for better health for an individual.
Data include traditional clinical data, PGHD and public health data, as well as
environmental and social data that could impact an individual’s health.
Augmented Personalized Healthcare
15. 15
Providing actionable information in a
timely manner is crucial to avoid
information overload or fatigue
Sleep data
Community
dataPersonal
Schedule Activity data
Personal
health
records
Data Overload for Patients/health aficionados
16. 16
Challenges in deriving actionable insights from Sensor Data
According to Forbes, the wearables market exceeded $2 billion in
2015, 3 billion in 2016 and will be over 4 billion in 2017.
Leading to vast volume of healthcare data: some key issues
● Sensor reliability and Quality
● Sensor data Heterogeneity
● Contextual Interpretation and Abstraction
● Personalized Health and Health Objective
https://www.linkedin.com/pulse/digital-health-why-doctors-should-care-doug-hart
17. 17
Sensor Data Reliability and Quality
1.https://wq.io/media/images/quality-large.png, 2https://readwrite.com/2017/05/26/study-wearables-counting-calories-dl4/,
3.https://www.wsj.com/articles/smartphones-open-a-new-world-for-medical-researchers-1498442821
● Consumer graded devices can be terrible and inaccurate [2]
● Is data generated from them useful for health application?
● Research shows they can be effective for health applications [3]
19. 19
Sensor Data Heterogeneity
Questionnaire Data
Electronic Medical
Records
Gene
Lab Test Wearables and Sensors
To enable the proper interpretation of data and for determining remediation measures,
it is essential to convert the data into abstractions ignoring inessential differences and
providing an integrated view for the clinician to take action
21. Personalized Health and Objectives: one size does not fit all
Millions of people - > one treatment
Wearable and Sensor data
22. 2222
ACTIONS
situation awareness useful
for decision making
Converting data to actions
Hyperthyroidism
Elevated
Blood
Pressure
Systolic blood pressure of 150
mmHg
“15
0”
..
.
..
.
ABSTRACTIONS
make sense to humans
KNOWLEDGE
for interpretation of
observations
Contextualization
Personalization
DATA
Observations from
machine and social
sensors
Kno.e.sis’ kHealth initiative follows this
approach -- currently for asthma in children,
bariatric surgery, ...
23. 23
• Asthma Management in Children
• Bariatrics Surgery for Obese Adults
• Pain Management
kHealth Personalized Digital Health Initiative
25. 25
kHealth: Health Signal Processing Architecture
Personal
level Signals
Public level
Signals
Population
level Signals
Domain
Knowledge
Risk Model
Events from Social
streams
Take Medication before
going to work
Avoid going out in the
evening due to high pollen
levels
Contact doctor
Analysis
Personalized
Actionable
Information
Data Acquisition
& aggregation
26. 26
26Asthma Domain Knowledge
Domain
Knowledge
ICS= inhaled corticosteroid, LABA = inhaled long-acting beta2-agonist, SABA= inhaled short-acting beta2-
agonist ;
*consider referral to specialist
Asthma Control and Actionable Information
Asthma Control
Daily Medication
choices for
stating therapy Not Well Controlled
Poor Controlled
Severity Level of
Asthma
Intermittent Asthma
Mild Persistent Asthma
Moderate Persistent Asthma
Severe Persistent Asthma
Recommended Action Recommended Action Recommended Action
SABA prn
Low dose ICS
Medium ICS alone or
with LABA/Montelukast
High dose
LABA/Montelukast
Medium ICS
Medium ICS +
LABA/ Montelukast or
High Dose ICS
Need specialist care
Medium ICS
Medium ICS +
LABA/ Montelukast or
High Dose ICS
Need specialist care
27. 27
Sensordrone – for monitoring
environmental air quality
Wheezometer – for
monitoring
wheezing sounds
Can I reduce my asthma
attacks at night?
What are the triggers?
What is the wheezing level?
What is the propensity
toward asthma?
What is the exposure level
over a day?
Commute to work
Decision Support for Doctors and Patients: A Scenario
Luminosity
CO
level
CO in gush
during day
time Actionable
Information
Personal level
Signals
Public level
Signals
Population
level Signals
What is the air quality indoors?
Close the window at home
during day to avoid CO2
inflow, to avoid asthma
attacks at night
28. 28
k-Health Dashboard: A Platform to Visually Analyse to find
Correlations (e.g., Patient Symptoms and Personalized Data)
Multimodal Data Streams & Anonymised Patient Data Visualized for Correlation Analysis
interpreted in with the help of knowledge graph (relevant medical knowledge)
36. kHealth Bariatrics
The purpose of our research is to determine if monitoring Bariatric patient’s pre- and postoperative
compliance with active and passive sensors can bolster bariatric patient’s progress and lessen
weight recidivism
37. ● 500 million people all over the world are obesity
● 36% of the adults in the United States suffer from obesity
● 65% of the world’s population lives in countries where the occurrence of death
due to overweight and obesity is higher than being underweight
Obesity
38. ● Chances of regaining weight as stomach can still expand after surgery
● Continuous monitoring of the patients by the surgeon is very essential
Bariatric Surgery
39. Challenges Post-Bariatric Surgery
● Patient acceptance and active participation involving continuous
monitoring of the patient
● Cost and reimbursement models
● Challenging research in understanding of variety of data over long period
40. A system that can
● monitor the patient continuously and remotely
● identify non-compliance before and after surgery
● nudge/assist for better compliance for improved outcomes and reduce
recidivism
Post-Bariatric Surgery Solution
41. kHealth Post-Bariatric Surgery Proposed Method
Aggregate the data collected from the sensors, questionnaires and use artificial
intelligence techniques to:
● analyse and predict the deviations that could cause the post surgical
complications and,
● serve as an assistant leading to better patient-compliance and outcomes
43. 43
43
How do we solve problems with real world complexity, gather vast
amount of data, diverse knowledge……. and come up with
intelligent decisions that works for an individual at a given time?
next: a pedagogical take
45. Interplay between Semantic, Cognitive and Perceptual Computing (SC, CC and PC) with examples
Related videos, papers, slides via: http://knoesis.org/vision
Semantic Perceptual Cognitive computing in two use cases:
Asthma and Traffic Management
46. 46
Thank you
Thank you, and please visit us at
http://knoesis.org
For more information on kHealth, please visit us at
http://knoesis.org/projects/khealth
Cognitive
Computing
Semantic
Computing
Perceptual
Computing
Contributors and collaborators for this talk:
Pramod
Anantharam
Cory
Henson
Dr. T.K.
Prasad
Sanjaya
Wijeratne
Utkarshani
Jaimini
47. Ohio Center of Excellence in Knowledge-enabled Computing
Wright State University