Internet of Things, IoT

1. 1Copyright © 2014 Tata Consultancy Services Limited
Dr. Arpan Pal
Principal Scientist and Head of Research
Innovation Lab, Kolkata
Tata Consultancy Services
Research Challenges in Internet of Things (IoT)
17 May 2015

2. 2
Internet-of-Things
M2M Communication
Sensing the human – quantified self
Embedded software
and Hardware
Cloud, Mobile, Big Data
and Analytics
Wireless Sensor Networks,
Pervasive Computing
Sensors
and Actuators
Revenue Potential - $300+ Billion for Technology and Services
Economic Value - $1.9 Trillion
IoT - “a world-wide network of uniquely addressable and interconnected
objects, based on standard communication protocols”.
Objects that are -
• uniquely addressable
• aware of their “characteristics, context and situation”
• share information about themselves and surroundings
• actively participate in business processes and offer services
• have embedded sensors / actuators
• enable data collection, monitoring, decision making & optimizations

3. 3
Pervading all aspects of our life – Internet-of-Everything
Humans
Physical
Objects and
Infrastructure
Computing
Infrastructure
Physical
Context
Discovery
INTERNET OF EVERYTHING
Physical Context
Discovery
What is happening, where
and when
People Context
Discovery
Who is doing what, where
and when, who is thinking
what
Internet
of
Digital
Internet
of
Things
Internet
of
Humans
ABI Research. May 7, 2014
• New Business / Pricing Models, Always On–Anytime–Anywhere, Secure, Context-
aware - need to guarantee ROI for sustainability
• Customer becomes the focus, not the product or service – key is understanding the
Customer
• Analytics need to understand the Physics and Chemistry of the Physical World and the
Physiology and Psychology of the Humans

4. 4
IoT Architecture – complex Ecosystem and complex Technology stack
Sensor Manufacturers
Board Manufacturers
Cloud Infrastructure Providers
BANKING
INSURANCE
AGRICULTURE
HEALTHCARE
GOVERNMENT
UTILITY
MANUFACTURING
TRANSPORT
APPLICATION SERVICES
INFRASTRUCTURE PLATFORM
INTERNET
GATEWAY
RESPONDSENSEANALYZEEXTRACT
Processor and Semiconductor
Manufacturers
Network Equipment Manufacturers
System Integrators and Application
Developers
Embedded System Developers
Domain Experts
Telecom / M2M Providers
Data Scientists
Edge
Network
Cloud
Embedded Devices - gateway, mobile,
wearable
Sensor Signal
Processing
Protocols
and
Networking
Parallel
and
Distributed
Computing
Analytics
Security
and
Privacy
Model-driven
Development
(MDD)

5. 5
Fall Detection
PPG extraction
Eye Image / Video
Cardiovascular
Model
Pulse Oxymetry
Pupilometry
Fingertip
Video
Lung Function
Blood Pressure
Microphone
Accelerometer
Digital Stethoscope
Heart Rate
Using Mobile Phone Sensors for Physiological Measurements
Activity / Calorie
ECG
Respiratory Rate
HRV / Stress
Signal Processing for Noise Cancellation and Feature Extraction
Machine Learning on top of Physical Models for human Physiology

6. 6
Mobile Phone based Automotive Insurance
• Phone Picture – VIN identification and damage assessment – OCR and real-time 3D
reconstruction under noisy conditions
Need to simplify and speed up car accident insurance claim
• Driving behavior analysis and Road Condition Monitoring using mobile phone
accelerometer – Noise Modeling, Signal Processing, Statistical Processing
Need to promote safe driving and preventive maintenance
Acceleration a(t) = f (H(t), v(t), R(t), D(t))

7. 7
• Shopper Localization in Retail Stores
• Emergency Evacuation in Large Buildings
• Occupancy Estimation for Energy Savings
Need to localize people indoors
Mobile Phone based Indoor Localization
Geo-
fencing
• Using Magnetometer
Proximity
Detection
• Using Bluetooth RSSI
Inertial
Navigation
• Step Count + Stride Length (personalized
model)
• Gyroscope and Magnetometer-corrected
Inertial Navigation
Wi-Fi
based
Zoning
• RSSI based using attenuation modeling of
the building - Unsupervised Learning
Fusion • Kalman Filter based Tracking with Particle
Filter based Correction

8. 8
• Personalize education based on real-time
measurement of cognitive load
• Getting unbiased feedback from subject on
usability
Why Measure Cognitive Load
Cognitive Load on Human Brain – EEG and GSR processing
Cognitive Load
 23+45=?
 1846890129 + 2374609823=?
EEG GSR
Signal Processing for Noise Cancellation and Feature Extraction
Machine Learning on top of Cognitive Models for human Psychology

9. 9
• Unobtrusive Human Identification at Home – TRP analytics
• Neuro-rehabilitation
Application for Skeleton Analytics
Kinect Signal Processing
Research
– 20 joints of skeleton data
– Gait cycle detection
– Feature extraction from skeleton joints
– Training
– Recognition
– Gait Analytics
• 2D Camera with IR
depth sensor
• Excitation by IR light
pattern

10. 10
Multi-sensor Fusion for Robot-assisted Sensing
www.ese.wustl.edu
Cloud point
from 3D
vision
Possible
gas / heat
source
(ROI)
Source
direction
and intensity
• Robot carries 2D camera and thermal / microphone array
• 3D reconstruction from the 2D vision
• Estimation of Heat / sound Source through passive directional signal
processing
• Fusion of thermal / acoustic map with optical 3D – computational
thermography and audiography
• Gas Sensors planned in future
Application in remote sensing in hazard-prone areas

11. 11
Requirements for IoT Platform
Applications need support for
Visibility
Capture & store data
from sensors
Insights
Patterns, relationships
and models
Control Optimize and actuate
TCUP – TCS Connected Universe Platform - horizontal platform to address IoT Software and Services market
TCUP Platform
• To balance between energy cost, communication cost and
computing cost
Distributed Computing on Edge Devices
• To reduce network congestion
Adaptive, Lightweight yet Secure Communication Protocols
• For economical scaling of sensor data store
Efficient Compression
Manage
Scale,
Reduce
Cost,
Improve
Battery
Life
Handle Privacy Easy to Use AnalyticsSemantic Interoperability

12. 12
Horizontal operators
(semantic integration) operates on data from heterogeneous sources to created integrated data streams.
Sensor Data Analytics and Semantics - From Data to Wisdom
temperature
humidity
odor
image
high temperature
gaseous odor
light
concentrated light
high temperature
indicates fire
gaseous odor indicates
gas discharge
Fire from
Gas Leak,
evacuate
immediately,
send fire fighting team
equipped with gas leakage
data
information
knowledge
wisdom
Vertical operators
(semantic abstraction) operates on
artifacts at each level and
transcends them to the next level
F PCS(Data, KB*) → Information
F PCS(Knowledge, KB) → Wisdom
F PCS(Information, KB) → Knowledge
KB: Knowledge base
Adopted from: Physical-Cyber-Social Computing: An early 21st Century Approach, Amit Sheth et. al.

13. 13
A bigger challenge for Analytics – a wide variety of stakeholders
I only know the
business logic, I do
not know how to
code, nor do I
understand analytics
algorithms…
I know how to code,
but I do not know
algorithms, nor do I
know about the
business logic…
Oh, I know
algorithms, but I
can’t code for your
mobile devices…
I have all these
cloud and edge
nodes which you can
use to deploy the
app…
Need for Knowledge based Model-driven-development

14. 14
Source: www.winlab.rutgers.edu/~gruteser/papers/fp023-roufPS.pdf
Privacy Breach in IoT Applications
Pattern of living, activity,
occupancy revealed
Even Sleeping Smartphones Could Soon Hear Spoken Commands
Nuance is working with chipmakers on technology that would enable “persistent
listening” apps.
http://www.technologyreview.com/news/429316/even-sleeping-smartphones-could-soon-hear-spoken-commands/
MIT Technology Review, Sept. 2012
Vehicle Trip Overlay Over a Year reveals your hub
locations (home, office??)
Source: https://www.aclu.org/technology-and-liberty/meet-jack-or-what-government-
could-do-all-location-data
Data cannot be both contextually useful as well as
forever privacy preserving
Need Balance between Privacy and Security

15. 15
Innovation Lab Kolkata -at-a-glance
• Associates in R&D100+
• Researchers40+
• PhDs6
• Pursuing Higher Study8
• Papers published in last two years – www, SenSys,
Mobihoc, UbiComp, Infocomm, ICASSP, …..
125+
• Patents filed in last two years60+
• Patents granted till date15+
• Standard Body Participation and ContributionIETF, GISFI, TSDSI
Partnering Institutes (RSP, Research Collaboration)
 Indian Statistical Institute
 Institute of Neuroscience
 IIT Kharagpur, Mumbai, Guwahati
 Jadavpur University
 Calcutta University
 Missouri S&T
 SMU
 University of Maryland
 MIT, University of Toronto / Waterloo
(Exploring)
Long term Masters /
PhD interns

16. 16
Awards and Mentions
TCUP - Winner in Leading Edge Proven Technology
CoAP - IETF Fellowship from ISOC
Mobile Blood Pressure - Best Demo Award
Editorships for IEEE and ACM Transactions

17. 17
References
1. Philip B. Gibbons, et.al, IrisNet: An Architecture for a Worldwide Sensor Web, October 2003 IEEE Pervasive Computing ,
Volume 2 Issue 4
2. Open Geospatial Consortium, OGC Sensor Web Enablement Architecture,, December 2008
3. Deborah Estrin , Participatory Sensing: Applications and Architecture, January/February 2010, IEEE Internet Computing
4. Michael Chui, et.al, The Internet of Things, McKinsey Quarterly 2010, Number 2
5. W3C Incubator Group, Semantic Sensor Network XG Final Report, Report 28, June 2011
6. Dennis Pfisterer et.al, SPITFIRE: Towards a Semantic Web of Things, November 2011, IEEE Communication Magazine
7. S Bandyopadhyay, P Balamuralidhar, A Pal, Interoperation among IoT Standards, Journal of ICT Standardization, 2013
8. P Balamuralidhara, P Misra, A Pal, Software Platforms for Internet of Things and M2M, Journal of the Indian Institute of
Science, 2013
9. Bandyopadhyay, S. and Bhattacharyya, Lightweight Internet protocols for web enablement of sensors using constrained
gateway devices , ICNC 2013
10. S. Bandyopadhyay, A. Bhattacharyya, and A. Pal, Adapting protocol characteristics of CoAP using sensed indication for
vehicular analytics SenSys, 2013
11. A. Ukil, S. Bandyopadhyay, A. Bhattacharyya, and A. Pal, Lightweight security scheme for vehicle tracking system using
CoAP, ACM ASPI-Ubicomp Adjunct, 2013.
12. A. Ukil, S. Bandyopadhyay, A. Bhattacharyya, A. Pal and T. Pal, Auth-Lite: Lightweight M2MAuthentication reinforcing DTLS
for CoAP, IEEE Percom, 2014.
13. A Bhattacharyya, S Bandyopadhyay, A Pal, ITS-Light: Adaptive Lightweight Scheme to Resource Optimize Intelligent
Transportation Tracking System (ITS)–Customizing CoAP for Opportunistic Optimization, Mobiquitous 2014
14.Arpan Pal, Aniruddha Sinha, Anirban Dutta Choudhury, Tanushyam Chattopadyay, Aishwarya Visvanathan, A robust heart rate
detection using smart-phone video, ACM MobiHoc workshop on Pervasive wireless healthcare, 2013.
15.Anirban Dutta Choudhury, Aishwarya Visvanathan, Rohan Banerjee, Aniruddha Sinha, Arpan Pal, Chirabatra Bhaumik, Anurag
Kumar, HeartSense: estimating blood pressure and ECG from photoplethysmograph using smart phones, ACM Conference on
Embedded Networked Sensor Systems, 2013.
16.A Pal, A Visvanathan, AD Choudhury, A Sinha, Improved heart rate detection using smart phone , ACM SAC, 2014.
17.A Visvanathan, A Sinha, A Pal, Estimation of blood pressure levels from reflective Photoplethysmograph using smart phones
BIBE 2013.

18. 18
References
18.Vivek Chandel, Anirban Dutta Choudhury, Avik Ghose, Chirabrata Bhaumik, AcTrak-Unobtrusive Activity Detection and
Step Counting Using Smartphones , Mobiquitous 2013
19.Avik Ghose, Provat Biswas, Chirabrata Bhaumik, Monika Sharma, Arpan Pal, Abhinav Jha, Road condition monitoring and
alert application: Using in-vehicle Smartphone as Internet-connected sensor , PerCom Workshops 2012
20.Tapas Chakravarty, Avik Ghose, Chirabrata Bhaumik, Arijit Chowdhury MobiDriveScore-A system for mobile sensor based
driving analyis: a risk assessment model for improving one’s driving, ICST 2013
21.Tanushyam Chattopadhyay, V Ramu Reddy, Utpal Garain, Automatic Selection of Binarization Method for Robust OCR ,
ICDAR 2013
22.Arindam Saha, Brojeshwar Bhowmick, Aniruddha Sinha, A System for Near Real-Time 3D Reconstruction from Multi-view
Using 4G Enabled Mobile, IEEE MS 2014
23.A Mukherjee, A Pal, P Misra, Data Analytics in Ubiquitous Sensor-Based Health Information Systems, NGMAST, 2012
24.A Mukherjee, S Dey, HS Paul, B Das, Utilising condor for data parallel analytics in an IoT context—An experience report,,
9th IEEE International Conference on Wireless and Mobile Computing, Networking and Communications - IoT 2013
workshop
25.Felix Büsching et. al, DroidCluster: Towards Smartphone Cluster Computing--The Streets are Paved with Potential
Computer Clusters, ICDCSW 2012
26.DP Anderson, Boinc: A system for public-resource computing and storage, Fifth IEEE/ACM International Workshop on Grid
Computing, 2004.
27.A Banerjee, A Mukherjee, H S Paul, S Dey, Offloading work to mobile devices: an availability-aware data partitioning
approach, MCS 2013.
28.S Dey, A Mukherjee, HS Paul, A Pal, Challenges of Using Edge Devices in IoT Computation Grids, ICPADS 2013
29.A Mukherjee, HS Paul, S Dey, A Banerjee, ANGELS for distributed analytics in IoT, WF-IoT 2013
30.R. Arasanal and D. Rumani, Improving MapReduce performance through complexity and performance based data
placement in heterogeneous hadoop clusters, ICDCIT 2013.
31.Pankesh Patel, Brice Morin, Sanjay Chaudhary, A model-driven development framework for developing sense-compute-
control applications, MoSEMInA 2014
32.Bonomi Flavio, Rodolfo Milito, Jiang Zhu, and Sateesh Addepalli. Fog computing and its role in the internet of things, MCC
workshop on Mobile cloud computing 2012.
33.Arpan Pal, Arijit Mukherjee, Balamuralidhar P, Model-driven Development for Internet of Things: Towards easing the
concerns of Application Developers, IoTaaS, IoT 360, 2014

19. 19
Thank You
arpan.pal@tcs.com