Exploring the Future Potential of AI-Enabled Smartphone Processors
Arpan pal u-world
1. 1
Arpan Pal
Research Group Manager
Innovation Lab
Tata Consultancy Services
India
May 17, 2015
Use of Ubiquitous Device based
Sensing for Building Smart Solutions
2. Outline
• Intelligent Infrastructure using Internet-of-Things
• Application using Mobile Phone based Gateway – Road
Monitoring
• Application using Set top Box based Gateway – Home
Monitoring
• Intelligent HMI
• Conclusion
3. 3
Intelligent Infrastructure
“Intelligent”
• Context-aware
• Sense the environment
• Extract meaningful information
• Analyze to make inferences
• Respond through value-added
information
“Infrastructure”
• Home, Building, Store, Vehicle,
Roads, Utility Generation / Distribution
Network
Intelligent
Infra
@Home
@Building
@Vehicle
@Utility
@Mobile
@Store
@Road
Applications
• Transportation
• Safety
• Electricity
• Water
• Healthcare
• Education
Need for
• Privacy Preservation
• Processing at Edge to reduce bandwidth and hence communication cost
4. 4
Internet-of-Things Architecture for Intelligent Infrastructure
Internet
Sensors
(Location, Audio , Camera, …)
Gateway (Stand-alone
Device / Smart
Phones)
Backend
Server & database
Portal
USB, CAN, Zigbee, BT,
NFC, WiFi
GSM/GPRS, 3G
ADSL
BANKING
INSURANCE
AGRICULTURE
HEALTHCARE
GOVERNMENT
UTILITY
MANUFACTURING
TRANSPORT
APPLICATION SERVICES
INFRASTRUCTURE PLATFORM
INTERNET
GATEWAY
RESPONDSENSEANALYZEEXTRACT
Television / Set top
Box
Mobile
Phones
5. 5
Mobile Phones as Sensor and Gateway
Internet
WiFi, 2G/GPRS, 3G
On-board sensors
Accelerometer, GPS,
Compass
Camera, Microphone
Internet
Sensors
(Location, Audio , Camera, …)
Gateway (Stand-alone
Device / Mobile Phones)
Backend
Server & database
Portal
USB, CAN, Zigbee, BT,
NFC, WiFi
GSM/GPRS, 3G
Other Interfaces
Bluetooth and USB
Internet
2G/GPRS
On-board sensors
Microphone, Camera
Basic Phone
Smart Phone
6. 6
Sensor Penetration in Indian Mobile Phones
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
Bluetooth
USB
Edge
GPRS
Wif i
3G
Camera
GPS
Accelarometer
Digital Compass
Basic Phones
0 0.2 0.4 0.6 0.8 1
Bluetooth
USB
Edge
GPRS
Wif i
3G
Camera
GPS
Accelarometer
Digital Compass
Smart Phones
7. 7
Power Consumption using Sensors
Source: Nericell: Rich Monitoring of Road and Traffic Conditions using Mobile Smartphones
Prashant Mohan et. al., Microsoft Research, SenSys 2008, North Carolina, USA
9. 9
Road Condition Monitoring – Smart Phones
Technology
Sense
• Accelerometer / GPS in the Smart Phone in Car
Extract
• Phone-orientation Correction
• Feature Extraction /Anomaly Detection
Analyze
• Learning-based Classification
• Multi-user Fusion
Respond
• Road Condition Map to municipal authority
• Pothole alert to end-user
Use Case
• Road Condition Monitoring (Potholes, Bumps, Rough Roads)
Value-add
• Low-power Analytics on mobile phones
• Processing at Mobile Phone to reduce Internet Data Transfer
• Privacy Preservation through Identity masking
• Enhanced classification Accuracy
12. 12
Intelligent Home Solution using Home Monitoring
Gateway
Technology
Sense
• Energy (Water) Meter / Sub-meter
• Water leakage sensor and Thermal Camera
Extract
• Noise Cancellation
Analyze
• Demand-supply response
• Anomaly detection
Respond
• Aggregated demand forecast to utility company
• Historical Consumption / Analytics view to consumer
• Water / Electrical Problem alert to Insurance company
• Remote Electrical Appliance on-off Control
Use Case
• Monitor Electricity Consumption, Control Electrical
Appliances, Monitor Water Leakage, Monitor Faulty
Electrical Wires
Value-add
• Low-cost solution using TV as monitoring screen
• Appliance level monitoring and control
14. 14
Intelligent HMI
Ongoing Work
• User context-aware HMI
• Kinect based multimodal UI using visual gesture and directional audio
• EEG based BCI
Four screens of our life
• PC
• Television
• Mobile
• In-car display
15. 15
Conclusion
• Use of Mobile Phones and Television – the two most ubiquitous devices in
our life as Internet-of-Things Gateway
• Intelligent Transportation Applications using Mobile Phone as a Gateway
Sensor Penetration in Basic Phones and Smart Phones
Power Consumption Considerations
Privacy Protection
• Home Monitoring Applications using Set top Box as a Gateway
Gateway Functionality for Energy Monitoring and Surveillance
Applications
TV as a display device
Appliance level monitoring and control
• Intelligent HMI