SlideShare uma empresa Scribd logo
1 de 67
3/13/2017 1Demetris Trihinas 1
Low-Cost Approximate & Adaptive Monitoring
Techniques for the Internet of Things
Demetris Trihinas
trihinas@cs.ucy.ac.cy
PhD Candidate
Department of Computer Science
University of Cyprus
3/13/2017 2Demetris Trihinas 2
Cloud Computing
Big Data Systems and Analytics
Internet Computing and IoT
Online Social Networks
Activity Areas
Current EU-Funded
Projects
http://linc.ucy.ac.cy/
3/13/2017 3Demetris Trihinas 3
The Internet of Things
The physical world is now becoming an information system
Exchanging continuous data streams with other network-
enabled devices, systems and humansā€¦
Physical (battery-powered) and network-enabled devices
with smart processing capabilitiesā€¦
3/13/2017 4Demetris Trihinas 4
Why is IoT Just Now Becoming a Reality?
Nano Bio-Sensors and
Printed Electronics
Communication
Protocols
Data-Mining
Learning Algorithms
- Distributed closed neighbors
- Outlier/event detection
- Pattern matching
- Stream processing
- Learning algorithms
- ā€¦.
ā€œThe Emerging Internet of Things Market Perspective: A Surveyā€, Perera et al., IEEE Trans. Computing, 2015
ā€œInternet of Things (IoT): A Vision, Architectural Elements, and Future Directionsā€, J. Gubbi and R. Buyya, FGCS, 2013
The Building Blocks of the Internet of
Things
3/13/2017 5Demetris Trihinas 5
IoT in Agriculture
Precision Farming
Detect crop condition and spread
nutrients to parts of field with need
Cattle as a Service
Detect optimum breading
period, monitor health and
dairy protection
ā€œConnected Cattle: Wearables are Changing the Dairy Industryā€, Kathy Pretz, IEEE Magazine: the Institute, May 2016.
3/13/2017 6Demetris Trihinas 6
IoT in Medicine and Health Tracking
Precision Medicine
Pill-shaped cameras traversing
human body
ā€œFitbit data led doctors to shock a patientā€™s heartā€, Steven Dent, Engadget, Apr 2016.
Wearables
Biosignal and activity tracking
3/13/2017 7Demetris Trihinas 7
IoT in Intelligent Transportation Systems
Smart Traffic Flow
In Toronto travel time
was reduced by 26%
Smart Parking
In California carbon emissions
were reduced by 730 tons per
15 block radius
ā€œCan smart traffic lights ease Torontoā€™s road congestion?ā€, Joseph Hall, Toronto Star, Jun 2014.
3/13/2017 8Demetris Trihinas 8
The ā€œBig Dataā€ in IoT
[IDC, Jun 2014]
[Cisco, Apr 2011]
IoT Monitoring Data
2% of digital universe in
2012
Projections for >12% in
2020
[Gartner, Mar 2015]
IP Traffic 1.6ZB in 2018, 57% from IoT Devices
[Cisco, Apr 2014]
3/13/2017 9Demetris Trihinas 9
Big Data and the Cloud
3/13/2017 10Demetris Trihinas 10
IoT and Cloud Computing
Driving Cloud Adoption for IoT Developers
- Simplified resource management
- low capital expenses to get started
[IoT Developer Survey, Eclipse Foundation, 2016]
67% of production-ready
IoT services reside in the
cloud
[State of IoT, Embarcadero, 2016]
3/13/2017 11Demetris Trihinas 11
Soā€¦is Cloud Monitoring Really
Ready for the Internet of Things?
3/13/2017 12Demetris Trihinas 12
Monitoring Topologies
Scalability and Single points of failure
Monitoring servers closer to the root are overloaded
with relay costs preventing scalability
ā€œJCatascopia: Monitoring Elastically Adaptive Applications in the Cloudā€, D. Trihinas, G. Pallis and M. D. Dikaiakos, IEEE/ACM CCGrid 2014
3/13/2017 13Demetris Trihinas 13
Monitoring as a Service (MaaS)
ā€¢ Eases management of the monitoring infrastructure
ā€¢ Service providers enable multi-tenant and scalable
monitoring over the internet
ā€¢ Pay-as-you-use business model (although usually hourly)
Monitoring libraries
for metric collection from
popular languages and
frameworks
REST API to
insert/extract
monitoring data
Shared data warehouse with
services to query, process and
view analytic insights
custom app-specific metrics
managed by cloud provider
3/13/2017 14Demetris Trihinas 14
Augmenting IoT with the Cloud
ā€œ
app
The Harsh Reality
Network latencies, bandwidth limitations,
pricing (in/out cloud traffic is billed),
constant location awareness
Datacenter
Interesting Articles
ā€œThe cloud is not enough: Saving IoT from the cloudā€, B. Zhang et al., Usenix HotCloud 2015
ā€œTaking the internet to the next physical levelā€, V. Cerf and M. Senges, IEEE Computer, 2016
The IoT Developer Perspective
3/13/2017 15Demetris Trihinas 15
The Cost of Monitoring
Based on AWS pricing scheme (May 2015)
for a 3-tier (video) streaming application
$0.12/GB
network traffic
ā€œMonitoring Elastically Adaptive Multi-Cloud Servicesā€, D. Trihinas, G. Pallis and M.D. Dikaiakos, IEEE TCC 2016
3/13/2017 16Demetris Trihinas 16
P2P Monitoring
ā€¢ Monitored elements form Gossip Overlay Network to propagate monitoring
info -> ease coordination and query computation
ā€¢ Gossip denotes periodic and pairwise propagation of the current state between
monitored peers
ā€œMonitoring Elastically Adaptive Multi-Cloud Servicesā€, D. Trihinas, G. Pallis and M.D. Dikaiakos, IEEE TCC 2016
ā€œSelf managing monitoring for highly elastic large scale cloud deploymentsā€, S. Ward et al., ACM DIDC 2014
3/13/2017 17Demetris Trihinas 17
Edge Computing
Init
Sense
Act
Sleep
Receive
Decide
Transmit
A term coined to reflect data processing and decision-making on
ā€œsmartā€ devices that sit at the edge of IoT networks
Init
Sense
Transmit
Sleep
Interesting Articles
ā€œThe promise of edge computingā€, W. Shi and S. Dustdar, IEEE Computer, 2016
ā€œThe swarm at the edge of the cloudā€, E. Lee et al., IEEE Design & Test, Jun 2014
Simple
Sensing
Edge
Computing
<10ms
Big Data
Traffic
>100ms
ProcessingNo Processing
3/13/2017 18Demetris Trihinas 18
Augmenting IoT with the Cloud
Challenge 1 ā€“ Taming data volume and data velocity with limited
processing and network capabilities in the IoT/Edge realm
Challenge 2 - IoT devices are usually battery-powered which
means intense processing leads to less battery-life
Processing and data dissemination are
the main energy drains in embedded
and mobile devices
Interesting Article
ā€œPractical data prediction for real-world wireless sensor networksā€, U. Raza and T. Palpanas, IEEE TKDE, 2015
3/13/2017 19Demetris Trihinas 19
Augmenting IoT with the Cloud
ā€œTaking the internet to the next physical levelā€, V. Cerf and M. Senges, IEEE Computer, 2016
Challenge 3 ā€“ Security and data privacy
October 2016, DDOS Attack
3/13/2017 20Demetris Trihinas 20
Formulating the Challenges as
Research Questions
3/13/2017 21Demetris Trihinas 21
Preliminaries
ā€¢ A monitoring stream š‘€ = {š‘‘š‘–}š‘–=0
š‘›
published by a monitoring source is a large
stochastic sequence of i.i.d datapoints, denoted as š‘‘š‘– , where š‘– = 0, 1, ā€¦ , š‘›
and š‘› ā†’ āˆž
ā€¢ A datapoint š‘‘š‘– is a tuple š‘šš‘–š‘‘, š‘”š‘–, š‘£š‘– described by a unique identifier for the
monitoring source š‘šš‘–š‘‘, a timestamp š‘”š‘– and a value š‘£š‘–
ā€¢ A datapoint may have a set of other attributes (e.g., location) although for
brevity we will omit other attributes without loss of generality
š‘‘š‘–
š‘‘š‘–+1
Monitoring Stream M
3/13/2017 22Demetris Trihinas 22
Periodic Sampling
ā€¢ The process of triggering the collection mechanism of a monitored source
every š‘‡ time units such that the š‘– š‘”ā„Ž datapoint is collected at time š‘”š‘– = š‘– āˆ™ š‘‡
š‘‘š‘–
š‘‘š‘–+1
Metric Stream M sampled every T = 1s
Compute resources and energy are wasted while generating large data volumes at a high velocity
Metric Stream M sampled every T = 10s
Sudden events and significant insights are missed
3/13/2017 23Demetris Trihinas 23
More processing
(e.g., peak detection
for hr monitoring) Driving Peripherals
(e.g., accelerometer,
leds)
Interesting Articles
ā€œEnergy-harvesting wearables for activity-aware servicesā€, S. Khalifa et al., IEEE Internet Computing, 2015
ā€œTaking the internet to the next physical levelā€, V. Cerf and M. Senges, IEEE Computer, 2016
Emulated behavior of wearable device on Raspberry Pi with ARM processor
(1 core 32MHz, 128MB RAM)
Periodic Sampling with Physical
Sensing
OS Monitoring
3/13/2017 24Demetris Trihinas 24
Periodic Sampling in an Energy Perspective
Lithium battery (35mAh) State of Charge (SoC) projection for a wearable
device with step, calorie and heartrate monitoring
Interesting Article
ā€œA universal state-of-charge algorithm for batteriesā€, B. Xiao et al., ACM DAC, 2010
3/13/2017 25Demetris Trihinas 25
Periodic Dissemination
Monitoring
Source
Receiving
Entity
ā€¢ The process of triggering the network controller of a monitored source every
š· time units such that the š‘– š‘”ā„Ž
datapoint is disseminated at time š‘”š‘– = š‘– āˆ™ š· to
interested receiving entities
No data loss
ā€¢ Aggregation-based policy dissemination:
ā€¢ Withhold triggering dissemination until a window of š‘² datapoints is collected
ā€¢ Withhold dissemination for fixed period of time and disseminate all datapoints
collected until then
3/13/2017 26Demetris Trihinas 26
Periodic Dissemination Monitoring Cost
šœ‡ š‘  + š›½š‘  āˆ™ Ļ‡ + Īµs
per message cost
per Ļ‡ byte cost
datapoint d (t,v)
-> message compression
Interesting Article
ā€œReal-Time Data Analytics in Sensor Networksā€, T. Palpanas, ACM Sensors, 2013
energy for state transitions
Example Values (Mica2)
šœ‡ š‘  = .645 mJ
Īµs = .331 mJ
š›½š‘  = .0144 mJ/byte
3/13/2017 27Demetris Trihinas 27
Low-Cost Approximate
and Adaptive Monitoring
3/13/2017 28Demetris Trihinas 28
Preliminaries
ā€¢ Approximate Techniques?
ā€¢ Decision mechanisms based on estimation models capturing and predicting
monitoring stream runtime evolution within certain accuracy guarantees
ā€¢ Adaptive?
ā€¢ Adapt monitoring source properties based on the predicaments of the
estimation models
ā€¢ Low-Cost?
ā€¢ The costs (time, resources) of applying adaptive monitoring techniques are
(much) less than leaving the monitoring process as is
ā€¢ We are interested in techniques running in-place and inexpensively
3/13/2017 29Demetris Trihinas 29
Adaptive Sampling
ā€¢ Dynamically adapt the sampling period š‘‡š‘– based on some estimation model
šœŒ(š‘€), containing runtime information of the monitoring stream evolution
ā€¢ How large of an adjustment is required, ā€œideallyā€, depends on some evaluation
(distance) metric denoting the ability of the model to (correctly) estimate and
follow metric stream evolution
š‘‘š‘–
š‘‘š‘–+1
š‘‡š‘–+1
Metric Stream š‘€ā€² dynamically sampledMetric Stream š‘€ with š‘‡ = 1š‘ 
3/13/2017 30Demetris Trihinas 30
Adaptive Sampling
š‘‘š‘–
š‘‘š‘–+1
š‘‡š‘–+1
Metric Stream š‘€ā€² dynamically sampledMetric Stream š‘€ with š‘‡ = 1š‘ 
Find the max š‘‡ āˆˆ [š‘‡ š‘šš‘–š‘›, š‘‡ š‘šš‘Žš‘„] to collect š‘‘š‘–+1 based on an estimation of
the monitoring stream evolution šœŒ(š‘€), such that š‘€ā€² differs from š‘€ less
than a user-defined imprecision value š›¾ (š‘‘š‘–š‘ š‘” < š›¾)
3/13/2017 31Demetris Trihinas 31
Adaptive Dissemination
ā€¢ Dynamically adapt dissemination rate by applying approximation
techniques to sensed datapoints to reduce communication overhead
ā€¢ By suppressing consecutive datapoints from dissemination with
ā€œlittleā€ change in their metric values
ā€¢ How much is change is considered as ā€œlittleā€ depends on:
ā€¢ The approximation technique
ā€¢ Accuracy guarantees given by the user and denoting the probability
(confidence Ī“) with which sensed datapoints are approximated
3/13/2017 32Demetris Trihinas 32
Adaptive Dissemination (Model-Based)
ā€¢ Monitoring source maintains runtime estimation model šœŒ(š‘€) capturing monitoring
stream evolution and variability
ā€¢ At the š‘– š‘”ā„Ž
time interval instead of metric values, the model is disseminated
ā€¢ Receiving entities predict the IoT device state from given model assuming subsequent
k-datapoints can be approximated di+k|i = f(šœŒ š‘€ , š‘‘š‘–) within the given guarantees
Metric Stream š‘€ Approximated Metric Stream š‘€ā€²
3/13/2017 33Demetris Trihinas 33
Adaptive Dissemination (Model-Based)
ā€¢ Monitoring source withholds further dissemination interacting with receiver
only when shifts in monitoring stream value distribution render model as
inconsistent with the actual IoT device state
ā€¢ If model parameterization is inconsistent, at this point, it must be updated
decision criteria?
decision
function?
Metric Stream š‘€ Approximated Metric Stream š‘€ā€²
3/13/2017 34Demetris Trihinas 34
Metric Filtering
ā€¢ A form of adaptive dissemination: suppress latest datapoint(s)
dissemination if their values have not ā€œchangedā€ since last reported
ā€¢ Metric stream suppression butā€¦ no model -> no forecasting
ā€¢ Fixed filter ranges assume monitoring stream value distribution will not
change in in the future (no guarantees any values will be filtered at all)
if (curValue āˆˆ [prevValue ā€“ R, prevValue + R ])
filter(curValue)
Interesting Article
ā€œMonitoring, aggregation and filtering for efficient management of virtual networksā€, S. Clayman, IEEE NOMS, 2011
3/13/2017 35Demetris Trihinas 35
Adaptive Filtering
ā€¢ Dynamically adjust the filter range š‘… based on the current variability of the
monitoring stream, denoted as š‘ž(š‘€)
Metric Stream š‘€ā€² with adaptive RangeMetric Stream š‘€
š‘‘š‘–
š‘‘š‘–+1
š‘…š‘–+1
š‘…š‘–
ā€¢ Find the max filter range š‘…š‘–+1 āˆˆ [š‘… š‘šš‘–š‘›, š‘… š‘šš‘Žš‘„] for š‘‘š‘–+1 such that š‘€ā€² differs
from š‘€ less than a user-defined imprecision value š›¾ based on the variability
of the metric stream š‘ž(š‘€)
3/13/2017 36Demetris Trihinas 36
AdaM
3/13/2017 37Demetris Trihinas 37
The Adaptive Monitoring Framework
Ī¼Processor
Ī¼OS
Memory
Low-Cost Approximate Stream Estimation
Adaptive
Sampling
Adaptive
Filtering
S
E
N
S
I
N
G
U
N
I
T
metric
updates
A
P
I
updated
periodicity
estimation
confidence
Adaptive
Dissemination
model updates &
buffer content
when shift
detected
updated filter range
stream variability
filtered metric stream
AdaM
N
E
T
W
O
R
K
U
N
I
T
Model
Base
stream
evolution
and online
stats
Datapoint
Buffer
IoT Device
metric
stream
compressed
metric
stream
3/13/2017 38Demetris Trihinas 38
Low-Cost Approximate Estimation Model
ā€¢ Three (3) phase approach to approximate monitoring stream
evolution and variability by exploiting knowledge (hidden) in
the monitoring stream
Step 1
Estimate Monitoring
Stream Evolution
and label datapoints
as ā€œ(un)expectedā€
Step 2
Detect if Gradual
Trends Exist in
Monitoring Stream
Evolution
Step 3
Test if Seasonality
Enrichment is
Beneficial to Stream
Evolution
3/13/2017 39Demetris Trihinas 39
Low-Cost Approximate Estimation Model
Phase 1: Update runtime monitoring stream evolution šœŒ Īœ
ā€¢ Probabilistic Exponential Weighted Moving Average (PEWMA) to estimate
š‘£š‘–+1 from šœ‡š‘– and the standard deviation šœŽš‘–+1:
š‘£š‘–+1 = šœ‡š‘– = š‘Ž āˆ™ šœ‡š‘–āˆ’1 + 1 āˆ’ š‘Ž š‘£š‘–
ā€¢ Datapoints labelled as ā€œexpectedā€ if estimation lands in prediction intervals
determined from user confidence guarantees or ā€œunexpectedā€ otherwise
ā€¢ ā€œUnexpectedā€ datapoints are stored in local buffer for dissemination
Looks like an exponential moving average, right?
But weighting š‘Ž = š›¼ 1 āˆ’ š›½Pš‘– is probabilistically applied!
3/13/2017 40Demetris Trihinas 40
Low-Cost Approximate Estimation Model
EWMA after spikes
overestimates
subsequent values
EWMA slow to
acknowledge spikes
With probabilistic reasoning each datapoint will contribute to the
estimation process depending on itā€™s p-value
Why use a Probabilistic EWMA?
3/13/2017 41Demetris Trihinas 41
AdaM: Adaptive Sampling and Filtering
ā€œAdaM: Adaptive Monitoring Framework for Sampling and Filtering on IoT Devicesā€, D. Trihinas, G. Pallis and M.D. Dikaiakos, IEEE BigData 2015
ā€œ[Low-Cost Adaptive Monitoring Techniques for the Internet of Thingsā€, D. Trihinas, G. Pallis and M.D. Dikaiakos, IEEE TSC 2016
Ī¼Processor
Ī¼OS
Memory
Low-Cost Approximate Stream Estimation
Adaptive
Sampling
Adaptive
Filtering
S
E
N
S
I
N
G
U
N
I
T
metric
updates
A
P
I
updated
periodicity
estimation
confidence
Adaptive
Dissemination
model updates &
buffer content
when shift
detected
updated filter range
stream variability
filtered metric stream
AdaM
N
E
T
W
O
R
K
U
N
I
T
Model
Base
stream
evolution
and online
stats
Datapoint
Buffer
IoT Device
metric
stream
compressed
metric
stream
3/13/2017 42Demetris Trihinas 42
Adaptive Sampling
ā€¢ After updating estimation model, we compute current confidence (š‘š‘– ā‰¤ 1)
of approach based on the actual and estimated standard deviation
ā€¢ Next, we update sampling period š‘‡š‘–+1 based on the current confidence
and the user-defined imprecision Ī³ (e.g. 10% tolerance to errors)
š‘š‘– = 1 āˆ’
| šœŽš‘– āˆ’ šœŽš‘–|
šœŽš‘–
ā€¦ the more ā€œconfidentā€ the algorithm is, the larger the
outputted sampling period š‘‡š‘–+1 can beā€¦
Ī» is an
aggressiveness
multiplier
(default Ī»=1)
3/13/2017 43Demetris Trihinas 43
Adaptive Filtering
ā€¢ Compute current variability of the metric stream using moving Fano
Factor š¹š‘– which is an indicator of the stream value dispersion
ā€¢ We then compare šœŽš‘’š‘Ÿš‘Ÿ to the user-provided maximum tolerable
imprecision guarantees š›¾ in attempt to widen filter range
š¹š‘– =
šœŽš‘–
2
šœ‡š‘–
ā€¦a low š¹š‘– (due to šœŽš‘–) indicates a currently in-dispersed data
stream which means low variability in the metric streamā€¦
3/13/2017 44Demetris Trihinas 44
AdaM in action!
Steps Heartrate
Calories
94%
accuracy!
96%
accuracy!
91%
accuracy!
3/13/2017 45Demetris Trihinas 45
6x less processing! 4x less network traffic!
4x less energy usage!
+ AdaM
3 - 5 days
7 - 8 days
AdaM in action!
3/13/2017 46Demetris Trihinas 46
Memory Trace
Java Sorting Benchmark
CPU Trace
Carnegie Mellon RainMon Project
Disk I/O Trace
Carnegie Mellon RainMon Project
TCP Port Monitoring Trace
Cyber Defence SANS Tech Institute
Step Trace
Fitbit Charge HR Wearable
Heartrate Trace
Fitbit Charge HR Wearable
More datasets!
3/13/2017 47Demetris Trihinas 47
Balance between efficiency and accuracy!
Energy Consumption
CPU Cycles Network Traffic
Error (MAPE)
FAST [L. Fan et al., SIGMOD 2014] L-SIP [E Gaura et al., IEEE Trans. on Sensors, 2013]
3/13/2017 48Demetris Trihinas 48
Dublin Smart City Intelligent Transportation Service (Dublin ITS)
.
.
.
.
.
.
1000 Buses* with GPS tracking
sending updates to ITS with 16
params (e.g. busID, location,
current route delay)
*Real data from Jan. 2014
Apache Kafka queuing
service on x-large VM
(16VCPU, 16GB RAM,
100GB Disk)
Apache Spark cluster with 5 workers on large VMs
(8VCPU, 8GB RAM, 40GB Disk)
Alerts ITS operators when more than 10 buses in a
Dublin city area, per 5min window, are reporting delays
over 1 standard deviation from their weekly average
AdaMā€¦ integrated in big data streaming service!
3/13/2017 49Demetris Trihinas 49
Spark total delay (processing + scheduling) for T=1, 5, 10 intervals and
AdaM with max imprecision Ī³ = 0.15
AdaMā€¦ integrated in big data streaming service!
3/13/2017 50Demetris Trihinas 50
AdaM achieves a >85% accuracy in major Dublin areas
AdaMā€¦ integrated in big data streaming service!
3/13/2017 51Demetris Trihinas 51
AdaM: Adaptive Dissemination
ā€œADMin: Adaptive Monitoring Dissemination for the Internet of Thingsā€, D. Trihinas, G. Pallis and M.D. Dikaiakos, IEEE INFOCOM, 2017
Ī¼Processor
Ī¼OS
Memory
Low-Cost Approximate Stream Estimation
Adaptive
Sampling
Adaptive
Filtering
S
E
N
S
I
N
G
U
N
I
T
metric
updates
A
P
I
updated
periodicity
estimation
confidence
Adaptive
Dissemination
model updates &
buffer content
when shift
detected
updated filter range
stream variability
filtered metric stream
AdaM
N
E
T
W
O
R
K
U
N
I
T
Model
Base
stream
evolution
and online
stats
Datapoint
Buffer
IoT Device
metric
stream
compressed
metric
stream
3/13/2017 52Demetris Trihinas 52
AdaM: Adaptive Dissemination
ā€œADMin: Adaptive Monitoring Dissemination for the Internet of Thingsā€, D. Trihinas, G. Pallis and M.D. Dikaiakos, IEEE INFOCOM, 2017
Network
Unit
ADMin
Seasonality
Enrichment
Shift
Detection
A
P
I
Model
Base
Approximate
Stream
Estimation
Remote seasonal
periodicity
update
test if enrichment
is beneficial?
metric
updates
update
stream
evolution
and trend
seasonal
enrichment of
stream evolution
compressed
metric stream
Seasonal Periodicity
Detection
The ADMin
plugin for AdaM
Donā€™t disseminate
datapoint values...
Instead disseminate
model updates!
3/13/2017 53Demetris Trihinas 53
Low-Cost Approximate Estimation Model
Step 2: Detect over time gradual trends in monitoring stream to reduce
ā€œlaggingā€ effects in monitoring stream evolution estimation
š‘£š‘–+š‘˜|š‘– šœ‡š‘– + š‘˜ š‘‹š‘–
Upward
trend
Downward
trend
ā€¢ Holtā€™s Trend Method used to bring moving average to
appropriate value base
ā€¢ Improve forecasting from 1-step ahead (moving
average) predictions to k-datapoint values
3/13/2017 54Demetris Trihinas 54
Low-Cost Approximate Estimation Model
Step 3: Test if seasonality enrichment is beneficial to estimation process
and update low-cost approximate model accordingly
seasonal
Seasonal with
exponential trend
Seasonal with
damped trend
ā€¢ Tendency of the metric stream to exhibit behavior that
repeats itself every L periods (e.g., hourly)
ā€¢ Seasonal effects highly evident in IoT data (e.g., human
biosignals, environmental data)
PV Panel Production Air Temperature
3/13/2017 55Demetris Trihinas 55
Low-Cost Approximate Estimation Model
ā€¢ Holt Winterā€™s Method used to estimate seasonal contribution
ā€¢ Forecasting k-subsequent datapoints with trend and seasonality
ā€¢ Howeverā€¦ perfect seasonal behavior is rarely observed in real-life
systems PV Panel Production
š‘£š‘–+š‘˜|š‘– šœ‡š‘– + š‘˜ š‘‹š‘– + š‘†š‘–
Considering day before hourly
average (Si-L) will lead to
overestimation
3/13/2017 56Demetris Trihinas 56
Low-Cost Approximate Estimation Model
ā€¢ Online testing (T-Test) to determine if seasonal contribution beneficial
to estimation or not
ā€¢ Detecting optimal seasonality cycle (L) is an open research challenge
especially when different cycles exist in monitoring stream
ā€¢ Approximate runtime seasonal periodicity detection
ā€¢ ComCube Framework (Matsubara et al., WWW, 2016): lightweight tensor-based
and parameter-free framework for near-optimal seasonal periodicity detection
š‘£š‘–+š‘˜|š‘– šœ‡š‘– + š‘˜ š‘‹š‘– + š‘†š‘–
ā€œNon-linear mining of competing local activities ā€, Y. Matsubara, Y. Sakurai and C. Faloutsos, WWW 2016
3/13/2017 57Demetris Trihinas 57
Detecting Shifts in a Monitoring Stream
ā€¢ Cumulative Sum (CUSUM) to detect shifts in monitoring stream value
distribution which render estimation model as inconsistent
ts
prior shift
after shift
š‘š‘– = ln
š‘ƒ(š‘€š‘–, šœ‡ā€²ā€²
)
š‘ƒ š‘€š‘–, šœ‡ā€²
šœ‡š‘–
ā€²ā€²
= šœ‡š‘–
ā€²
+ šœ€
where Īµ magnitude
of change
1. log-likelihood ratio 2. Detecting shifts in mean
However, šœ€ not known beforehand
butā€¦ estimation model model provides us with an approximate šœŗ
3/13/2017 58Demetris Trihinas 58
Detecting Shifts in a Monitoring Stream
3. Online CUSUM
4. Decision Function
Challenge 1: Linear timeā€¦ Can š’•š’Š be used
instead of š’• š’”?
ts ti
ti may greatly from ts differ in
cases of gradual trends
š‘–š‘“ šŗš‘–,{š‘™š‘œš‘¤,ā„Žš‘–š‘”ā„Ž} > ā„Ž ā†’ š‘ ā„Žš‘–š‘“š‘” š‘‘š‘’š‘”š‘’š‘š‘”š‘’š‘‘
h is measured
in standard
deviation units
5. Actual Shift Time
the time the
CUSUM detects
the shifts
3/13/2017 59Demetris Trihinas 59
Detecting Shifts in a Monitoring Stream
ts ti
Trend and seasonality knowledge provide model with greater accuracy ( šœ€ ā†’ šœ€) and to
adapt to unexpected, abrupt and and volatile changes is monitoring stream
Challenge 2: CUSUM threshold h static and
sensitive when stream variability is low
(šœŽ ā†’ 0 ) thus triggeringā€¦ false alarms
Adapt CUSUM sensitivity by adapting
h after dissemination triggered and
restrict h with hmin
3/13/2017 60Demetris Trihinas 60
ADMin Evaluation
ā€¢ ADMin in 3 configurations
ā€¢ No seasonality enrichment (ADMin)
ā€¢ Static seasonality enrichment - previous day hourly average (ADMin_S1)
ā€¢ Dynamic seasonality enrichment - ComCube integration (ADMin_S2)
ā€¢ Under comparison frameworks
ā€¢ LANCE [Werner et al., ACM SenSys 2011]
ā€¢ G-SIP [Gaura et al., IEEE Trans. on Sensors 2013]
ā€¢ ADWIN [Bifet et al., SIAM 2010]
All three framework parameters configured to output best results
3/13/2017 61Demetris Trihinas 61
Photovoltaic Panel Current (IDC) Production
2 Weeks of data collected every 1 second
Data reduction: 87% -- Accuracy: 93%
Weather Station Air Temperature (oC)
2 Weeks of data collected every 1 second
Data reduction: 85% -- Accuracy: 92%
Wearable Human Heartrate (bpm)
2 Weeks of data collected every 1 minute
Data reduction: 80% -- Accuracy: 90%
ADMin in Action!
3/13/2017 62Demetris Trihinas 62
Shift Detection Evaluation
ground truth
(offline PELT
algorithm)
false
alarms
correctly
detected
shifts
Air Temperature Dataset Wearable Dataset
Shift Detection Delay
3/13/2017 63Demetris Trihinas 63
Energy Consumption
Data Volume Reduction
1O
interval
aggregation
ADWIN large energy consumption due to
complexity and absence of data reduction
scheme
LANCE large energy consumption due to enabling
dissemination x2 more than ADMin (false alarms)
and downloading each time all available datapoints
G-SIP large energy consumption due to shift
detection sensitivity (only EWMA based) which
enables false alarm disseminations
Overhead Evaluation
3/13/2017 64Demetris Trihinas 64
Facebook DB Cluster Adaptive Monitoring
ā€œInside the Social Networkā€™s (Datacenter) Networkā€, A. Roy et al., ACM SIGCOMM, 2015
3/13/2017 65Demetris Trihinas 65
ā€¢ Edge-mining on IoT devices is both resource and energy
intensive
ā€¢ Big data streaming engines struggle to cope as the volume
and velocity of IoT-generated data keep increasing
ā€¢ Adapts the monitoring intensity based on current metric
evolution and variability
ā€¢ Reduces processing, network traffic and energy
consumption on IoT devices and the IoT network
ā€¢ Achieves a balance between efficiency and accuracy
The AdaM Framework
Conclusion
3/13/2017 66Demetris Trihinas 66
http://linc.ucy.ac.cy/AdaM
3/13/2017 67Demetris Trihinas 67
Low-Cost Approximate & Adaptive Monitoring
Techniques for the Internet of Things
Demetris Trihinas
trihinas@cs.ucy.ac.cy
PhD Candidate
Department of Computer Science
University of Cyprus

Mais conteĆŗdo relacionado

Mais procurados

grid computing
grid computinggrid computing
grid computingelliando dias
Ā 
Gridcomputingppt
GridcomputingpptGridcomputingppt
Gridcomputingpptnavjasser
Ā 
Big data visualization frameworks and applications at Kitware
Big data visualization frameworks and applications at KitwareBig data visualization frameworks and applications at Kitware
Big data visualization frameworks and applications at Kitwarebigdataviz_bay
Ā 
Application-Aware Big Data Deduplication in Cloud Environment
Application-Aware Big Data Deduplication in Cloud EnvironmentApplication-Aware Big Data Deduplication in Cloud Environment
Application-Aware Big Data Deduplication in Cloud EnvironmentSafayet Hossain
Ā 
Unit i introduction to grid computing
Unit i   introduction to grid computingUnit i   introduction to grid computing
Unit i introduction to grid computingsudha kar
Ā 
Fog Computing An Empirical Study
Fog Computing An Empirical StudyFog Computing An Empirical Study
Fog Computing An Empirical Studyijtsrd
Ā 
Big Data Visualization
Big Data VisualizationBig Data Visualization
Big Data Visualizationbigdataviz_bay
Ā 
Challenges and Issues of Next Cloud Computing Platforms
Challenges and Issues of Next Cloud Computing PlatformsChallenges and Issues of Next Cloud Computing Platforms
Challenges and Issues of Next Cloud Computing PlatformsFrederic Desprez
Ā 
Grid computing notes
Grid computing notesGrid computing notes
Grid computing notesSyed Mustafa
Ā 
Predictive Maintenance Using Recurrent Neural Networks
Predictive Maintenance Using Recurrent Neural NetworksPredictive Maintenance Using Recurrent Neural Networks
Predictive Maintenance Using Recurrent Neural NetworksJustin Brandenburg
Ā 
Grid computing by vaishali sahare [katkar]
Grid computing by vaishali sahare [katkar]Grid computing by vaishali sahare [katkar]
Grid computing by vaishali sahare [katkar]vaishalisahare123
Ā 
Grid computing
Grid computingGrid computing
Grid computingWipro
Ā 
ISWC 2016 Tutorial: Semantic Web of Things M3 framework & FIESTA-IoT EU project
ISWC 2016 Tutorial: Semantic Web of Things  M3 framework & FIESTA-IoT EU projectISWC 2016 Tutorial: Semantic Web of Things  M3 framework & FIESTA-IoT EU project
ISWC 2016 Tutorial: Semantic Web of Things M3 framework & FIESTA-IoT EU projectFIESTA-IoT
Ā 
Grid computing ppt 2003(done)
Grid computing ppt 2003(done)Grid computing ppt 2003(done)
Grid computing ppt 2003(done)TASNEEM88
Ā 

Mais procurados (20)

grid computing
grid computinggrid computing
grid computing
Ā 
Sensor Cloud
Sensor CloudSensor Cloud
Sensor Cloud
Ā 
Gridcomputingppt
GridcomputingpptGridcomputingppt
Gridcomputingppt
Ā 
Big data visualization frameworks and applications at Kitware
Big data visualization frameworks and applications at KitwareBig data visualization frameworks and applications at Kitware
Big data visualization frameworks and applications at Kitware
Ā 
Application-Aware Big Data Deduplication in Cloud Environment
Application-Aware Big Data Deduplication in Cloud EnvironmentApplication-Aware Big Data Deduplication in Cloud Environment
Application-Aware Big Data Deduplication in Cloud Environment
Ā 
Unit i introduction to grid computing
Unit i   introduction to grid computingUnit i   introduction to grid computing
Unit i introduction to grid computing
Ā 
Fog Computing An Empirical Study
Fog Computing An Empirical StudyFog Computing An Empirical Study
Fog Computing An Empirical Study
Ā 
Big Data Visualization
Big Data VisualizationBig Data Visualization
Big Data Visualization
Ā 
Grid computing
Grid computing Grid computing
Grid computing
Ā 
Grid Computing
Grid ComputingGrid Computing
Grid Computing
Ā 
Challenges and Issues of Next Cloud Computing Platforms
Challenges and Issues of Next Cloud Computing PlatformsChallenges and Issues of Next Cloud Computing Platforms
Challenges and Issues of Next Cloud Computing Platforms
Ā 
Grid computing
Grid computingGrid computing
Grid computing
Ā 
Grid computing notes
Grid computing notesGrid computing notes
Grid computing notes
Ā 
Predictive Maintenance Using Recurrent Neural Networks
Predictive Maintenance Using Recurrent Neural NetworksPredictive Maintenance Using Recurrent Neural Networks
Predictive Maintenance Using Recurrent Neural Networks
Ā 
Grid computing by vaishali sahare [katkar]
Grid computing by vaishali sahare [katkar]Grid computing by vaishali sahare [katkar]
Grid computing by vaishali sahare [katkar]
Ā 
Grid computing
Grid computingGrid computing
Grid computing
Ā 
Grid computing
Grid computingGrid computing
Grid computing
Ā 
Grid computing
Grid computingGrid computing
Grid computing
Ā 
ISWC 2016 Tutorial: Semantic Web of Things M3 framework & FIESTA-IoT EU project
ISWC 2016 Tutorial: Semantic Web of Things  M3 framework & FIESTA-IoT EU projectISWC 2016 Tutorial: Semantic Web of Things  M3 framework & FIESTA-IoT EU project
ISWC 2016 Tutorial: Semantic Web of Things M3 framework & FIESTA-IoT EU project
Ā 
Grid computing ppt 2003(done)
Grid computing ppt 2003(done)Grid computing ppt 2003(done)
Grid computing ppt 2003(done)
Ā 

Destaque

The internet of things is for people
The internet of things is for peopleThe internet of things is for people
The internet of things is for peopleyiibu
Ā 
Will Internet of Things change the world or it is yet another buzzword?
Will Internet of Things change the world or it is yet another buzzword?Will Internet of Things change the world or it is yet another buzzword?
Will Internet of Things change the world or it is yet another buzzword?Philip Yankov
Ā 
An Introduction to the Internet of Things
An Introduction to the Internet of ThingsAn Introduction to the Internet of Things
An Introduction to the Internet of ThingsMohammad Babaee
Ā 
Internet of Things and its applications
Internet of Things and its applicationsInternet of Things and its applications
Internet of Things and its applicationsPasquale Puzio
Ā 
Adam - Adaptive Monitoring in 5min
Adam - Adaptive Monitoring in 5minAdam - Adaptive Monitoring in 5min
Adam - Adaptive Monitoring in 5minDemetris Trihinas
Ā 
Estimation of types of alkalinity in waste water samples
Estimation of types of alkalinity in waste water samplesEstimation of types of alkalinity in waste water samples
Estimation of types of alkalinity in waste water samplesRudradityo Saha
Ā 
Tracking SLAs In Cloud
Tracking SLAs In CloudTracking SLAs In Cloud
Tracking SLAs In CloudSatish Agrawal
Ā 
Cnam2015 m2 m -iot - course 2 - warming - v(0.2)
Cnam2015   m2 m -iot - course 2 - warming - v(0.2)Cnam2015   m2 m -iot - course 2 - warming - v(0.2)
Cnam2015 m2 m -iot - course 2 - warming - v(0.2)Thierry Lestable
Ā 
Sigfox presentation [Sep 2016]
Sigfox presentation [Sep 2016]Sigfox presentation [Sep 2016]
Sigfox presentation [Sep 2016]Nicolas Lesconnec
Ā 
Internet of Things Innovations & Megatrends Update 12/14/16
Internet of Things Innovations & Megatrends Update 12/14/16Internet of Things Innovations & Megatrends Update 12/14/16
Internet of Things Innovations & Megatrends Update 12/14/16Mark Goldstein
Ā 
The Internet of Things (and the myth of the ā€œSmartā€ Fridge)
The Internet of Things (and the myth of the ā€œSmartā€ Fridge)The Internet of Things (and the myth of the ā€œSmartā€ Fridge)
The Internet of Things (and the myth of the ā€œSmartā€ Fridge)Avi Itzkovitch
Ā 
Internet of things
Internet of thingsInternet of things
Internet of thingsAJITHKUMAR RAVI
Ā 
Iot based smart helmet used in mining industry
Iot based smart helmet used in mining industryIot based smart helmet used in mining industry
Iot based smart helmet used in mining industryarrss arf
Ā 
Water level indicator
Water level indicatorWater level indicator
Water level indicatorJyotsna Rani
Ā 
Internet of things executive overview
Internet of things executive overviewInternet of things executive overview
Internet of things executive overviewProsun Roy
Ā 

Destaque (20)

Internet of Things
Internet of Things Internet of Things
Internet of Things
Ā 
The internet of things is for people
The internet of things is for peopleThe internet of things is for people
The internet of things is for people
Ā 
Will Internet of Things change the world or it is yet another buzzword?
Will Internet of Things change the world or it is yet another buzzword?Will Internet of Things change the world or it is yet another buzzword?
Will Internet of Things change the world or it is yet another buzzword?
Ā 
An Introduction to the Internet of Things
An Introduction to the Internet of ThingsAn Introduction to the Internet of Things
An Introduction to the Internet of Things
Ā 
Internet of Things and its applications
Internet of Things and its applicationsInternet of Things and its applications
Internet of Things and its applications
Ā 
Adam - Adaptive Monitoring in 5min
Adam - Adaptive Monitoring in 5minAdam - Adaptive Monitoring in 5min
Adam - Adaptive Monitoring in 5min
Ā 
Š­ŠŗсŠæŠµŃ€Ń‚Š½Ń‹Šµ сŠøстŠµŠ¼Ń‹: Š»ŠµŠŗцŠøя ā„–1
Š­ŠŗсŠæŠµŃ€Ń‚Š½Ń‹Šµ сŠøстŠµŠ¼Ń‹: Š»ŠµŠŗцŠøя ā„–1Š­ŠŗсŠæŠµŃ€Ń‚Š½Ń‹Šµ сŠøстŠµŠ¼Ń‹: Š»ŠµŠŗцŠøя ā„–1
Š­ŠŗсŠæŠµŃ€Ń‚Š½Ń‹Šµ сŠøстŠµŠ¼Ń‹: Š»ŠµŠŗцŠøя ā„–1
Ā 
Estimation of types of alkalinity in waste water samples
Estimation of types of alkalinity in waste water samplesEstimation of types of alkalinity in waste water samples
Estimation of types of alkalinity in waste water samples
Ā 
KL10TCH.School : Š’Š²ŠµŠ“ŠµŠ½ŠøŠµ Š² Linked Data Šø Semantic Web
KL10TCH.School : Š’Š²ŠµŠ“ŠµŠ½ŠøŠµ Š² Linked Data Šø Semantic WebKL10TCH.School : Š’Š²ŠµŠ“ŠµŠ½ŠøŠµ Š² Linked Data Šø Semantic Web
KL10TCH.School : Š’Š²ŠµŠ“ŠµŠ½ŠøŠµ Š² Linked Data Šø Semantic Web
Ā 
SemIoT (Semantic technologies for Internet of Things) - Project Overview
SemIoT (Semantic technologies for Internet of Things) - Project OverviewSemIoT (Semantic technologies for Internet of Things) - Project Overview
SemIoT (Semantic technologies for Internet of Things) - Project Overview
Ā 
Tracking SLAs In Cloud
Tracking SLAs In CloudTracking SLAs In Cloud
Tracking SLAs In Cloud
Ā 
Cnam2015 m2 m -iot - course 2 - warming - v(0.2)
Cnam2015   m2 m -iot - course 2 - warming - v(0.2)Cnam2015   m2 m -iot - course 2 - warming - v(0.2)
Cnam2015 m2 m -iot - course 2 - warming - v(0.2)
Ā 
Sigfox presentation [Sep 2016]
Sigfox presentation [Sep 2016]Sigfox presentation [Sep 2016]
Sigfox presentation [Sep 2016]
Ā 
smart glasses
smart glassessmart glasses
smart glasses
Ā 
Internet of Things Innovations & Megatrends Update 12/14/16
Internet of Things Innovations & Megatrends Update 12/14/16Internet of Things Innovations & Megatrends Update 12/14/16
Internet of Things Innovations & Megatrends Update 12/14/16
Ā 
The Internet of Things (and the myth of the ā€œSmartā€ Fridge)
The Internet of Things (and the myth of the ā€œSmartā€ Fridge)The Internet of Things (and the myth of the ā€œSmartā€ Fridge)
The Internet of Things (and the myth of the ā€œSmartā€ Fridge)
Ā 
Internet of things
Internet of thingsInternet of things
Internet of things
Ā 
Iot based smart helmet used in mining industry
Iot based smart helmet used in mining industryIot based smart helmet used in mining industry
Iot based smart helmet used in mining industry
Ā 
Water level indicator
Water level indicatorWater level indicator
Water level indicator
Ā 
Internet of things executive overview
Internet of things executive overviewInternet of things executive overview
Internet of things executive overview
Ā 

Semelhante a Low-Cost Approximate and Adaptive Monitoring Techniques for the Internet of Things

Low-Cost Approximate and Adaptive Techniques for the Internet of Things
Low-Cost Approximate and Adaptive Techniques for the Internet of ThingsLow-Cost Approximate and Adaptive Techniques for the Internet of Things
Low-Cost Approximate and Adaptive Techniques for the Internet of ThingsDemetris Trihinas
Ā 
CROM Digital Twins and IoT
CROM Digital Twins and IoTCROM Digital Twins and IoT
CROM Digital Twins and IoTJuan C. Vasquez
Ā 
Making Actionable Decisions at the Network's Edge
Making Actionable Decisions at the Network's EdgeMaking Actionable Decisions at the Network's Edge
Making Actionable Decisions at the Network's EdgeCognizant
Ā 
Semantics in Sensor Networks
Semantics in Sensor NetworksSemantics in Sensor Networks
Semantics in Sensor NetworksOscar Corcho
Ā 
StreamSight - Query-Driven Descriptive Analytics for IoT and Edge Computing
StreamSight - Query-Driven Descriptive Analytics for IoT and Edge ComputingStreamSight - Query-Driven Descriptive Analytics for IoT and Edge Computing
StreamSight - Query-Driven Descriptive Analytics for IoT and Edge ComputingDemetris Trihinas
Ā 
StreamSight: A Query-Driven Framework Extending Streaming IoT Analytics to th...
StreamSight: A Query-Driven Framework Extending Streaming IoT Analytics to th...StreamSight: A Query-Driven Framework Extending Streaming IoT Analytics to th...
StreamSight: A Query-Driven Framework Extending Streaming IoT Analytics to th...Demetris Trihinas
Ā 
The Internet of Things (IoT) and its evolution
The Internet of Things (IoT) and its evolutionThe Internet of Things (IoT) and its evolution
The Internet of Things (IoT) and its evolutionSathvik N Prasad
Ā 
A Smart ITS based Sensor Network for Transport System with Integration of Io...
A Smart ITS based Sensor Network for Transport System with Integration of  Io...A Smart ITS based Sensor Network for Transport System with Integration of  Io...
A Smart ITS based Sensor Network for Transport System with Integration of Io...IRJET Journal
Ā 
The Internet of Things: What's next?
The Internet of Things: What's next? The Internet of Things: What's next?
The Internet of Things: What's next? PayamBarnaghi
Ā 
How to make data more usable on the Internet of Things
How to make data more usable on the Internet of ThingsHow to make data more usable on the Internet of Things
How to make data more usable on the Internet of ThingsPayamBarnaghi
Ā 
Data stream mining techniques: a review
Data stream mining techniques: a reviewData stream mining techniques: a review
Data stream mining techniques: a reviewTELKOMNIKA JOURNAL
Ā 
Data processing in Cyber-Physical Systems
Data processing in Cyber-Physical SystemsData processing in Cyber-Physical Systems
Data processing in Cyber-Physical SystemsBob Marcus
Ā 
Data Processing and Semantics for Advanced Internet of Things (IoT) Applicati...
Data Processing and Semantics for Advanced Internet of Things (IoT) Applicati...Data Processing and Semantics for Advanced Internet of Things (IoT) Applicati...
Data Processing and Semantics for Advanced Internet of Things (IoT) Applicati...Artificial Intelligence Institute at UofSC
Ā 
Low-Cost Adaptive Monitoring Techniques for the Internet of Things
Low-Cost Adaptive Monitoring Techniques for the Internet of ThingsLow-Cost Adaptive Monitoring Techniques for the Internet of Things
Low-Cost Adaptive Monitoring Techniques for the Internet of ThingsDemetris Trihinas
Ā 
Term Paper Presentation
Term Paper PresentationTerm Paper Presentation
Term Paper PresentationShubham Singh
Ā 
Roberto minerva 20181130
Roberto minerva 20181130  Roberto minerva 20181130
Roberto minerva 20181130 Roberto Minerva
Ā 
Data Streaming in Big Data Analysis
Data Streaming in Big Data AnalysisData Streaming in Big Data Analysis
Data Streaming in Big Data AnalysisVincenzo Gulisano
Ā 
sensors-22-00196-v2.pdf
sensors-22-00196-v2.pdfsensors-22-00196-v2.pdf
sensors-22-00196-v2.pdfAsiyaKhan63
Ā 

Semelhante a Low-Cost Approximate and Adaptive Monitoring Techniques for the Internet of Things (20)

Low-Cost Approximate and Adaptive Techniques for the Internet of Things
Low-Cost Approximate and Adaptive Techniques for the Internet of ThingsLow-Cost Approximate and Adaptive Techniques for the Internet of Things
Low-Cost Approximate and Adaptive Techniques for the Internet of Things
Ā 
CROM Digital Twins and IoT
CROM Digital Twins and IoTCROM Digital Twins and IoT
CROM Digital Twins and IoT
Ā 
Making Actionable Decisions at the Network's Edge
Making Actionable Decisions at the Network's EdgeMaking Actionable Decisions at the Network's Edge
Making Actionable Decisions at the Network's Edge
Ā 
Semantics in Sensor Networks
Semantics in Sensor NetworksSemantics in Sensor Networks
Semantics in Sensor Networks
Ā 
StreamSight - Query-Driven Descriptive Analytics for IoT and Edge Computing
StreamSight - Query-Driven Descriptive Analytics for IoT and Edge ComputingStreamSight - Query-Driven Descriptive Analytics for IoT and Edge Computing
StreamSight - Query-Driven Descriptive Analytics for IoT and Edge Computing
Ā 
StreamSight: A Query-Driven Framework Extending Streaming IoT Analytics to th...
StreamSight: A Query-Driven Framework Extending Streaming IoT Analytics to th...StreamSight: A Query-Driven Framework Extending Streaming IoT Analytics to th...
StreamSight: A Query-Driven Framework Extending Streaming IoT Analytics to th...
Ā 
1. GRID COMPUTING
1. GRID COMPUTING1. GRID COMPUTING
1. GRID COMPUTING
Ā 
The Internet of Things (IoT) and its evolution
The Internet of Things (IoT) and its evolutionThe Internet of Things (IoT) and its evolution
The Internet of Things (IoT) and its evolution
Ā 
A Smart ITS based Sensor Network for Transport System with Integration of Io...
A Smart ITS based Sensor Network for Transport System with Integration of  Io...A Smart ITS based Sensor Network for Transport System with Integration of  Io...
A Smart ITS based Sensor Network for Transport System with Integration of Io...
Ā 
The Internet of Things: What's next?
The Internet of Things: What's next? The Internet of Things: What's next?
The Internet of Things: What's next?
Ā 
How to make data more usable on the Internet of Things
How to make data more usable on the Internet of ThingsHow to make data more usable on the Internet of Things
How to make data more usable on the Internet of Things
Ā 
Data stream mining techniques: a review
Data stream mining techniques: a reviewData stream mining techniques: a review
Data stream mining techniques: a review
Ā 
Data processing in Cyber-Physical Systems
Data processing in Cyber-Physical SystemsData processing in Cyber-Physical Systems
Data processing in Cyber-Physical Systems
Ā 
Data Processing and Semantics for Advanced Internet of Things (IoT) Applicati...
Data Processing and Semantics for Advanced Internet of Things (IoT) Applicati...Data Processing and Semantics for Advanced Internet of Things (IoT) Applicati...
Data Processing and Semantics for Advanced Internet of Things (IoT) Applicati...
Ā 
Low-Cost Adaptive Monitoring Techniques for the Internet of Things
Low-Cost Adaptive Monitoring Techniques for the Internet of ThingsLow-Cost Adaptive Monitoring Techniques for the Internet of Things
Low-Cost Adaptive Monitoring Techniques for the Internet of Things
Ā 
Big Data and IOT
Big Data and IOTBig Data and IOT
Big Data and IOT
Ā 
Term Paper Presentation
Term Paper PresentationTerm Paper Presentation
Term Paper Presentation
Ā 
Roberto minerva 20181130
Roberto minerva 20181130  Roberto minerva 20181130
Roberto minerva 20181130
Ā 
Data Streaming in Big Data Analysis
Data Streaming in Big Data AnalysisData Streaming in Big Data Analysis
Data Streaming in Big Data Analysis
Ā 
sensors-22-00196-v2.pdf
sensors-22-00196-v2.pdfsensors-22-00196-v2.pdf
sensors-22-00196-v2.pdf
Ā 

Mais de Demetris Trihinas

Rapidly Testing ML-Driven Drone Applications - The FlockAI Framework
Rapidly Testing ML-Driven Drone Applications - The FlockAI FrameworkRapidly Testing ML-Driven Drone Applications - The FlockAI Framework
Rapidly Testing ML-Driven Drone Applications - The FlockAI FrameworkDemetris Trihinas
Ā 
Towards Energy and Carbon Footprint and Testing for AI-driven IoT Services
Towards Energy and Carbon Footprint and Testing for AI-driven IoT ServicesTowards Energy and Carbon Footprint and Testing for AI-driven IoT Services
Towards Energy and Carbon Footprint and Testing for AI-driven IoT ServicesDemetris Trihinas
Ā 
Composable Energy Modeling for ML-Driven Drone Applications
Composable Energy Modeling for ML-Driven Drone ApplicationsComposable Energy Modeling for ML-Driven Drone Applications
Composable Energy Modeling for ML-Driven Drone ApplicationsDemetris Trihinas
Ā 
Telling a Story ā€“ or Even Propaganda ā€“ Through Data Visualization
Telling a Story ā€“ or Even Propaganda ā€“ Through Data VisualizationTelling a Story ā€“ or Even Propaganda ā€“ Through Data Visualization
Telling a Story ā€“ or Even Propaganda ā€“ Through Data VisualizationDemetris Trihinas
Ā 
Machine Learning Introduction
Machine Learning IntroductionMachine Learning Introduction
Machine Learning IntroductionDemetris Trihinas
Ā 
Ī‘Ļ€ĪµĪ¹ĪŗĻŒĪ½Ī¹ĻƒĪ· ĪŗĪ±Ī¹ Ī‘Ī»Ī»Ī·Ī»ĪµĻ€ĪÆĪ“ĻĪ±ĻƒĪ· Ī”ĪµĪ“ĪæĪ¼Ī­Ī½Ļ‰Ī½ ĪœĪµĪ³Ī¬Ī»ĪæĻ… ĪŒĪ³ĪŗĪæĻ… Ī¼Īµ Ī”Ī¹Ī±Ī“ĻĪ±ĻƒĻ„Ī¹ĪŗĪæĻĻ‚ Ī§Ī¬ĻĻ„ĪµĻ‚
Ī‘Ļ€ĪµĪ¹ĪŗĻŒĪ½Ī¹ĻƒĪ· ĪŗĪ±Ī¹ Ī‘Ī»Ī»Ī·Ī»ĪµĻ€ĪÆĪ“ĻĪ±ĻƒĪ· Ī”ĪµĪ“ĪæĪ¼Ī­Ī½Ļ‰Ī½ ĪœĪµĪ³Ī¬Ī»ĪæĻ… ĪŒĪ³ĪŗĪæĻ… Ī¼Īµ Ī”Ī¹Ī±Ī“ĻĪ±ĻƒĻ„Ī¹ĪŗĪæĻĻ‚ Ī§Ī¬ĻĻ„ĪµĻ‚Ī‘Ļ€ĪµĪ¹ĪŗĻŒĪ½Ī¹ĻƒĪ· ĪŗĪ±Ī¹ Ī‘Ī»Ī»Ī·Ī»ĪµĻ€ĪÆĪ“ĻĪ±ĻƒĪ· Ī”ĪµĪ“ĪæĪ¼Ī­Ī½Ļ‰Ī½ ĪœĪµĪ³Ī¬Ī»ĪæĻ… ĪŒĪ³ĪŗĪæĻ… Ī¼Īµ Ī”Ī¹Ī±Ī“ĻĪ±ĻƒĻ„Ī¹ĪŗĪæĻĻ‚ Ī§Ī¬ĻĻ„ĪµĻ‚
Ī‘Ļ€ĪµĪ¹ĪŗĻŒĪ½Ī¹ĻƒĪ· ĪŗĪ±Ī¹ Ī‘Ī»Ī»Ī·Ī»ĪµĻ€ĪÆĪ“ĻĪ±ĻƒĪ· Ī”ĪµĪ“ĪæĪ¼Ī­Ī½Ļ‰Ī½ ĪœĪµĪ³Ī¬Ī»ĪæĻ… ĪŒĪ³ĪŗĪæĻ… Ī¼Īµ Ī”Ī¹Ī±Ī“ĻĪ±ĻƒĻ„Ī¹ĪŗĪæĻĻ‚ Ī§Ī¬ĻĻ„ĪµĻ‚Demetris Trihinas
Ā 
The Data Science Process: From Mining Raw Data to Story Visualization
The Data Science Process: From Mining Raw Data to Story VisualizationThe Data Science Process: From Mining Raw Data to Story Visualization
The Data Science Process: From Mining Raw Data to Story VisualizationDemetris Trihinas
Ā 
From Mining Raw Data to Story Visualization
From Mining Raw Data to Story VisualizationFrom Mining Raw Data to Story Visualization
From Mining Raw Data to Story VisualizationDemetris Trihinas
Ā 
Designing Scalable and Secure Microservices by Embracing DevOps-as-a-Service ...
Designing Scalable and Secure Microservices by Embracing DevOps-as-a-Service ...Designing Scalable and Secure Microservices by Embracing DevOps-as-a-Service ...
Designing Scalable and Secure Microservices by Embracing DevOps-as-a-Service ...Demetris Trihinas
Ā 
AdaM: an Adaptive Monitoring Framework for Sampling and Filtering on IoT Devices
AdaM: an Adaptive Monitoring Framework for Sampling and Filtering on IoT DevicesAdaM: an Adaptive Monitoring Framework for Sampling and Filtering on IoT Devices
AdaM: an Adaptive Monitoring Framework for Sampling and Filtering on IoT DevicesDemetris Trihinas
Ā 
Cloud Elasticity and the CELAR Project
Cloud Elasticity and the CELAR ProjectCloud Elasticity and the CELAR Project
Cloud Elasticity and the CELAR ProjectDemetris Trihinas
Ā 
[ccgrid2014] JCatascopia: Monitoring Elastically Adaptive Applications in the...
[ccgrid2014] JCatascopia: Monitoring Elastically Adaptive Applications in the...[ccgrid2014] JCatascopia: Monitoring Elastically Adaptive Applications in the...
[ccgrid2014] JCatascopia: Monitoring Elastically Adaptive Applications in the...Demetris Trihinas
Ā 
[SummerSoc 2014] Monitoring Elastic Cloud Services
[SummerSoc 2014] Monitoring Elastic Cloud Services[SummerSoc 2014] Monitoring Elastic Cloud Services
[SummerSoc 2014] Monitoring Elastic Cloud ServicesDemetris Trihinas
Ā 

Mais de Demetris Trihinas (14)

Rapidly Testing ML-Driven Drone Applications - The FlockAI Framework
Rapidly Testing ML-Driven Drone Applications - The FlockAI FrameworkRapidly Testing ML-Driven Drone Applications - The FlockAI Framework
Rapidly Testing ML-Driven Drone Applications - The FlockAI Framework
Ā 
Towards Energy and Carbon Footprint and Testing for AI-driven IoT Services
Towards Energy and Carbon Footprint and Testing for AI-driven IoT ServicesTowards Energy and Carbon Footprint and Testing for AI-driven IoT Services
Towards Energy and Carbon Footprint and Testing for AI-driven IoT Services
Ā 
Composable Energy Modeling for ML-Driven Drone Applications
Composable Energy Modeling for ML-Driven Drone ApplicationsComposable Energy Modeling for ML-Driven Drone Applications
Composable Energy Modeling for ML-Driven Drone Applications
Ā 
Telling a Story ā€“ or Even Propaganda ā€“ Through Data Visualization
Telling a Story ā€“ or Even Propaganda ā€“ Through Data VisualizationTelling a Story ā€“ or Even Propaganda ā€“ Through Data Visualization
Telling a Story ā€“ or Even Propaganda ā€“ Through Data Visualization
Ā 
Machine Learning Introduction
Machine Learning IntroductionMachine Learning Introduction
Machine Learning Introduction
Ā 
Ī‘Ļ€ĪµĪ¹ĪŗĻŒĪ½Ī¹ĻƒĪ· ĪŗĪ±Ī¹ Ī‘Ī»Ī»Ī·Ī»ĪµĻ€ĪÆĪ“ĻĪ±ĻƒĪ· Ī”ĪµĪ“ĪæĪ¼Ī­Ī½Ļ‰Ī½ ĪœĪµĪ³Ī¬Ī»ĪæĻ… ĪŒĪ³ĪŗĪæĻ… Ī¼Īµ Ī”Ī¹Ī±Ī“ĻĪ±ĻƒĻ„Ī¹ĪŗĪæĻĻ‚ Ī§Ī¬ĻĻ„ĪµĻ‚
Ī‘Ļ€ĪµĪ¹ĪŗĻŒĪ½Ī¹ĻƒĪ· ĪŗĪ±Ī¹ Ī‘Ī»Ī»Ī·Ī»ĪµĻ€ĪÆĪ“ĻĪ±ĻƒĪ· Ī”ĪµĪ“ĪæĪ¼Ī­Ī½Ļ‰Ī½ ĪœĪµĪ³Ī¬Ī»ĪæĻ… ĪŒĪ³ĪŗĪæĻ… Ī¼Īµ Ī”Ī¹Ī±Ī“ĻĪ±ĻƒĻ„Ī¹ĪŗĪæĻĻ‚ Ī§Ī¬ĻĻ„ĪµĻ‚Ī‘Ļ€ĪµĪ¹ĪŗĻŒĪ½Ī¹ĻƒĪ· ĪŗĪ±Ī¹ Ī‘Ī»Ī»Ī·Ī»ĪµĻ€ĪÆĪ“ĻĪ±ĻƒĪ· Ī”ĪµĪ“ĪæĪ¼Ī­Ī½Ļ‰Ī½ ĪœĪµĪ³Ī¬Ī»ĪæĻ… ĪŒĪ³ĪŗĪæĻ… Ī¼Īµ Ī”Ī¹Ī±Ī“ĻĪ±ĻƒĻ„Ī¹ĪŗĪæĻĻ‚ Ī§Ī¬ĻĻ„ĪµĻ‚
Ī‘Ļ€ĪµĪ¹ĪŗĻŒĪ½Ī¹ĻƒĪ· ĪŗĪ±Ī¹ Ī‘Ī»Ī»Ī·Ī»ĪµĻ€ĪÆĪ“ĻĪ±ĻƒĪ· Ī”ĪµĪ“ĪæĪ¼Ī­Ī½Ļ‰Ī½ ĪœĪµĪ³Ī¬Ī»ĪæĻ… ĪŒĪ³ĪŗĪæĻ… Ī¼Īµ Ī”Ī¹Ī±Ī“ĻĪ±ĻƒĻ„Ī¹ĪŗĪæĻĻ‚ Ī§Ī¬ĻĻ„ĪµĻ‚
Ā 
The Data Science Process: From Mining Raw Data to Story Visualization
The Data Science Process: From Mining Raw Data to Story VisualizationThe Data Science Process: From Mining Raw Data to Story Visualization
The Data Science Process: From Mining Raw Data to Story Visualization
Ā 
From Mining Raw Data to Story Visualization
From Mining Raw Data to Story VisualizationFrom Mining Raw Data to Story Visualization
From Mining Raw Data to Story Visualization
Ā 
Designing Scalable and Secure Microservices by Embracing DevOps-as-a-Service ...
Designing Scalable and Secure Microservices by Embracing DevOps-as-a-Service ...Designing Scalable and Secure Microservices by Embracing DevOps-as-a-Service ...
Designing Scalable and Secure Microservices by Embracing DevOps-as-a-Service ...
Ā 
AdaM: an Adaptive Monitoring Framework for Sampling and Filtering on IoT Devices
AdaM: an Adaptive Monitoring Framework for Sampling and Filtering on IoT DevicesAdaM: an Adaptive Monitoring Framework for Sampling and Filtering on IoT Devices
AdaM: an Adaptive Monitoring Framework for Sampling and Filtering on IoT Devices
Ā 
Find A Project
Find A ProjectFind A Project
Find A Project
Ā 
Cloud Elasticity and the CELAR Project
Cloud Elasticity and the CELAR ProjectCloud Elasticity and the CELAR Project
Cloud Elasticity and the CELAR Project
Ā 
[ccgrid2014] JCatascopia: Monitoring Elastically Adaptive Applications in the...
[ccgrid2014] JCatascopia: Monitoring Elastically Adaptive Applications in the...[ccgrid2014] JCatascopia: Monitoring Elastically Adaptive Applications in the...
[ccgrid2014] JCatascopia: Monitoring Elastically Adaptive Applications in the...
Ā 
[SummerSoc 2014] Monitoring Elastic Cloud Services
[SummerSoc 2014] Monitoring Elastic Cloud Services[SummerSoc 2014] Monitoring Elastic Cloud Services
[SummerSoc 2014] Monitoring Elastic Cloud Services
Ā 

ƚltimo

Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxKatpro Technologies
Ā 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processorsdebabhi2
Ā 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024Results
Ā 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...Neo4j
Ā 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
Ā 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfsudhanshuwaghmare1
Ā 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
Ā 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...Martijn de Jong
Ā 
šŸ¬ The future of MySQL is Postgres šŸ˜
šŸ¬  The future of MySQL is Postgres   šŸ˜šŸ¬  The future of MySQL is Postgres   šŸ˜
šŸ¬ The future of MySQL is Postgres šŸ˜RTylerCroy
Ā 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
Ā 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
Ā 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Enterprise Knowledge
Ā 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
Ā 
Scaling API-first ā€“ The story of a global engineering organization
Scaling API-first ā€“ The story of a global engineering organizationScaling API-first ā€“ The story of a global engineering organization
Scaling API-first ā€“ The story of a global engineering organizationRadu Cotescu
Ā 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?Igalia
Ā 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
Ā 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel AraĆŗjo
Ā 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...apidays
Ā 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessPixlogix Infotech
Ā 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEarley Information Science
Ā 

ƚltimo (20)

Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Ā 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
Ā 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
Ā 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Ā 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
Ā 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
Ā 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
Ā 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
Ā 
šŸ¬ The future of MySQL is Postgres šŸ˜
šŸ¬  The future of MySQL is Postgres   šŸ˜šŸ¬  The future of MySQL is Postgres   šŸ˜
šŸ¬ The future of MySQL is Postgres šŸ˜
Ā 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
Ā 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
Ā 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
Ā 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
Ā 
Scaling API-first ā€“ The story of a global engineering organization
Scaling API-first ā€“ The story of a global engineering organizationScaling API-first ā€“ The story of a global engineering organization
Scaling API-first ā€“ The story of a global engineering organization
Ā 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
Ā 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
Ā 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Ā 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Ā 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your Business
Ā 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Ā 

Low-Cost Approximate and Adaptive Monitoring Techniques for the Internet of Things

  • 1. 3/13/2017 1Demetris Trihinas 1 Low-Cost Approximate & Adaptive Monitoring Techniques for the Internet of Things Demetris Trihinas trihinas@cs.ucy.ac.cy PhD Candidate Department of Computer Science University of Cyprus
  • 2. 3/13/2017 2Demetris Trihinas 2 Cloud Computing Big Data Systems and Analytics Internet Computing and IoT Online Social Networks Activity Areas Current EU-Funded Projects http://linc.ucy.ac.cy/
  • 3. 3/13/2017 3Demetris Trihinas 3 The Internet of Things The physical world is now becoming an information system Exchanging continuous data streams with other network- enabled devices, systems and humansā€¦ Physical (battery-powered) and network-enabled devices with smart processing capabilitiesā€¦
  • 4. 3/13/2017 4Demetris Trihinas 4 Why is IoT Just Now Becoming a Reality? Nano Bio-Sensors and Printed Electronics Communication Protocols Data-Mining Learning Algorithms - Distributed closed neighbors - Outlier/event detection - Pattern matching - Stream processing - Learning algorithms - ā€¦. ā€œThe Emerging Internet of Things Market Perspective: A Surveyā€, Perera et al., IEEE Trans. Computing, 2015 ā€œInternet of Things (IoT): A Vision, Architectural Elements, and Future Directionsā€, J. Gubbi and R. Buyya, FGCS, 2013 The Building Blocks of the Internet of Things
  • 5. 3/13/2017 5Demetris Trihinas 5 IoT in Agriculture Precision Farming Detect crop condition and spread nutrients to parts of field with need Cattle as a Service Detect optimum breading period, monitor health and dairy protection ā€œConnected Cattle: Wearables are Changing the Dairy Industryā€, Kathy Pretz, IEEE Magazine: the Institute, May 2016.
  • 6. 3/13/2017 6Demetris Trihinas 6 IoT in Medicine and Health Tracking Precision Medicine Pill-shaped cameras traversing human body ā€œFitbit data led doctors to shock a patientā€™s heartā€, Steven Dent, Engadget, Apr 2016. Wearables Biosignal and activity tracking
  • 7. 3/13/2017 7Demetris Trihinas 7 IoT in Intelligent Transportation Systems Smart Traffic Flow In Toronto travel time was reduced by 26% Smart Parking In California carbon emissions were reduced by 730 tons per 15 block radius ā€œCan smart traffic lights ease Torontoā€™s road congestion?ā€, Joseph Hall, Toronto Star, Jun 2014.
  • 8. 3/13/2017 8Demetris Trihinas 8 The ā€œBig Dataā€ in IoT [IDC, Jun 2014] [Cisco, Apr 2011] IoT Monitoring Data 2% of digital universe in 2012 Projections for >12% in 2020 [Gartner, Mar 2015] IP Traffic 1.6ZB in 2018, 57% from IoT Devices [Cisco, Apr 2014]
  • 9. 3/13/2017 9Demetris Trihinas 9 Big Data and the Cloud
  • 10. 3/13/2017 10Demetris Trihinas 10 IoT and Cloud Computing Driving Cloud Adoption for IoT Developers - Simplified resource management - low capital expenses to get started [IoT Developer Survey, Eclipse Foundation, 2016] 67% of production-ready IoT services reside in the cloud [State of IoT, Embarcadero, 2016]
  • 11. 3/13/2017 11Demetris Trihinas 11 Soā€¦is Cloud Monitoring Really Ready for the Internet of Things?
  • 12. 3/13/2017 12Demetris Trihinas 12 Monitoring Topologies Scalability and Single points of failure Monitoring servers closer to the root are overloaded with relay costs preventing scalability ā€œJCatascopia: Monitoring Elastically Adaptive Applications in the Cloudā€, D. Trihinas, G. Pallis and M. D. Dikaiakos, IEEE/ACM CCGrid 2014
  • 13. 3/13/2017 13Demetris Trihinas 13 Monitoring as a Service (MaaS) ā€¢ Eases management of the monitoring infrastructure ā€¢ Service providers enable multi-tenant and scalable monitoring over the internet ā€¢ Pay-as-you-use business model (although usually hourly) Monitoring libraries for metric collection from popular languages and frameworks REST API to insert/extract monitoring data Shared data warehouse with services to query, process and view analytic insights custom app-specific metrics managed by cloud provider
  • 14. 3/13/2017 14Demetris Trihinas 14 Augmenting IoT with the Cloud ā€œ app The Harsh Reality Network latencies, bandwidth limitations, pricing (in/out cloud traffic is billed), constant location awareness Datacenter Interesting Articles ā€œThe cloud is not enough: Saving IoT from the cloudā€, B. Zhang et al., Usenix HotCloud 2015 ā€œTaking the internet to the next physical levelā€, V. Cerf and M. Senges, IEEE Computer, 2016 The IoT Developer Perspective
  • 15. 3/13/2017 15Demetris Trihinas 15 The Cost of Monitoring Based on AWS pricing scheme (May 2015) for a 3-tier (video) streaming application $0.12/GB network traffic ā€œMonitoring Elastically Adaptive Multi-Cloud Servicesā€, D. Trihinas, G. Pallis and M.D. Dikaiakos, IEEE TCC 2016
  • 16. 3/13/2017 16Demetris Trihinas 16 P2P Monitoring ā€¢ Monitored elements form Gossip Overlay Network to propagate monitoring info -> ease coordination and query computation ā€¢ Gossip denotes periodic and pairwise propagation of the current state between monitored peers ā€œMonitoring Elastically Adaptive Multi-Cloud Servicesā€, D. Trihinas, G. Pallis and M.D. Dikaiakos, IEEE TCC 2016 ā€œSelf managing monitoring for highly elastic large scale cloud deploymentsā€, S. Ward et al., ACM DIDC 2014
  • 17. 3/13/2017 17Demetris Trihinas 17 Edge Computing Init Sense Act Sleep Receive Decide Transmit A term coined to reflect data processing and decision-making on ā€œsmartā€ devices that sit at the edge of IoT networks Init Sense Transmit Sleep Interesting Articles ā€œThe promise of edge computingā€, W. Shi and S. Dustdar, IEEE Computer, 2016 ā€œThe swarm at the edge of the cloudā€, E. Lee et al., IEEE Design & Test, Jun 2014 Simple Sensing Edge Computing <10ms Big Data Traffic >100ms ProcessingNo Processing
  • 18. 3/13/2017 18Demetris Trihinas 18 Augmenting IoT with the Cloud Challenge 1 ā€“ Taming data volume and data velocity with limited processing and network capabilities in the IoT/Edge realm Challenge 2 - IoT devices are usually battery-powered which means intense processing leads to less battery-life Processing and data dissemination are the main energy drains in embedded and mobile devices Interesting Article ā€œPractical data prediction for real-world wireless sensor networksā€, U. Raza and T. Palpanas, IEEE TKDE, 2015
  • 19. 3/13/2017 19Demetris Trihinas 19 Augmenting IoT with the Cloud ā€œTaking the internet to the next physical levelā€, V. Cerf and M. Senges, IEEE Computer, 2016 Challenge 3 ā€“ Security and data privacy October 2016, DDOS Attack
  • 20. 3/13/2017 20Demetris Trihinas 20 Formulating the Challenges as Research Questions
  • 21. 3/13/2017 21Demetris Trihinas 21 Preliminaries ā€¢ A monitoring stream š‘€ = {š‘‘š‘–}š‘–=0 š‘› published by a monitoring source is a large stochastic sequence of i.i.d datapoints, denoted as š‘‘š‘– , where š‘– = 0, 1, ā€¦ , š‘› and š‘› ā†’ āˆž ā€¢ A datapoint š‘‘š‘– is a tuple š‘šš‘–š‘‘, š‘”š‘–, š‘£š‘– described by a unique identifier for the monitoring source š‘šš‘–š‘‘, a timestamp š‘”š‘– and a value š‘£š‘– ā€¢ A datapoint may have a set of other attributes (e.g., location) although for brevity we will omit other attributes without loss of generality š‘‘š‘– š‘‘š‘–+1 Monitoring Stream M
  • 22. 3/13/2017 22Demetris Trihinas 22 Periodic Sampling ā€¢ The process of triggering the collection mechanism of a monitored source every š‘‡ time units such that the š‘– š‘”ā„Ž datapoint is collected at time š‘”š‘– = š‘– āˆ™ š‘‡ š‘‘š‘– š‘‘š‘–+1 Metric Stream M sampled every T = 1s Compute resources and energy are wasted while generating large data volumes at a high velocity Metric Stream M sampled every T = 10s Sudden events and significant insights are missed
  • 23. 3/13/2017 23Demetris Trihinas 23 More processing (e.g., peak detection for hr monitoring) Driving Peripherals (e.g., accelerometer, leds) Interesting Articles ā€œEnergy-harvesting wearables for activity-aware servicesā€, S. Khalifa et al., IEEE Internet Computing, 2015 ā€œTaking the internet to the next physical levelā€, V. Cerf and M. Senges, IEEE Computer, 2016 Emulated behavior of wearable device on Raspberry Pi with ARM processor (1 core 32MHz, 128MB RAM) Periodic Sampling with Physical Sensing OS Monitoring
  • 24. 3/13/2017 24Demetris Trihinas 24 Periodic Sampling in an Energy Perspective Lithium battery (35mAh) State of Charge (SoC) projection for a wearable device with step, calorie and heartrate monitoring Interesting Article ā€œA universal state-of-charge algorithm for batteriesā€, B. Xiao et al., ACM DAC, 2010
  • 25. 3/13/2017 25Demetris Trihinas 25 Periodic Dissemination Monitoring Source Receiving Entity ā€¢ The process of triggering the network controller of a monitored source every š· time units such that the š‘– š‘”ā„Ž datapoint is disseminated at time š‘”š‘– = š‘– āˆ™ š· to interested receiving entities No data loss ā€¢ Aggregation-based policy dissemination: ā€¢ Withhold triggering dissemination until a window of š‘² datapoints is collected ā€¢ Withhold dissemination for fixed period of time and disseminate all datapoints collected until then
  • 26. 3/13/2017 26Demetris Trihinas 26 Periodic Dissemination Monitoring Cost šœ‡ š‘  + š›½š‘  āˆ™ Ļ‡ + Īµs per message cost per Ļ‡ byte cost datapoint d (t,v) -> message compression Interesting Article ā€œReal-Time Data Analytics in Sensor Networksā€, T. Palpanas, ACM Sensors, 2013 energy for state transitions Example Values (Mica2) šœ‡ š‘  = .645 mJ Īµs = .331 mJ š›½š‘  = .0144 mJ/byte
  • 27. 3/13/2017 27Demetris Trihinas 27 Low-Cost Approximate and Adaptive Monitoring
  • 28. 3/13/2017 28Demetris Trihinas 28 Preliminaries ā€¢ Approximate Techniques? ā€¢ Decision mechanisms based on estimation models capturing and predicting monitoring stream runtime evolution within certain accuracy guarantees ā€¢ Adaptive? ā€¢ Adapt monitoring source properties based on the predicaments of the estimation models ā€¢ Low-Cost? ā€¢ The costs (time, resources) of applying adaptive monitoring techniques are (much) less than leaving the monitoring process as is ā€¢ We are interested in techniques running in-place and inexpensively
  • 29. 3/13/2017 29Demetris Trihinas 29 Adaptive Sampling ā€¢ Dynamically adapt the sampling period š‘‡š‘– based on some estimation model šœŒ(š‘€), containing runtime information of the monitoring stream evolution ā€¢ How large of an adjustment is required, ā€œideallyā€, depends on some evaluation (distance) metric denoting the ability of the model to (correctly) estimate and follow metric stream evolution š‘‘š‘– š‘‘š‘–+1 š‘‡š‘–+1 Metric Stream š‘€ā€² dynamically sampledMetric Stream š‘€ with š‘‡ = 1š‘ 
  • 30. 3/13/2017 30Demetris Trihinas 30 Adaptive Sampling š‘‘š‘– š‘‘š‘–+1 š‘‡š‘–+1 Metric Stream š‘€ā€² dynamically sampledMetric Stream š‘€ with š‘‡ = 1š‘  Find the max š‘‡ āˆˆ [š‘‡ š‘šš‘–š‘›, š‘‡ š‘šš‘Žš‘„] to collect š‘‘š‘–+1 based on an estimation of the monitoring stream evolution šœŒ(š‘€), such that š‘€ā€² differs from š‘€ less than a user-defined imprecision value š›¾ (š‘‘š‘–š‘ š‘” < š›¾)
  • 31. 3/13/2017 31Demetris Trihinas 31 Adaptive Dissemination ā€¢ Dynamically adapt dissemination rate by applying approximation techniques to sensed datapoints to reduce communication overhead ā€¢ By suppressing consecutive datapoints from dissemination with ā€œlittleā€ change in their metric values ā€¢ How much is change is considered as ā€œlittleā€ depends on: ā€¢ The approximation technique ā€¢ Accuracy guarantees given by the user and denoting the probability (confidence Ī“) with which sensed datapoints are approximated
  • 32. 3/13/2017 32Demetris Trihinas 32 Adaptive Dissemination (Model-Based) ā€¢ Monitoring source maintains runtime estimation model šœŒ(š‘€) capturing monitoring stream evolution and variability ā€¢ At the š‘– š‘”ā„Ž time interval instead of metric values, the model is disseminated ā€¢ Receiving entities predict the IoT device state from given model assuming subsequent k-datapoints can be approximated di+k|i = f(šœŒ š‘€ , š‘‘š‘–) within the given guarantees Metric Stream š‘€ Approximated Metric Stream š‘€ā€²
  • 33. 3/13/2017 33Demetris Trihinas 33 Adaptive Dissemination (Model-Based) ā€¢ Monitoring source withholds further dissemination interacting with receiver only when shifts in monitoring stream value distribution render model as inconsistent with the actual IoT device state ā€¢ If model parameterization is inconsistent, at this point, it must be updated decision criteria? decision function? Metric Stream š‘€ Approximated Metric Stream š‘€ā€²
  • 34. 3/13/2017 34Demetris Trihinas 34 Metric Filtering ā€¢ A form of adaptive dissemination: suppress latest datapoint(s) dissemination if their values have not ā€œchangedā€ since last reported ā€¢ Metric stream suppression butā€¦ no model -> no forecasting ā€¢ Fixed filter ranges assume monitoring stream value distribution will not change in in the future (no guarantees any values will be filtered at all) if (curValue āˆˆ [prevValue ā€“ R, prevValue + R ]) filter(curValue) Interesting Article ā€œMonitoring, aggregation and filtering for efficient management of virtual networksā€, S. Clayman, IEEE NOMS, 2011
  • 35. 3/13/2017 35Demetris Trihinas 35 Adaptive Filtering ā€¢ Dynamically adjust the filter range š‘… based on the current variability of the monitoring stream, denoted as š‘ž(š‘€) Metric Stream š‘€ā€² with adaptive RangeMetric Stream š‘€ š‘‘š‘– š‘‘š‘–+1 š‘…š‘–+1 š‘…š‘– ā€¢ Find the max filter range š‘…š‘–+1 āˆˆ [š‘… š‘šš‘–š‘›, š‘… š‘šš‘Žš‘„] for š‘‘š‘–+1 such that š‘€ā€² differs from š‘€ less than a user-defined imprecision value š›¾ based on the variability of the metric stream š‘ž(š‘€)
  • 37. 3/13/2017 37Demetris Trihinas 37 The Adaptive Monitoring Framework Ī¼Processor Ī¼OS Memory Low-Cost Approximate Stream Estimation Adaptive Sampling Adaptive Filtering S E N S I N G U N I T metric updates A P I updated periodicity estimation confidence Adaptive Dissemination model updates & buffer content when shift detected updated filter range stream variability filtered metric stream AdaM N E T W O R K U N I T Model Base stream evolution and online stats Datapoint Buffer IoT Device metric stream compressed metric stream
  • 38. 3/13/2017 38Demetris Trihinas 38 Low-Cost Approximate Estimation Model ā€¢ Three (3) phase approach to approximate monitoring stream evolution and variability by exploiting knowledge (hidden) in the monitoring stream Step 1 Estimate Monitoring Stream Evolution and label datapoints as ā€œ(un)expectedā€ Step 2 Detect if Gradual Trends Exist in Monitoring Stream Evolution Step 3 Test if Seasonality Enrichment is Beneficial to Stream Evolution
  • 39. 3/13/2017 39Demetris Trihinas 39 Low-Cost Approximate Estimation Model Phase 1: Update runtime monitoring stream evolution šœŒ Īœ ā€¢ Probabilistic Exponential Weighted Moving Average (PEWMA) to estimate š‘£š‘–+1 from šœ‡š‘– and the standard deviation šœŽš‘–+1: š‘£š‘–+1 = šœ‡š‘– = š‘Ž āˆ™ šœ‡š‘–āˆ’1 + 1 āˆ’ š‘Ž š‘£š‘– ā€¢ Datapoints labelled as ā€œexpectedā€ if estimation lands in prediction intervals determined from user confidence guarantees or ā€œunexpectedā€ otherwise ā€¢ ā€œUnexpectedā€ datapoints are stored in local buffer for dissemination Looks like an exponential moving average, right? But weighting š‘Ž = š›¼ 1 āˆ’ š›½Pš‘– is probabilistically applied!
  • 40. 3/13/2017 40Demetris Trihinas 40 Low-Cost Approximate Estimation Model EWMA after spikes overestimates subsequent values EWMA slow to acknowledge spikes With probabilistic reasoning each datapoint will contribute to the estimation process depending on itā€™s p-value Why use a Probabilistic EWMA?
  • 41. 3/13/2017 41Demetris Trihinas 41 AdaM: Adaptive Sampling and Filtering ā€œAdaM: Adaptive Monitoring Framework for Sampling and Filtering on IoT Devicesā€, D. Trihinas, G. Pallis and M.D. Dikaiakos, IEEE BigData 2015 ā€œ[Low-Cost Adaptive Monitoring Techniques for the Internet of Thingsā€, D. Trihinas, G. Pallis and M.D. Dikaiakos, IEEE TSC 2016 Ī¼Processor Ī¼OS Memory Low-Cost Approximate Stream Estimation Adaptive Sampling Adaptive Filtering S E N S I N G U N I T metric updates A P I updated periodicity estimation confidence Adaptive Dissemination model updates & buffer content when shift detected updated filter range stream variability filtered metric stream AdaM N E T W O R K U N I T Model Base stream evolution and online stats Datapoint Buffer IoT Device metric stream compressed metric stream
  • 42. 3/13/2017 42Demetris Trihinas 42 Adaptive Sampling ā€¢ After updating estimation model, we compute current confidence (š‘š‘– ā‰¤ 1) of approach based on the actual and estimated standard deviation ā€¢ Next, we update sampling period š‘‡š‘–+1 based on the current confidence and the user-defined imprecision Ī³ (e.g. 10% tolerance to errors) š‘š‘– = 1 āˆ’ | šœŽš‘– āˆ’ šœŽš‘–| šœŽš‘– ā€¦ the more ā€œconfidentā€ the algorithm is, the larger the outputted sampling period š‘‡š‘–+1 can beā€¦ Ī» is an aggressiveness multiplier (default Ī»=1)
  • 43. 3/13/2017 43Demetris Trihinas 43 Adaptive Filtering ā€¢ Compute current variability of the metric stream using moving Fano Factor š¹š‘– which is an indicator of the stream value dispersion ā€¢ We then compare šœŽš‘’š‘Ÿš‘Ÿ to the user-provided maximum tolerable imprecision guarantees š›¾ in attempt to widen filter range š¹š‘– = šœŽš‘– 2 šœ‡š‘– ā€¦a low š¹š‘– (due to šœŽš‘–) indicates a currently in-dispersed data stream which means low variability in the metric streamā€¦
  • 44. 3/13/2017 44Demetris Trihinas 44 AdaM in action! Steps Heartrate Calories 94% accuracy! 96% accuracy! 91% accuracy!
  • 45. 3/13/2017 45Demetris Trihinas 45 6x less processing! 4x less network traffic! 4x less energy usage! + AdaM 3 - 5 days 7 - 8 days AdaM in action!
  • 46. 3/13/2017 46Demetris Trihinas 46 Memory Trace Java Sorting Benchmark CPU Trace Carnegie Mellon RainMon Project Disk I/O Trace Carnegie Mellon RainMon Project TCP Port Monitoring Trace Cyber Defence SANS Tech Institute Step Trace Fitbit Charge HR Wearable Heartrate Trace Fitbit Charge HR Wearable More datasets!
  • 47. 3/13/2017 47Demetris Trihinas 47 Balance between efficiency and accuracy! Energy Consumption CPU Cycles Network Traffic Error (MAPE) FAST [L. Fan et al., SIGMOD 2014] L-SIP [E Gaura et al., IEEE Trans. on Sensors, 2013]
  • 48. 3/13/2017 48Demetris Trihinas 48 Dublin Smart City Intelligent Transportation Service (Dublin ITS) . . . . . . 1000 Buses* with GPS tracking sending updates to ITS with 16 params (e.g. busID, location, current route delay) *Real data from Jan. 2014 Apache Kafka queuing service on x-large VM (16VCPU, 16GB RAM, 100GB Disk) Apache Spark cluster with 5 workers on large VMs (8VCPU, 8GB RAM, 40GB Disk) Alerts ITS operators when more than 10 buses in a Dublin city area, per 5min window, are reporting delays over 1 standard deviation from their weekly average AdaMā€¦ integrated in big data streaming service!
  • 49. 3/13/2017 49Demetris Trihinas 49 Spark total delay (processing + scheduling) for T=1, 5, 10 intervals and AdaM with max imprecision Ī³ = 0.15 AdaMā€¦ integrated in big data streaming service!
  • 50. 3/13/2017 50Demetris Trihinas 50 AdaM achieves a >85% accuracy in major Dublin areas AdaMā€¦ integrated in big data streaming service!
  • 51. 3/13/2017 51Demetris Trihinas 51 AdaM: Adaptive Dissemination ā€œADMin: Adaptive Monitoring Dissemination for the Internet of Thingsā€, D. Trihinas, G. Pallis and M.D. Dikaiakos, IEEE INFOCOM, 2017 Ī¼Processor Ī¼OS Memory Low-Cost Approximate Stream Estimation Adaptive Sampling Adaptive Filtering S E N S I N G U N I T metric updates A P I updated periodicity estimation confidence Adaptive Dissemination model updates & buffer content when shift detected updated filter range stream variability filtered metric stream AdaM N E T W O R K U N I T Model Base stream evolution and online stats Datapoint Buffer IoT Device metric stream compressed metric stream
  • 52. 3/13/2017 52Demetris Trihinas 52 AdaM: Adaptive Dissemination ā€œADMin: Adaptive Monitoring Dissemination for the Internet of Thingsā€, D. Trihinas, G. Pallis and M.D. Dikaiakos, IEEE INFOCOM, 2017 Network Unit ADMin Seasonality Enrichment Shift Detection A P I Model Base Approximate Stream Estimation Remote seasonal periodicity update test if enrichment is beneficial? metric updates update stream evolution and trend seasonal enrichment of stream evolution compressed metric stream Seasonal Periodicity Detection The ADMin plugin for AdaM Donā€™t disseminate datapoint values... Instead disseminate model updates!
  • 53. 3/13/2017 53Demetris Trihinas 53 Low-Cost Approximate Estimation Model Step 2: Detect over time gradual trends in monitoring stream to reduce ā€œlaggingā€ effects in monitoring stream evolution estimation š‘£š‘–+š‘˜|š‘– šœ‡š‘– + š‘˜ š‘‹š‘– Upward trend Downward trend ā€¢ Holtā€™s Trend Method used to bring moving average to appropriate value base ā€¢ Improve forecasting from 1-step ahead (moving average) predictions to k-datapoint values
  • 54. 3/13/2017 54Demetris Trihinas 54 Low-Cost Approximate Estimation Model Step 3: Test if seasonality enrichment is beneficial to estimation process and update low-cost approximate model accordingly seasonal Seasonal with exponential trend Seasonal with damped trend ā€¢ Tendency of the metric stream to exhibit behavior that repeats itself every L periods (e.g., hourly) ā€¢ Seasonal effects highly evident in IoT data (e.g., human biosignals, environmental data) PV Panel Production Air Temperature
  • 55. 3/13/2017 55Demetris Trihinas 55 Low-Cost Approximate Estimation Model ā€¢ Holt Winterā€™s Method used to estimate seasonal contribution ā€¢ Forecasting k-subsequent datapoints with trend and seasonality ā€¢ Howeverā€¦ perfect seasonal behavior is rarely observed in real-life systems PV Panel Production š‘£š‘–+š‘˜|š‘– šœ‡š‘– + š‘˜ š‘‹š‘– + š‘†š‘– Considering day before hourly average (Si-L) will lead to overestimation
  • 56. 3/13/2017 56Demetris Trihinas 56 Low-Cost Approximate Estimation Model ā€¢ Online testing (T-Test) to determine if seasonal contribution beneficial to estimation or not ā€¢ Detecting optimal seasonality cycle (L) is an open research challenge especially when different cycles exist in monitoring stream ā€¢ Approximate runtime seasonal periodicity detection ā€¢ ComCube Framework (Matsubara et al., WWW, 2016): lightweight tensor-based and parameter-free framework for near-optimal seasonal periodicity detection š‘£š‘–+š‘˜|š‘– šœ‡š‘– + š‘˜ š‘‹š‘– + š‘†š‘– ā€œNon-linear mining of competing local activities ā€, Y. Matsubara, Y. Sakurai and C. Faloutsos, WWW 2016
  • 57. 3/13/2017 57Demetris Trihinas 57 Detecting Shifts in a Monitoring Stream ā€¢ Cumulative Sum (CUSUM) to detect shifts in monitoring stream value distribution which render estimation model as inconsistent ts prior shift after shift š‘š‘– = ln š‘ƒ(š‘€š‘–, šœ‡ā€²ā€² ) š‘ƒ š‘€š‘–, šœ‡ā€² šœ‡š‘– ā€²ā€² = šœ‡š‘– ā€² + šœ€ where Īµ magnitude of change 1. log-likelihood ratio 2. Detecting shifts in mean However, šœ€ not known beforehand butā€¦ estimation model model provides us with an approximate šœŗ
  • 58. 3/13/2017 58Demetris Trihinas 58 Detecting Shifts in a Monitoring Stream 3. Online CUSUM 4. Decision Function Challenge 1: Linear timeā€¦ Can š’•š’Š be used instead of š’• š’”? ts ti ti may greatly from ts differ in cases of gradual trends š‘–š‘“ šŗš‘–,{š‘™š‘œš‘¤,ā„Žš‘–š‘”ā„Ž} > ā„Ž ā†’ š‘ ā„Žš‘–š‘“š‘” š‘‘š‘’š‘”š‘’š‘š‘”š‘’š‘‘ h is measured in standard deviation units 5. Actual Shift Time the time the CUSUM detects the shifts
  • 59. 3/13/2017 59Demetris Trihinas 59 Detecting Shifts in a Monitoring Stream ts ti Trend and seasonality knowledge provide model with greater accuracy ( šœ€ ā†’ šœ€) and to adapt to unexpected, abrupt and and volatile changes is monitoring stream Challenge 2: CUSUM threshold h static and sensitive when stream variability is low (šœŽ ā†’ 0 ) thus triggeringā€¦ false alarms Adapt CUSUM sensitivity by adapting h after dissemination triggered and restrict h with hmin
  • 60. 3/13/2017 60Demetris Trihinas 60 ADMin Evaluation ā€¢ ADMin in 3 configurations ā€¢ No seasonality enrichment (ADMin) ā€¢ Static seasonality enrichment - previous day hourly average (ADMin_S1) ā€¢ Dynamic seasonality enrichment - ComCube integration (ADMin_S2) ā€¢ Under comparison frameworks ā€¢ LANCE [Werner et al., ACM SenSys 2011] ā€¢ G-SIP [Gaura et al., IEEE Trans. on Sensors 2013] ā€¢ ADWIN [Bifet et al., SIAM 2010] All three framework parameters configured to output best results
  • 61. 3/13/2017 61Demetris Trihinas 61 Photovoltaic Panel Current (IDC) Production 2 Weeks of data collected every 1 second Data reduction: 87% -- Accuracy: 93% Weather Station Air Temperature (oC) 2 Weeks of data collected every 1 second Data reduction: 85% -- Accuracy: 92% Wearable Human Heartrate (bpm) 2 Weeks of data collected every 1 minute Data reduction: 80% -- Accuracy: 90% ADMin in Action!
  • 62. 3/13/2017 62Demetris Trihinas 62 Shift Detection Evaluation ground truth (offline PELT algorithm) false alarms correctly detected shifts Air Temperature Dataset Wearable Dataset Shift Detection Delay
  • 63. 3/13/2017 63Demetris Trihinas 63 Energy Consumption Data Volume Reduction 1O interval aggregation ADWIN large energy consumption due to complexity and absence of data reduction scheme LANCE large energy consumption due to enabling dissemination x2 more than ADMin (false alarms) and downloading each time all available datapoints G-SIP large energy consumption due to shift detection sensitivity (only EWMA based) which enables false alarm disseminations Overhead Evaluation
  • 64. 3/13/2017 64Demetris Trihinas 64 Facebook DB Cluster Adaptive Monitoring ā€œInside the Social Networkā€™s (Datacenter) Networkā€, A. Roy et al., ACM SIGCOMM, 2015
  • 65. 3/13/2017 65Demetris Trihinas 65 ā€¢ Edge-mining on IoT devices is both resource and energy intensive ā€¢ Big data streaming engines struggle to cope as the volume and velocity of IoT-generated data keep increasing ā€¢ Adapts the monitoring intensity based on current metric evolution and variability ā€¢ Reduces processing, network traffic and energy consumption on IoT devices and the IoT network ā€¢ Achieves a balance between efficiency and accuracy The AdaM Framework Conclusion
  • 66. 3/13/2017 66Demetris Trihinas 66 http://linc.ucy.ac.cy/AdaM
  • 67. 3/13/2017 67Demetris Trihinas 67 Low-Cost Approximate & Adaptive Monitoring Techniques for the Internet of Things Demetris Trihinas trihinas@cs.ucy.ac.cy PhD Candidate Department of Computer Science University of Cyprus

Notas do Editor

  1. You cant have a gigantic pc on your wrist! The building blocks of IoT
  2. What makes IoT interesting and worth studying since we are obviously not device or hardware makersā€¦ big data
  3. So it only seems as natural that the cloud could be used
  4. So it only seems as natural that the cloud could be used
  5. Monitoring server (tree) hierarchy to increase fault-tolerance, robustness and scalability Monitoring servers poll assigned agents that form monitoring clusters for metric updates pushing aggregated data further up the hierarchy If you have large monitoring clusters there is a lot of network traffic going on intracluster-wise but if you have too small then the hierarchy grows too large and you have relaying costs when aggregating data and answering queries
  6. These sensors are not source code sitting on VMsā€¦. They are real devices scattered across the logical extremes of the network (incoming/outcoming cloud traffic is priced)
  7. Monitored elements (e.g., VMs, sensors)
  8. These sensors are not source code sitting on VMsā€¦. They are real devices scattered across the logical extremes of the network
  9. These sensors are not source code sitting on VMsā€¦. They are real devices scattered across the logical extremes of the network
  10. How physical sensing differs from OS monitoringā€¦ These sensors are not source code sitting on VMsā€¦. They are real devices scattered across the logical extremes of the network
  11. We use the word ā€œideallyā€ because this is not the norm used by most techniques which use fixed step increments/decrements
  12. So our confidence metric is actually a different way to denote this dist as we actually donā€™t have both the original and reconstructed version
  13. Filtering (but its not really adaptive!)ā€¦
  14. Why probabilistic? ā€œUnexpectedā€ datapoints are stored in local buffer for dissemination (if needed e.g., for anomaly detection, otherwise as discussed only model updates need be sent to receivers) Next slideā€¦ datapoints contribute to estimation process based on p-value
  15. Estimation base needs an uplift to not trail behind
  16. Add faloutsos paper here
  17. Shift in monitoring stream statistical propertiesā€¦. We are not interested here for anomalies
  18. Accumulating (small) portions of drift until threshold surpassed