This document provides an overview of emerging machine learning architectures, including cloud, edge, fog, and mist computing. It discusses the timeline of remote and machine learning computing from early cloud computing to current edge and fog approaches. The need for edge computing to address latency issues for applications like augmented reality and face recognition is explained. Key aspects of fog computing like its role in scalably extending cloud computing to network edges are covered. The document also provides an example of implementing deep learning for an IoT video recognition application across edge and cloud resources.

1. Daniel Holmberg 22.3.2020
Emerging Computing
Architectures in
Machine Learning
1

2. TABLE OF
CONTENTS
Introduction
Cloud and ML background
Cloud Computing
Compute resources leased in an
on-demand fashion
01
03
02
04
05
06
Emergence of Edge
Importance of latency
Fog Computing
IoT infrastructure scalability
ML Implementation
Machine learning IoT application
Conclusion
Concluding words
2

3. Introduction
01
3

4. 4
Remote Computing Timeline
early
2000s
- Application service providers (ASPs)
- Rapid development of processing and storage
technologies,
- More widely-spread and faster internet
2006 2015
2012 2020s
Fog computing introduced to
facilitate the deployment of
distributed, latency-aware
services
5G + edge computing = <3
Low latency first hop to edge
servers
AWS EC2 release:
enabled compute in the
cloud, pay only for capacity
that you actually use
Edge computing for
applications where
real-time processing of data
is required
4

5. Modern ML Computing Timeline
2011
DistBelief: ML on Google’s
internal computing clusters
2015 2017
2016 2019
ML application-specific
integrated circuits (ASICs)
- AWS Sagemaker Neo: neural net reduces model
memory footprint for running on edge devices
- TensorFlow 2.0 released
- TensorFlow Lite Micro for microcontrollers
- Tensor processing unit (TPU) announced
- AlphaGo beats Lee Sedol at Go
- TPUs available in the cloud
- TensorFlow 1.0: open source ML and dataflow library
- TensorFlow Lite with mobiles in mind
5

6. Cloud Computing
02
6

7. Cloud Computing Architecture
Business Applications, Web Services, Multimedia
7
Software Framework (Java/Python/.NET)
Storage (DB/File)
Computation (VM) Storage (block)
CPU, Memory, Disk, Bandwidth
Application
Platforms
Infrastructure
Hardware
End User

8. Cloud Computing Architecture
Business Applications, Web Services, Multimedia
8
Software Framework (Java/Python/.NET)
Storage (DB/File)
Computation (VM) Storage (block)
CPU, Memory, Disk, Bandwidth
Application
Platforms
Infrastructure
Hardware
End User

9. Cloud Computing Architecture
Business Applications, Web Services, Multimedia
9
Software Framework (Java/Python/.NET)
Storage (DB/File)
Computation (VM) Storage (block)
CPU, Memory, Disk, Bandwidth
Application
Platforms
Infrastructure
Hardware
End User

10. Cloud Computing Architecture
Business Applications, Web Services, Multimedia
10
Software Framework (Java/Python/.NET)
Storage (DB/File)
Computation (VM) Storage (block)
CPU, Memory, Disk, Bandwidth
Application
Platforms
Infrastructure
Hardware
End User

11. Cloud Computing Architecture
Business Applications, Web Services, Multimedia
11
Software Framework (Java/Python/.NET)
Storage (DB/File)
Computation (VM) Storage (block)
CPU, Memory, Disk, Bandwidth
Application
Platforms
Infrastructure
Hardware
End User

12. Virtualization
A full operating system is
installed on top of the
virtualized hardware.
Isolate processes at the
OS level instead of
virtualizing hardware
and drivers.
Specialized into standalone
kernels at compile time,
modification is impossible
after deployment.
12

13. Virtualization
13

14. Virtualization
Instantation
time
Image size
Programming
language
dependency
Hardware
portability
VM ~5/10 s ~ 1000 Mb No High
Container ~800/1000 ms ~ 50 Mb No High
Unikernel ~< 50 ms ~< 5Mb Yes High
14

15. Architectural Service Models
IaaS
Control over operating
systems, storage, and
deployed applications, but
not the underlying physical
components.
PaaS
Customer provided with software
development frameworks:
programming languages, libraries
and services + configuration of the
application-hosting environment.
SaaS
Access to provider’s application,
but no modification of underlying
code or infrastructure.
Only user-specific application
configuration settings.
MLaaS
Cloud provider toolkits with
user friendly interfaces and
APIs with access to ML
algorithms, analytics and
visualisation capabilities.
15

16. Architectural Service Models
IaaS
Control over operating
systems, storage, and
deployed applications, but
not the underlying physical
components.
PaaS
Customer provided with software
development frameworks:
programming languages, libraries
and services + configuration of the
application-hosting environment.
SaaS
Access to provider’s application,
but no modification of underlying
code or infrastructure.
Only user-specific application
configuration settings.
MLaaS
Cloud provider toolkits with
user friendly interfaces and
APIs with access to ML
algorithms, analytics and
visualisation capabilities.
16
Azure App Service
Google
Compute
Engine

17. Emergence of Edge
03
17

18. Edge Computing Need
Mobile response time distribution and per-operation energy cost
of an (a) augmented reality and (b) face recognition application.
An image from a mobile device in Pittsburgh is transmitted over
Wi-Fi to a cloudlet or an AWS datacenter. The graphs demonstrate
the need for low-latency offload services.
18
(a) (b)

19. 2.
physical proximity lowers end-to-end
latency and achieves high bandwidth
Advantages of Edge Computing
19
1.
feed to cloudlets for feature extracting and
send only that to the central server
3. cloudlets can invoke own privacy policies prior to
releasing IoT data prior further to the cloud provider
4. if a cloud service becomes unavailable, the cloudlet
can conceal it temporarily with a fallback service

20. Fog Computing
04
20

21. Cloud-based Ecosystem
21

22. Terminology
Components Location Use
Fog
Physical: gateways,
servers, routers, ...
Virtual: VMs, virtualized
switches, cloudlets, ...
Multi-layer architecture
between cloud datacenters
and end devices
Computation, networking,
storage, control and
data-processing acceleration
Mist
Microcomputers and
microcontrollers
Edge of the network fabric Feed into fog computing nodes
Edge
Same as fog except limited
to a small number of
peripheral devices
IoT Network: layer with
end-devices and users
Run specific applications in a
fixed logic location and provide
a direct transmission service
22

23. ML Implementation
05
23

24. Deep Learning IoT Application
■ Task: IoT video recognition
■ Videofeed bit-rate: 3000 kb/s
■ Enters neural net initial layer
■ Output from previous layer to next layer
■ Target features extracted from final layer
24

25. Deep Learning IoT Application
■ Split neural net at suitable layer
■ Feed reduced to 2300 kb/s in edge
server
■ Balancing:
● Latency benefit from more
preprocessed data to the cloud
● Limited capacity and power of
fog node processing
25

26. Conclusion
06
26

27. Recap
Cloud Mist
Fog Edge
- Multi-layer architecture
- IoT infrastructure scalability
- Few devices
- Mobile applications’ interactive
performance in mind
- Feed into fog
- Reside in network fabric
- Microdevices
- On-demand datacenter
resources
- SaaS, PaaS, Iaas and
MLaaS
27

28. “Since the 1960s, computing has alternated between centralization and
decentralization. The centralized approaches of batch processing and
timesharing prevailed in the 1960s and 1970s. The 1980s and 1990s
saw decentralization through the rise of personal computing.
By the mid-2000s, the centralized approach of cloud computing began its
ascent to the preeminent position that it holds today. Edge computing
represents the latest phase of this ongoing dialectic”
- Dr. M. Satyanarayanan
28

29. Credits: This presentation template was created by Slidesgo, including
icons by Flaticon, and infographics & images by Freepik.
daniel.holmberg@helsinki.fi
THANK YOU
29

30. Figure References
p. 12 & 13 Roberto Morabito, Vittorio Cozzolino, Aaron Yi Ding, Nicklas Beijar, and Jörg Ott.
Consolidate iot edge computing with lightweight virtualization. IEEE Network, 2018.
p. 18 K. Ha et. al. The impact of mobile multimedia applications on data center
consolidation. p. 166–176, 2013.
p. 21 National Institute of Standards and Technology. Fog Computing Conceptual Model:
NIST SP 500-325 500-291. CreateSpace Independent Publishing Platform, 2018.
p. 24 & 25 He Li, Kaoru Ota, and Mianxiong Dong. Learning iot in edge: Deep learning for the
internet of things with edge computing. IEEE Network, 32:96–101, 01 2018.
30