It is not rare when relatively primitive charts or diagrams are referred to as visualizations. When one is monitoring/managing performance at the scale of a data center, it is especially obvious that such visualizations are not very helpful as they fail to capture or present information in a holistic way, thus making it hard to make management decisions. This paper proposes a holistic visualization method which is based on the theory of information visualization, namely (1) metaphor as base, (2) density maximization and (3) proximity packing. The proposed method can concurrently digest multiple streams of packet traffic but can also incorporate streams of alternative monitoring data.

2. .
Mission Statement
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 2/24
...
2/24

3. .
Mission Statement / Goals
1. a holistic method for performance visualization
in DCs
2. a clean slate + there are no rivals
3. a generic design applicable to various kinds of performance data
◦ traffic in this paper, but ideally anything
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 3/24
...
3/24

4. .
Visualization Primer
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 4/24
...
4/24

5. .
Visualization Theory
.
Visualization Tradition...
..
.... is to maximize information density and minimize labeling
• metrics are density and proximity -- proximity packing problem
• Tufte 05 and others 06
.
New (proposed) Formulation...
..
.... adds metaphor and enhances the labeling metric
• metaphor is a strict requirement -- 3D/3D charts are not metaphors
• higher number of metrics
05 E.Tufte+0 "The Visual Display of Quantitative Information" Graphics Press (1983)
06 J.Steele+1 "Beautiful Visualization" O'Reilly Media (2010)
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 5/24
...
5/24

6. .
Visualization Methods
• Waterpipe Model (the proposal) is explained later
• key point: 2D → 3D may actually make things worse!
Waterpipe
Model
2D 3D
Metaphor YES NO NO
Information Density HIGH LOW BETTER
Labeling minimization GOOD BAD BETTER
Perceptual enhancement GOOD BAD WORSE
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 6/24
...
6/24

7. .
Visualization Examples
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 7/24
...
7/24

8. .
Viz : Example (1)
• Community QoS visualization from 04
0 6400 12800 19200
Batch size (bytes)
0
800
1600
2400
3200
4000
OWD(ms)+TXtime(x0.1ms)
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 8/24
...
8/24

9. .
Viz : Example (2)
Meter Merger
Analyzer
Analyzer
Analyzer
Buffer 5sTimeout 5s
Sampling
timeout 10s
Buffer 1s
Buffer 10s
Buffer 30s
Lower cluster
Upper cluster
47kb
25kb
21kb
12kb
0.6kb
0.3kb
Per second
Per buffer
10kb
7.4kb
32kb
26kb
Different
buffer size
• also from 04
• Petri Nets are based on the
tree metaphor -- simple
but powerful
04 myself+0 "A holistic... for measuring end-to-end QoS at data centres" IJCSE (2014)
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 9/24
...
9/24

10. .
Proposal : The Waterpipe Model
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 10/24
...
10/24

11. .
Proposal : Waterpipe Model
• Existing (1) : there is already fluid model in traffic simulations
◦ differential equations
◦ data unit is aggregate flow rather than packets
◦ you do not actually run such a simulation -- you take a snapshot at a given point in time
• Existing (2) : Bucket Model in queueing theory -- a bucket of water
• Waterpipe Model is similar, only with a strong visual component --
the metaphor
• Data Center is literally viewed as a system of water pipes -- a job for plumbers?
• the metaphor is flexible, can support multiple variants
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 11/24
...
11/24

12. .
Waterpipe : Model 1
• one tap is one VM/PM/etc.
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 12/24
...
12/24

13. .
Waterpipe : Model 2
• same, but more focus on in/out balance of goodput, congestion, etc
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 13/24
...
13/24

14. .
Implementation
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 14/24
...
14/24

15. .
Implementation : Status
• 80% complete
• already have all the components from past research -- see below
• the visual part is new but not hard to design
• components:
1. multicore traffic capture and processing
2. client-side web aggregator based on WebSockets and WebWorkers
(HTLM5)
3. interactive design -- this part is pending but easy
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 15/24
...
15/24

16. .
Implementation : Multicore
• heavy use of multicore and realtime processing -- recent paper in 07
Global
Networks
Data Center
Internals
Gateway
Switch
Capture
Manager
CPU
CPU
CPU
CPU
CPU
CPU
…
Storage
Mirror
07 myself+0 "A Software Design and Algorithms for Multicore Capture..." ACM ASIA-CCS-SFCS (2014)
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 16/24
...
16/24

17. .
Implementation : Sockets
• WebSockets -- streaming, native binary support, heavily interactive,
client-side aggregation 01
• recent study shows that 900Mbps can be supported using parallel sockets 02
• again, WebWorkers expoit multicore if present
Network
Chrome Browser
Control
……
Media
requests
Server
Buffer
Client
WebSocket
Apache
web server
WebSocket
server
WebWorker
WebSocket
WebWorker
Network
Server
Round
Robin
Add
01 myself+0 "Multi-Source Stream Aggregation in the Cloud" ADCN Wiley Book (2013)
02 myself+0 "Experiments with application throughput in a browser with full HTML5 support" IEICE ComEx (2013)
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 17/24
...
17/24

18. .
Design : Capture/Processing
• any level of abstraction: PMs, VMs, storage, etc.
• realtime processing and decision-making -- towards interactive visualization 07
Web
Worker
Web
Worker
CPU
Core(s)
….
PM
PM
PM
PM
Switch To Cloud
Split Merge
Traffic
Mirroring
Interactive
Display
07 myself+0 "A Software Design and Algorithms for Multicore Capture..." ACM ASIA-CCS-SFCS (2014)
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 18/24
...
18/24

19. .
Design : Interactive Display
• aggregation and processing on multicore, highly interactive display
• basically, a player (of performance info) plus interaction
• recent study on the most cost effective visualization technologies in
browser 03
Web
Worker
Web
Worker
Web
Worker
Web
Worker Player
1 2 3
….
….Web
Socket
Browser Window (GUI)
03 myself+0 "... the Most Cost Effective Method in Browser-Based Visualizations" IIEEJ (2014)
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 19/24
...
19/24

20. .
Wrapup
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 20/24
...
20/24

21. .
Wrapup
• 80% complete with most important components fully available
• currently testing on a small cloud at Kyutech
• only 2 variants shown, but more are possible -- all based on waterpipes
• important: not limited to traffic, can work on any performance data
◦ CPU, memory, storage, etc.
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 21/24
...
21/24

22. .
That’s all, thank you ...
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 22/24
...
22/24

23. .
[01] myself+0 (2013)
Multi-Source Stream Aggregation in the Cloud
ADCN Wiley Book
[02] myself+0 (2013)
Experiments with application throughput in a browser with full HTML5 support
IEICE ComEx
[03] myself+0 (2014)
... the Most Cost Effective Method in Browser-Based Visualizations
IIEEJ
[04] myself+0 (2014)
A holistic... for measuring end-to-end QoS at data centres
IJCSE
[05] E.Tufte+0 (1983)
The Visual Display of Quantitative Information
Graphics Press
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 22/24
...
22/24

24. .
[06] J.Steele+1 (2010)
Beautiful Visualization
O'Reilly Media
[07] myself+0 (2014)
A Software Design and Algorithms for Multicore Capture...
ACM ASIA-CCS-SFCS
[08] myself+0 (current)
1100+ Examples of Information Visualization
http://www.infovis.info/index.php?words=diagrams
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 23/24
...
23/24

25. .
Extras : Processing Cost
Hashing
Increasing
Per-Unit Cost
Manager
Prefix
Matching
Cores that
do not
match
Process
Stage 1 Stage 2 Stage 3
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 23/24
...
23/24

26. .
Extras : Multicore Processing
PF_
RING
PF_
RING
Time
Manager
Shared Memory
Capture
Capture
…
Core 1
Core 2
Core 3
….
Core X
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 24/24
...
24/24