This document describes the implementation and evaluation of a virtual cluster on a cloud. The architecture uses OpenNebula for resource management across two physical sites connected by an overlay network. Benchmark results show the multi-site virtual cluster has acceptable performance for CPU-intensive and memory-intensive applications, though network performance is reduced compared to the physical network. The virtual cluster approach works well for applications that are not highly network dependent. Future work involves improving the overlay network using SDN and developing intelligent scheduling of VMs.

1. An Implementation of
Virtual Cluster on a Cloud
Pongsakorn U-chupala
Kasetsart University
#PRAGMA24

2. Agenda
• Introduction
• Design & Implementation
• Benchmarking Results

3. Virtual
Cluster
User
Frontend
GPU Accelerated Nodes
High-speed
Network
Compute Nodes
Storage Farm
*Example setup

4. Managing virtual cluster Manually
is a
Headache
http://openphoto.net/gallery/image.html?image_id=24436

5. Virtual Cluster Cloud
User Virtual Cluster Template
*Not limited to Rocks Cluster Virtual Cluster

6. Multi-site Infrastructure
Japan
Overlay Network
USA
Thailand
*Countries listed are just for examples

7. Design Goals
• Automatic resources management
• Designed with scalability in mind
• Acceptable performance
• Virtual Cluster management as a single entity
– Create – Stop
– Deploy – Delete
– Suspend

8. Agenda
• Introduction
• Design & Implementation
• Benchmarking Results

9. Architecture
O ebula
penN onevc extension Virtual Infras ture
truc
Multi-tier V N
P O erlayN o
v etw rk
R ourceS 1(R )
es ite ocks
R ource
es
V -C iner
M onta V -C iner
M onta S 2
ite P ic R o e
hys al es urc

10. Multi-tier VPN
P a VN S
rim ry P erver P a V NS
rim ry P erver
V NC
P lient
S 1
ite
B e
ridg V NJ
P unction
S 2
ite
V NS
P erver
V -C iner
M onta V -C iner
M onta
V -C iner V NC
M onta P lients

11. Agenda
• Introduction
• Design & Implementation
• Benchmarking Results

12. Cloud Testbed
OpenNebula (CentOS 6.2)
Public Network
Site A (Rocks 6.0) Site B (Rocks 6.0)
Frontend Frontend
Private Network Private Network
VM-Container-0-0 VM-Container-0-1 VM-Container-0-0 VM-Container-0-1
Overlay Network

13. Network Benchmarking
• Average Bandwidth: measured with Iperf
– Physical Network: 776.62 Mbits/s
– Overlay Network: 314.26 Mbits/s
• Round-Trip-Time: measured with ping
– Physical Network: 0.135 ms
– Overlay Network: 0.455 ms

14. Virtual Cluster Testset
Site A Site B
VM-Container-0-0 VM-Container-0-1 VM-Container-0-0 VM-Container-0-1
Multi-site Virtual Cluster
Frontend Compute-0-0 Compute-0-1 Compute-0-2
Single-site Virtual Cluster
Frontend Compute-0-1
Compute-0-0 Compute-0-2
Single-site Virtual Cluster with Physical Network
Frontend Compute-0-1
Compute-0-0 Compute-0-2
Single-host VC
Frontend
Compute-0-0
Compute-0-1
Compute-0-2

15. HPCC Benchmarking Results

16. DGEMM (CPU Intensive)
DGEMM (CPU Intensive)
10 9.045199 9.100802 9.303938
9
8
Average Gflops
7
6
5 4.197254
4
3
2
1
0
Single-host Single-site Single-site with Multi-site
Physical Network

17. Average GB/s
0
5
10
15
20
20.318253 25
20.153016
19.75952
Single-host
18.470081
18.605038
Copy
19.260255
18.560227
Scale
Single-site
17.8158
Add
18.669311
18.415776
Triad
17.819891
17.472441
Single-site with
Physical Network
21.183269
21.509165
21.116839
STREAM (Memory Bandwidth)
Multi-site
19.738259

18. HPL (Peak Performance)
2 1.873
1.8
1.6
1.4
1.2
Gflops
1
0.8
0.6 0.5229
0.4117
0.4 0.3126
0.2
0
Single-host Single-site Single-site with Multi-site
Physical Network

19. Conclusion
• The virtual cluster cloud works as expected
• This cloud is suitable for non-network-
intensive application
– Bio-Chemical-Related Research
– Video Rendering
– Brute-force Searches
– Bitcoin Mining :D

20. Future Works
• Better Overlay Network implementation
– Software-defined Networking (SDN)
• Intelligence VM placement mechanism
(scheduling)
– Swarm algorithm
– Firefly algorithm

21. Thank you for you attention
Q&A