An independent US-based cloud analyst group released the CloudSpecs Performance Report in which it underlines that "Lunacloud outperforms Amazon and Rackspace consistently over a period of 30 days". This CloudSpecs Performance Report compares cloud servers of Amazon EC2, Rackspace Cloud Servers and Lunacloud. Performance is measured over a period of 30 days to collect data on server performance capabilities as well as consistency in performance. The CloudSpecs system runs open source benchmark tests to measure performance in CPU, RAM, disk, and internal network. Some of the key findings were that in the general server test, Lunacloud outperforms Amazon and Rackspace consistently over a period of 30 days; Lunacloud disk performance scores up to 8x better than Amazon or Rackspace disk performance; and Lunacloud’s internal network throughput is 2x faster than Rackspace’s internal network throughput, and significantly more stable than Amazon in a period of 30 days. http://www.lunacloud.com/en/lunacloud-outperforms-amazon-ec2

1. CLOUDSPECS PERFORMANCE REPORT
LUNACLOUD, AMAZON EC2, RACKSPACE CLOUD
AUTHOR: KENNY LI
NOVEMBER 2012

2. EXECUTIVE SUMMARY
This publication of the CloudSpecs Performance Report compares cloud servers of Amazon EC2, Rackspace Cloud Servers (not OpenStack Cloud), and Lunacloud.
Performance is measured over a period of 30 days to collect data on server performance capabilities as well as consistency in performance. The CloudSpecs system
runs open source benchmark tests to measure performance in CPU, RAM, disk, and internal network. The software is set to automatically run 4 times a day for 7
days a week.
The report is comprised of:

1 general server performance test

5 CPU performance tests



3 Disk performance tests
2 RAM performance tests
7 network performance tests
Key Findings



In the general server test, Lunacloud outperforms Amazon and
Rackspace consistently over a period of 30 days.
Lunacloud disk performance scores up to 8x better than Amazon
or Rackspace disk performance. This is an important
consideration for database performance.
Lunacloud’s internal network throughput is 2x faster than
Rackspace’s internal network throughput, and significantly more
stable than Amazon in a period of 30 days. With applications
requiring more than one server, internal network speed may be
considered a bottleneck in many cases if other components of the
server are optimized.


Amazon EC2’s CPU performance generally scores slightly better
than Rackspace and Lunacloud CPU performance. CPU is a vital
component of the server, and application performance depends
significantly on CPU functionality.
Rackspace’s RAM is more consistent in performance than
Amazon or Lunacloud, though it scores the lowest in performance
over a period of 30 days. Though the processor speed is an
important factor in server performance, RAM plays an equally
important role. Because modern CPUs can perform many of the
applications run today, bottleneck instances may actually occur
with RAM.
METHODOLOGY
Cloud Spectator sets up two servers on each cloud provider that meets the following minimum requirement:
Primary Server
Secondary Server
CPU Cores
RAM in GB
Disk in GB
4 vCPUs
16
50
CPU Cores
RAM in GB
Disk in GB
1 vCPU
4
50
Because Amazon and Rackspace offer packaged cloud servers with preset allocations of CPU, RAM, and disk, the closest package that meets the minimum
requirements is selected.
AMAZON
The servers are set up on Amazon EC2 availability zone US-East-1a. Virtualized server OS is Amazon Linux.
Primary Server
Secondary Server
CPU Cores
RAM in GB
Disk in GB
4 vCPUs
15
1,690
CPU Cores
RAM in GB
Disk in GB
1 vCPUs
3.75
410
CPU Cores
RAM in GB
Disk in GB
2 vCPUs
4
160
RACKSPACE
The servers are set up in Rackspace’s Dallas (DFW) data center. Virtualized server OS is Debian 6.
Primary Server
Secondary Server
CPU Cores
RAM in GB
Disk in GB
4 vCPUs
15.872
620
2

3. LUNACLOUD
The servers are set up in Lunacloud’s Virtualized server OS is Debian 6.
Primary Server
Secondary Server
CPU Cores
RAM in GB
Disk in GB
4 vCPUs
16
50
CPU Cores
RAM in GB
Disk in GB
1 vCPUs
4
50
Cloud Spectator maintains root access to all servers. Using SSH, the CloudSpecs application is installed on to the servers. Both Primary and Secondary servers are
located within the same virtual private network.
Once the CloudSpecs performance application is installed, it automatically runs 4 times a day, 365 days a year.
PERFORMANCE TESTS
Cloud Spectator runs basic, open source benchmark tests that are precise and accurate for measuring the performance of a cloud server. Tests included come from
the Unixbench Suite (https://code.google.com/p/byte-unixbench) and Phoronix Test Suite (http://www.phoronix-test-suite.com).
The benchmarks for network tests and the disk benchmark for local file copy may be reproduced using the Linux command line interface. The network tests assume
$SERVER is the IP of the secondary server and $CLIENT is the IP of the primary server. Command line instructions for the internal network tests are included below
in each of the tests within the network description.
DATA COLLECTION
The CloudSpecs test results for this publication run from October 17, 2012 to November 16, 2012 (a total of 30 days). Cloud Spectator obtains cloud servers by either
purchasing the server space directly from the client, or the client reimburses Cloud Spectator for data collected relevant to that active project.
CLOUDSPECS SCORE
The CloudSpecs Score represents the true value a cloud user receives when using a server. A higher score means better value. The highest value
performer receives a score of 100 and other providers are pegged in relativity to value. Value is calculated based on performance over time and the cost of the
Primary Server hourly (730 hours per month).
Provider
Amazon EC2 East
Rackspace
Lunacloud
Cost per Hour
$0.64
$0.90
$0.32
Below is the formula process for calculating the CloudSpecs Score (Provider’s CloudSpecs Score is the calculation for the final CloudSpecs score of each
provider). The CloudSpecs score is rounded off to the nearest whole number. The CloudSpecs score in this report were calculated over a period of 30 days:

provider_value(P) = [Provider test score over a period of time] / [Provider price]

best_provider_value = max(provider_values)

Provider’s CloudSpecs Score = 100 * provider_value(P) / best_provider_value
The Sectional CloudSpecs Score is calculated by averaging the CloudSpecs scores of each section’s available tests.
3

4. PERFORMANCE SUMMARY
General Server Performance
UnixBench (Score)
Provider
Lunacloud
Amazon
Rackspace
30-day Average
1,305
1,037
955
CloudSpecs Score
100
40
26
CPU Performance
7-Zip Compression (MIPS)
Provider
Lunacloud
Amazon
Rackspace
30-day Average
4,498
6,777
6,152
Video Encoding Using x264 (FPS)
CloudSpecs Score
100
75
49
Provider
Lunacloud
Amazon
Rackspace
30-day Average
29
52
47
CloudSpecs Score
100
90
58
Linux Kernel Compilation (Seconds to Compile)
Audio Encoding Using LAME MP3 (Seconds to Compile)
Provider
Lunacloud
Amazon
Rackspace
Provider
Lunacloud
Amazon
Rackspace
30-day Average
449
234
251
CloudSpecs Score
100
96
64
30-day Average
25
31
32
CloudSpecs Score
100
48
28
Apache Kernel Compilation (Seconds to Compile)
Provider
Lunacloud
Amazon
Rackspace
30-day Average
69
67
63
CloudSpecs Score
100
55
37
Disk Performance
DBench (MiB/sec)
Provider
Lunacloud
Amazon
Rackspace
30-day Average
399
129
95
Iozone (MiB/sec)
CloudSpecs Score
100
16
8
Provider
Lunacloud
Amazon
Rackspace
30-day Average
1,871
960
716
CloudSpecs Score
100
26
14
File Copy (MiB/sec)
Provider
Lunacloud
Amazon
Rackspace
30-day Average
754
26
39
CloudSpecs Score
100
2
2
RAM Performance
STREAM (MiB/sec)
Provider
Lunacloud
Amazon
Rackspace
30-day Average
11,114
15,497
6,648
RAMSpeed SMP (MiB/sec)
CloudSpecs Score
100
70
21
Provider
Lunacloud
Amazon
Rackspace
30-day Average
7,665
7,676
5,501
CloudSpecs Score
100
50
25
4

5. Internal Network Performance
Network Throughput with Iperf (Mibit/sec)
Apache WGET (MiBit/sec)
Provider
Lunacloud
Amazon
Rackspace
Provider
Lunacloud
Amazon
Rackspace
30-day Average
767
384
396
CloudSpecs Score
100
25
18
FTP to RAM (MiBit/sec)
Provider
Lunacloud
Amazon
Rackspace
30-day Average
916
446
187
30-day Average
933
279
187
CloudSpecs Score
100
31
8
Nginx WGET (MiBit/sec)
CloudSpecs Score
100
24
7
FTP to Disk (MiBit/sec)
Provider
Lunacloud
Amazon
Rackspace
30-day Average
864
528
188
Provider
Lunacloud
Amazon
Rackspace
30-day Average
891
517
187
CloudSpecs Score
100
29
7
Network SCP (MiBit/sec)
CloudSpecs Score
100
15
7
Provider
Lunacloud
Amazon
Rackspace
30-day Average
169
118
184
CloudSpecs Score
100
35
39
5

6. GENERAL SERVER PERFORMANCE
Cloud Spectator runs the UnixBench benchmark suite to gauge overall performance of a cloud server. Over the test period of 30 days, Lunacloud outperforms
Amazon and Rackspace in the UnixBench test.
UNIXBENCH
Description
The purpose of UnixBench is to provide a basic indicator of the performance of a Unix-like system; hence, multiple tests are used to test various aspects of the
system's performance. These test results are then compared to the scores from a baseline system to produce an index value, which is generally easier to handle
than the raw scores. The entire set of index values is then combined to make an overall index for the system.
Why It Matters
The UnixBench is a suite of approximately 10 tests that can measure how many instructions per second a CPU can perform. If CPU A performs 2x better than CPU B
at only 1.5x the cost, then CPU A provides more performance per unit of cost.
Provider
Lunacloud
Amazon
Rackspace
30-day Average
1,305
1,037
955
CloudSpecs Score
100
40
26
6

7. CPU PERFORMANCE
7ZIP COMPRESSION
Description
This is a test of 7-Zip using p7zip with its integrated benchmark feature, measured in million instructions per second (MIPS). The test consists of compressing a file
with random data using the 7-zip program and then dividing the number of CPU instructions executed during the compression by the number of seconds. The result
is then divided by 1 million to calculate the value in MIPS.
Why It Matters
The 7-zip compression test results are important in comparison because the 7-zip compression algorithm uses many kinds of CPU instructions, resulting in an
accurate overall score reflecting the performance of the CPU. The fact that it is a compression program is irrelevant in the test.
Provider
Lunacloud
Amazon
Rackspace
30-day Average
4,498
6,777
6,152
CloudSpecs Score
100
75
49
TIMED LINUX KERNEL COMPILATION
Description
This test times how long (in seconds) it takes to build the Linux kernel. Unlike the 7-zip compression, this is a compilation source code. Though many compilation
tests exist, an advantage of the Linux Kernel compilation is the portability and compatibility with a majority of Linux distributions.
Why It Matters
Because of the variety of CPU instructions and the length of time for the test, the Linux kernel compilation can accurately measure a CPU’s performance.
Provider
Lunacloud
Amazon
Rackspace
30-day Average
449
234
251
CloudSpecs Score
100
96
64
7

8. TIMED APACHE COMPILATION
Description
This test times how long (in seconds) it takes to build the Apache HTTP Server. Though many compilation tests exist, an advantage of the Apache compilation is the
portability and compatibility with a majority of Linux distributions.
Why It Matters
Because of the variety of CPU instructions and the length of time for the test, the Apache compilation can accurately measure a CPU’s performance.
Provider
Lunacloud
Amazon
Rackspace
30-day Average
69
67
63
CloudSpecs Score
100
55
37
X264 VIDEO ENCODING
Description
The x264 test benchmarks the speed it takes a server to encode a movie file using the x264 H.264/AVC encoder.
Why It Matters
When results are shown alone, they have little meaning, but the process requires a large amount of computations, so when compared side-by-side using results from
several cloud providers, the test provides an accurate insight of CPU performance.
Provider
Lunacloud
Amazon
Rackspace
30-day Average
29
52
47
CloudSpecs Score
100
90
58
8

9. LAME MP3 ENCODING
Description
LAME is an MP3 encoder licensed under the LGPL. This test measures the time (in seconds) required to encode a WAV file to MP3 format.
Why It Matters
Similar to the x264 test, results alone have little meaning, but the process requires a large amount of computations, so when compared side-by-side using results
from several cloud providers, the test provides accurate insight of CPU performance.
Provider
Lunacloud
Amazon
Rackspace
30-day Average
25
31
32
CloudSpecs Score
100
48
28
9

10. DISK PERFORMANCE
DBENCH
Description
DBench is a benchmark designed by the Samba project as a free alternative to netbench, but dbench contains only file-system calls for testing the disk performance.
Why It Matters
DBench can be used to stress a filesystem or a server to see which workload it becomes saturated and can also be used for prediction analysis to determine how
many concurrent clients/applications performing this workload the server can handle before response starts to lag.
Provider
Lunacloud
Amazon
Rackspace
30-day Average
399
129
95
CloudSpecs Score
100
16
8
FILE COPY
Description
This measures the performance of the operating system's 'cp' command, followed by a 'sync' (to flush any cache), executed on the local storage device. The file size
is 10GB, and is copied to the same disk measured in mebibytes per second.
Why It Matters
File copy measures read and write performance of the disk, which is important when processing massive amounts of data.
Provider
Lunacloud
Amazon
Rackspace
30-day Average
754
26
39
CloudSpecs Score
100
2
2
10

11. IOZONE FILESYSTEM BENCHMARK
Description
Iozone benchmarks file IO performance of a server for the operations: Read, write, re-read, re-write, read backwards, read strided, fread, fwrite, random read,
pread ,mmap, aio_read, aio_write
Why It Matters
Iozone can collect data on a broad range of file system performance to locate hot or cold spots in the file systems.
Provider
Lunacloud
Amazon
Rackspace
30-day Average
1,871
960
716
CloudSpecs Score
100
26
14
11

12. RAM TESTS
STREAM
Description
The STREAM benchmark is a simple synthetic benchmark program that measures sustainable memory bandwidth (in MB/s) and the corresponding computation rate
for simple vector kernels. The STREAM benchmark is specifically designed to work with datasets much larger than the available cache on any given system, so that
the results are (presumably) more indicative of the performance of very large, vector style applications.
Why It Matters
STREAM measures memory bandwidth to see how the RAM accommodates to the processes running on the server.
Provider
Lunacloud
Amazon
Rackspace
30-day Average
11,114
15,497
6,648
CloudSpecs Score
100
70
21
RAMSPEED SMP
Description
This benchmark tests the system memory (RAM) performance.
Why It Matters
RAMspeed results correlate highly with STREAM results. Though the algorithms are different, RAMspeed SMP also measures the RAM accommodation to
processes running on the server.
Provider
Lunacloud
Amazon
Rackspace
30-day Average
7,665
7,676
5,501
CloudSpecs Score
100
50
25
12

13. NETWORK PERFORMANCE TESTS
NETWORK THROUGHPUT WITH IPERF
Description
Iperf is a commonly used network testing tool that can create TCP and UDP data streams and measure the throughput of a network that is carrying them. Our test
consists of transferring as much data as possible through the local network (where possible, private network IPs were used) for 120 seconds over TCP port 5001.
Why It Matters
Iperf tests the quality of the network. Using this test between two cloud servers in the same data center of the same provider, results can be collected on how efficient
the internal network speed is, which is important for servers inside the same virtual private network that may be required to transfer data between or among each
other quickly and efficiently to effectively run applications.
Code
wget 'http://downloads.sourceforge.net/project/iperf/iperf2.0.5.tar.gz?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fiperf%2F&ts=1341586993&use_mirror=ignum' -O iperf.tgz
tar zxvf iperf.tgz
cd iperf-2.0.5
./configure
make
./src/iperf -s -p 5001
wget 'http://downloads.sourceforge.net/project/iperf/iperf2.0.5.tar.gz?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fiperf%2F&ts=1341586993&use_mirror=ignum' -O iperf.tgz
tar zxvf iperf.tgz
cd iperf-2.0.5
./configure
make
./src/iperf -c $SERVER -p 5001 -t 120
Provider
Lunacloud
Amazon
Rackspace
30-day Average
767
384
396
CloudSpecs Score
100
25
18
13

14. NETWORK FTP TO RAMDISK
Description
Using vsftpd (Very Secure FTP Daemon) on Server and lftp on Client, the test transfer a 1GB file from RAM drive to RAM drive.
Why It Matters
Transmitting from a RAM disk bypasses the disk in the network test, preventing a possible bottleneck at the disk level and enabling a more accurate measurement of
the internal network transfer speed.
Code
mkdir -p /mnt/ramfs
chmod 0555 /mnt/ramfs
mount -t ramfs -o size=1050mb ramfs /mnt/ramfs
dd if=/dev/zero of=/mnt/ramfs/file.big bs=1MiB count=1024
time lftpget -d ftp://$SERVER/file.big
Provider
Lunacloud
Amazon
Rackspace
30-day Average
916
446
187
CloudSpecs Score
100
24
7
14

15. NETWORK FTP TO HARD DISK
Description
Using vsftpd (Very Secure FTP Daemon) on Server and lftp on Client, the test transfer a 5GB file from hard disk to hard disk.
Why It Matters
The File Transfer Speed of the Disk is similar to the FTP speed of the RAM disk, but in a manner that measures the speed efficiency of a disk-to-disk transfer, rather
than RAM-to-RAM.
Code
dd if=/dev/zero of=/tmp/file.big bs=5MiB count=1024
time lftpget -d ftp://$SERVER/file.big
Provider
Lunacloud
Amazon
Rackspace
30-day Average
933
279
187
CloudSpecs Score
100
15
7
15

16. NETWORK HTTP Apache
Description
Using apache2 as Server and wget as Client, transfer of a 1 GB file.
Why It Matters
Performing a more natural test, the Apache Network HTTP, Nginx Network HTTP, and Network SCP tests all combine the internal network with CPU usage to see
how fast servers perform when they have added CPU usage.
Code
dd if=/dev/zero of=/tmp/file.big bs=1MiB count=1024
time wget http://$SERVER/file.big
Provider
Lunacloud
Amazon
Rackspace
30-day Average
864
528
188
CloudSpecs Score
100
31
8
16

17. NETWORK HTTP NGINX
Description
Using nginx (a web server application) as server and wget as Client, transfer of a 1 GB file.
Why It Matters
Performing a more natural test, the Apache Network HTTP, Nginx Network HTTP, and Network SCP tests all combine the internal network with CPU usage to see
how fast servers perform when they have added CPU usage.
Code
dd if=/dev/zero of=/tmp/file.big bs=1MiB count=1024
time wget http://$SERVER/file.big
Provider
Lunacloud
Amazon
Rackspace
30-day Average
891
517
187
CloudSpecs Score
100
29
7
17

18. NETWORK SCP
Description
Copy of a 5GB file using Secure Copy (SCP).
Why It Matters
Performing a more natural test, the Apache Network HTTP, Nginx Network HTTP, and Network SCP tests all combine the internal network with CPU usage to see
how fast servers perform when they have added CPU usage.
Code
dd if=/dev/zero of=/home/guest/file.big bs=5MiB count=1024
time scp guest@$SERVER:file.big .
Provider
Lunacloud
Amazon
Rackspace
30-day Average
169
118
184
CloudSpecs Score
100
35
39
About Cloud Spectator
Cloud Spectator is the premier cloud analyst group focused on infrastructure pricing and server performance. Since 2011, Cloud Spectator has monitored the cloud
Infrastructure industry on a global scale and continues to produce research reports for businesses to make informed purchase decisions by leveraging its
CloudSpecs utility, an application that automates live server performance tests 4 times a day, 365 days a year with use of open source benchmark tests. The data is
compared relatively with pricing from each provider to allow business decision-makers to smart decisions.
Cloud Spectator
800 Boylston Street
16TH Floor
Boston, MA 02199
Website: www.cloudspectator.com
Phone: (USA) 1-617-300-0711
18