This document summarizes the results of performance testing Azure Web Sites hosting plans. It finds that the Shared plan can scale well if usage quotas are not exceeded. Scaling out by adding multiple smaller instances, such as 3 Basic Medium instances, provides better performance and value than a single larger instance like Standard Large. Migrating the database to Azure SQL significantly improves page load times and ability to handle load compared to using a local SQL Compact database. Overall, scaling out instances and optimizing the application are more cost effective ways to improve performance than simply upgrading to larger VM sizes.

1. Microsoft Azure Web Sites
Performance Testing
Lessons Learned
Christopher Woodill
cwoodill@hotmail.com
@microsofttrend
www.microsofttrends.com
December, 2014

2. What is Azure Web Sites?
Azure Websites is a fully
managed Platform-as-a-
Service (PaaS) that enables
you to build, deploy and
scale enterprise-grade web
Apps in seconds.
Azure Web Sites is Available in Different
Hosting Plans:
Free
Shared
Basic (Small, Medium and Large)
Standard (Small, Medium and Large)

3. Azure Web Site Pricing (Dec 2014)
Hosting Plan Features ~ Price per Month
Free 5 connections, shared VM, no SSL, 1 GB disk space, lots of usage quotas $0
Shared 35 connections, 1 GB disk space, shared VM, SSL support, 100 web sites per plan, some usage quotas, custom domain
support, no SLA, scale up to 10 instances
$10
Basic 350 connections, 500 web sites per plan, dedicated VM, 64 bit, custom domain SSL support, scale up to 3 instances, 10 GB
storage, no usage quotas, 99.9% SLA.
Basic Small 1 Core, 1.75 GB RAM $60
Basic Medium 2 Cores, 3.5 GB RAM $119
Basic Large 4 Cores, 7 GB RAM $235
Standard Unlimited connections, 500 web sites per plan, dedicated VM, 64 bit, custom domain SSL support, scale up to 10 instances,
50 GB storage, no usage quotas, 99.9% SLA.
Standard Small 1 Core, 1.75 GB RAM $79
Standard Medium 2 Cores, 3.5 GB RAM $157
Standard Large 4 Cores, 7 GB RAM $314
Prices subject to change…

4. Testing Methodology
 Implemented the ASP.NET Bakery Site in two configurations:
 Out of the box using SQL Compact (e.g. File system based database) (average page load
time unloaded is 300 ms)
 Migrated database to Azure SQL (average page load time unloaded is 29 ms)
 Apache Jmeter used to create load test to simulate requests for home page and product
pages
 Tested 1, 2, 4, 8, 12, 20, 50, 100, 200 and 500 concurrent users on all Azure Web Site scale
up options
 Tested scale out with 3 x Shared, 3 x Basic Medium and 3 x Standard Medium instances
 In each test, ran 300 page requests per thread or until ~30,000 page views

5. Jmeter Example

6. New Relic for Monitoring
Web Requests Queuing Up…

7. Planning for Performance
 What is your unloaded web page performance? 30 ms or 300 ms?
 How many average concurrent users do you expect? 4 or 20 or 200?
 How many peak concurrent users do you expect? 10 or 50 or 200?
 How long do you expect to run at peak? 5 minutes or 1 hour or 12 hours?
 What is an acceptable maximum page load time? 1 second? 3 seconds? 10 seconds?
 What is your target hosting spend per month?

8. Calculating Projected Page Views
Maximum Page Views per Day =
8,640,000 ms per day / average page load time * number of concurrent users.
For example, if we can serve 200 concurrent users a page every 27 ms that means we
have capacity to serve 675 million page views per day!

9. Results: Bakery Running SQL Compact
Average Page Load Speed (ms) Threads
Hosting Plan 1 2 4 8 12 20 50 100 200
Free 384 344 343 324 360 346 713 Fail Fail
Shared 399 353 345 338 383 350 404 1290 1788
Basic Small 427 434 795 1542 2246 3802 Fail Fail Fail
Basic Medium 410 387 412 415 445 655 1133 6600 11700
Basic Large 329 299 293 308 303 403 650 1543 2500
Standard Small 366 428 880 1816 2502 4341 10061 Fail Fail
Standard Medium 318 312 273 287 369 492 1212 5147 5024
Standard Large 319 292 275 288 272 350 741 1435 2644

10. Bakery Running SQL Compact
0
2000
4000
6000
8000
10000
12000
ONE TWO FOUR EIGHT TWELVE TWENTY FIFTY ONE HUNDRED TWO HUNDRED
AveragePageLoadTime(ms)
Azure Web Site Average Page Load (SQL Compact)
Free
Shared
Standard Small
Standard Medium
Standard Large

11. Results: Bakery Running SQL Azure
Average Page Load Speed (ms) Threads
Hosting Plan 1 2 4 8 12 20 50 100 200 500
Shared 42 36 29 30 26 28 29 36 49Fail
Basic Small 35 34 34 34 49 93 270 500 1115 2843
Basic Medium 30 26 26 29 29 40 108 251 534 1320
Basic Large 27 27 27 28 32 37 116 237 540 1367
Standard Small 29 25 32 35 70 136 356 717 1351 3000
Standard Medium 26 25 29 30 32 34 107 215 563 1300
Standard Large 26 26 26 27 28 31 55 118 200 620
Shared 3 Instances 45 28 30 27 28 27 27 26 27 64
Basic Medium x 3 Instances 28 27 27 27 29 29 33 53 68 124
Standard Large x 3 Instances 30 27 26 27 26 27 32 31 36 40

12. Bakery Running Azure SQL
0
500
1000
1500
2000
2500
3000
3500
ONE TWO FOUR EIGHT TWELVE TWENTY FIFTY ONE
HUNDRED
TWO
HUNDRED
FIVE
HUNDRED
AveragePageLoadTime(ms)
Azure Web Site Average Page Load (Azure SQL)
Shared with SQL
Standard Small with SQL
Standard Medium with SQL
Standard Large with SQL

13. Bakery with Scale Out
0
20
40
60
80
100
120
140
ONE TWO FOUR EIGHT TWELVE TWENTY FIFTY ONE
HUNDRED
TWO
HUNDRED
FIVE
HUNDRED
AveragePageLoadTime(ms)
Azure Web Site Average Page Load with Scaled Out Instances (Azure SQL)
Shared with SQL
Shared with SQL x 3 Instances
Basic Medium with SQL x 3 Instances
Standard Large with SQL x 3 Instances

14. Maximum Page Views per Hosting Plan
Hosting Plan Price
Maximum Page Views Per
Day with Azure SQL
~ Cost Per Million
Page Views
Maximum Page Views Per Day
with SQL Compact
~ Cost per Million
Page Views
Shared with SQL $10 240,000,000* $0.0054 10,693,069* $0.031
Basic Small with SQL $60 17,280,000 $0.1157 461,620 $4.333
Basic Medium with SQL $119 43,200,000 $0.0918 2,638,167 $1.504
Basic Large with SQL $235 36,455,696 $0.2149 4,287,841 $1.827
Standard Small with SQL $79 12,705,882 $0.2073 414,388 $6.355
Standard Medium with SQL $157 50,823,529 $0.1030 3,564,356 $1.468
Standard Large with SQL $314 55,741,935 $0.1878 4,937,142 $2.120
Shared with SQL x 3 $30 675,000,000* $0.0015
Basic Medium with SQL x 3 $357 348,387,096 $0.0342
Standard Medium with SQL x 3 $471 360,000,000 $0.0436
Standard Large with SQL x 3 $942 1,080,000,000 $0.0291
* Theoretical maximum only, will likely overload quota limits before achieving this target.

15. Real World: WordPress on Azure Web Sites
Average Page Load Speed (ms) Threads
Hosting Plan 1 2 4 8 12 20 50 100
Shared 1111 986 Fail Fail Fail Fail Fail Fail
Basic Small 1131 1617 2911 5890 8850 14700 35000 Fail
Basic Medium 1139 1155 1485 2825 4185 4268 17000 Fail
Basic Large 1021 1063 1012 1445 2186 3743 9000 17300
Shared x 3 1063 1122 1065 999 Fail Fail Fail Fail
Basic Medium x 3 1132 1119 1219 1355 1650 2377 5747 0
Standard Large x 3 1116 1127 1259 1050 1130 1250 3059 6253

16. Key Conclusions: Performance
 Baseline performance of page makes a big difference in ability to scale and the size of hosting plan
required to maintain performance under load.
 Azure SQL ran flawlessly without any bottlenecks under all scenarios – web server was always slower
than the DB.
 Shared works awesome – it scaled better than Standard Large! However, it can be quickly overloaded
because of quota restrictions.
 Shared x 3 instances means also 3x the usage quotas – good approach to handling sites that need
additional CPU or memory.
 Performance of Basic and Standard are similar – you’re paying more for features, not performance.
 Basic Small and Standard Small are too small to scale – performance degrades very quickly.
 Scaling out number of instances improves load better than scaling up to the next size of VM. Basic
Medium x 3 performed significantly better than Standard Large x 1.

17. Key Conclusions: Economics
 Even running SQL Compact on Basic Small, we could still crank out more than 400K page views per day!
 Baseline performance makes a big difference in $$ / million page views. Investments in optimization can lower your
hosting costs by allowing you to run leaner and cheaper.
 Only use what you absolutely need to sustain your traffic – you can always scale up to a more expensive plan and you
only pay for when you’re using the higher performance.
 Clear winner in delivering pages per $$ is Shared hosting plan, if you can live within the quotas.
 Shared x 3 at $30 / month scales much better than a single Standard Large at $314.00 / month and quotas are tripled!
 Basic / Standard Medium more economical than Basic / Standard Small under load because they can scale better for
the price.
 Scaling out instances is cheaper than scaling up to a bigger VM – e.g. Basic Medium X 3 instances performed much
better than a single Standard Large VM and is only a little bit more expensive ($357 / month vs. $314 / month).
 Scaling out instances is also better because it supports auto-scaling. No auto-scaling support for moving up and down
sizes of VMs.

18. Thanks!
Christopher Woodill
 cwoodill@hotmail.com
 @microsofttrend
 www.microsofttrends.com