How to make the most out of "Utility" computing. Paul Byrom, Head of Quality, News International

1. News International
How to look for a service provider with an agile, resilient
infrastructure for your online assets?
Paul Byrom, Head Of Quality

2. Utility Computing
Who is this guy
Paul Byrom - Head Of Quality - News International
- In charge of Quality Assurance across digital projects at News International.
What do we test
All major titles across web and mobile apps such as:
The Times
The Sun
The Sunday Times
What is Utility Computing
- Think about it like how you pay for your electricity based on the consumption you use. Or as
wikipedia says computation, storage and services, as a metered service.

3. Problem & Challenge
Problem Statement @ NI
- Our load testing was no longer valid as we could not afford to run
live like load tests.
The Challenge
- To reduce risk of failure by performing representative load tests at
minimal cost, and minimal impact.
- To fail fast and provide useful feedback that allows the team to
easily investigate and fix potential bottlenecks caused by load.

4. Where we were
- Very expensive Load Runner. HP wanted 121k to allow us to run 1000 VUs.
- Limited virtual users due to cost
- A ton of tin (environments) on-site
- Performance testing occurred too late to make positive change
- Load Runner experts are expensive.
- Existing tool cloud model not very flexible or cost effective.
- Couldn't run projects in parallel on perf env causing a queue (shared VUs)
- Difficult to integrate into a CI system (cmd line support, results in HP format)
DEV SI UAT STAGE PERF PROD

5. Where we are now
- Decommissioned old perf env
- Jmeter is a free open source tool which is more adaptable to our needs
- Unlimited virtual users allowing us to perform live like load tests
- Presentation layer enables quick comparison of data for rapid resolution
- Automated script creates, runs test, and destroys test hardware for each run in the cloud.
- Failing faster as perf testing now occurs on CI, SI, UAT, and STAGE
CI
DEV SI UAT STAGE PROD
AWS
Load

6. The Proof
- Using 30 EC2 instances last year, DreamTeam tested with 10000
virtual users against the website and achieved 8000 hits per second
(600 page views/second).
- That would have previously costed £1.2mil to run the test on Load
Runner.
- It cost us $17.00 using Jmeter and AWS.
- Currently Cost is about £70/mon right now across all products for
AWS load injector costs.

7. What's Next
- Trending as data is now all in one place
- Standardised reporting across all products
- Continuous automated load testing

8. Jmeter Presentation Layer