1. +
People don’t know what they want until you show
it to them.
Why “Average” Response Time is not a right
measure of your web application's performance?

2. +For the next 15 minutes
n Web Application Performance Testing
n Average Response Time
n Solution to Average Response Time
n Beyond Average Response Time
n Questions

3. +Web Application Performance Testing

4. +
Web Application Performance Testing
How to do
Design Test/
Scenarios
Establish
Performance
Acceptance
Criteria
Identify
Environment
Execute Test
Analyze
Reports / Tune/
Retests
What we
collect
Response Time
Throughput
CPU Utilization
Memory

5. +Average Response Time
n Response Time is defined as the time it takes for each Web page
to load.
n How we capture?
n Hit the server with a specified number of user (could be in pattern)
and then capture the response time of the server.
n Refer the plot of ART on next page.

6. +Average Response Time
0
100
200
300
400
500
600
Page 1 (in ms)
Page 2 (in ms)
Page 3 (in ms)

7. +Let’s take a real life scenario…
n Consider a typical web application
n Load Scenario (1):
n 60% of requests | 300ms
n 20% of requests |10ms
n went through real quick.
n mostly served from cache.
n 20% of requests | 10 sec
n Requests stuck up due to external api’s or
n let’s say some DB locks.
n Average Response Time: 2.182 sec

8. +Fallacy - #1
What’s wrong with 2.182 sec?
n Bad Indicator of user “satisfaction” given that 20% of your
users are unhappy with your site.
n 80% of users who are facing fast response time can’t
compensate the others who are leaving the website.
n 20% of users are very critical depending on which part of
application they are on.
n E.g. Sign Up, Log In, Check out
Learning: It hides important outliers.

9. +Fallacy - #2
Increasing complexity of website Huh !
n Websites are little complicated these days. Different pages have
different normal response time.
n Login thru LinkedIn API taking 5 sec is ok.
n A kernel cached image taking 5 sec to serve is not ok.
n Averaging these two together wouldn’t really let me know
anything.
n It’s unfair to compare the Average Response Time of a page over
time.
n Every time the component of the traffic changes, it will have
impact on ART.
Learning:Your Average Response Time is not a fixed number.

10. +Fallacy - #3
Averages are meant more for averaging the total quantity
being measured, not the number of samples !
n Load scenario (2):
n 10 request | ~300ms
n 1 request | 5 minute
n hit triggered a cache miss, which got stuck in some almost
never hit DB query.
n Average Response Time spiked to 27.3 sec !
n First impression “Something is seriously wrong!”
Learning:Your average response time never tells you the number
of users affected by the problem.

11. What we need?

12. + Apdex
n Created by Peter Sevcik of Apdex Alliance (apdex.org).
n How is it calculated?
n ApdexT = (# Satisfied requests + # Tolerating requests / 2)
(# Total requests)
APDEX: Application Performance Index

13. +
How it scores over Average Response
Time
n Load Scenario(1). 60% of requests take 300ms, 20% take 10ms, and
20 percent are getting unacceptable latencies at 10 seconds.
n Average Response Time = 2.182 sec; Wrong Confidence
n Apdex2s is 80%; Pin points the problem
n Load Scenario(2). 10 requests takes 300ms and 1 request takes 5
minute.
n Average Response Time = 27.3 seconds; False Alarm
n Apdex2s is 90.9 % ; Good health
ü It does a good job.
ü Indicates correct health of application.
ü Highlights even when the small percentage of user face
performance issue (so called outliers).

14. + One last thing…
Load Testing
Stress
Testing
Soak Testing
Performance
Tuning

15. +