121 Test Automation Day
Dublin, May 23rd 2018
https://1point21gws.com/testingsummit/dublin/
Stating the obvious: adding performance and scalability tests to a Continuous Integration pipeline
Performance and scalability are core quality attributes of any system; unit testing, integration testing, UI testing, they all focus on functional requirements. Good performances mean happy users, less resource usage which translates to lower running costs (power, cloud bills) and customer retention.
In this session we will recall some basic concept of performance testing and demonstrate some of the many tools available in the cloud.
Stating the obvious - 121 Test Automation Day, Dublin, 2018
1. Stating the Obvious
Adding Performance and Scalability Tests
to a Continuous Integration Pipeline
121 Test Automation Day
Dublin, 23 May 2018
Giulio Vian
DevOps Lead
Glass Lewis & Co.
6. Performance is
a Feature
«there are two
kinds of websites:
the quick and the
dead»
https://blog.codinghorror.com/performance-is-a-feature/
7. Response time
0.1 second is about the limit for having the user
feel that the system is reacting instantaneously,
meaning that no special feedback is necessary
except to display the result.
1.0 second is about the limit for the user's flow of
thought to stay uninterrupted, even though the
user will notice the delay. Normally, no special
feedback is necessary during delays of more than
0.1 but less than 1.0 second, but the user does lose
the feeling of operating directly on the data.
10 seconds is about the limit for keeping the user's
attention focused on the dialogue. For longer
delays, users will want to perform other tasks
while waiting for the computer to finish, so they
should be given feedback indicating when the
computer expects to be done. Feedback during
the delay is especially important if the response
time is likely to be highly variable, since users will
then not know what to expect.
Miller, R. B. (1968).
Response time in man-
computer
conversational
transactions. Proc. AFIPS
Fall Joint Computer
Conference Vol. 33, 267-
277.
8. Hard facts
Google
Half a second delay
caused a 20% drop in
traffic.
Amazon.com
Even very small delays
(100 ms) would result in
substantial and costly
drops in revenue.
43. Bibliografy (2)
Writing High-Performance .NET Code —
Ben Watson (Ben Watson)
https://www.amazon.it/Writing-High-
Performance-NET-Code-
Watson/dp/0990583430/
Time Is Money: The Business Value of
Web Performance — Tammy Everts
(O'Reilly Media)
https://www.amazon.com/Time-Money-
Business-Value-Performance/dp/1491928743
44
44. Bibliografy (3)
Software Performance and Scalability: A
Quantitative Approach — Henry H. Liu
(Wiley)
https://www.amazon.com/Software-
Performance-Scalability-Quantitative-
Approach/dp/0470462531
Continuous Delivery with Windows and .NET
— Matthew Skelton and Chris O'Dell
(O'Reilly)
http://www.oreilly.com/webops-
perf/free/continuous-delivery-with-windows-and-
net.csp
45
Performance is one of those areas that you could spend an infinite amount of time on, so failing to recognise it as a business interest could mean either chronic under-investment in it – or massive amounts of invisible over-spending if the technical team have placed too much emphasis on it. – Steve Fenton
https://www.stevefenton.co.uk/2016/06/performance-is-a-feature/