1. Performance &
Load Testing
Stéphane Landelle
CTO
eBusiness Information
@slandelle

2. Concepts

3. Define performance?
● fast?
● robust?
● resource effective?
=> Define your requirements!

4. Web speed is a user experience
● <0,5 s : loading looks instantaneous
● 0,5-2 s : not instantaneous, but acceptable
● > 3 s : users start leaving your site

5. Put things into perspective
Product Owner requirement :
Every page should show up under 200ms
WRONG !
User expectations = per use case!

6. Speed matters
● More traffic
● Better conversion rate
● Less angry users

7. Some figures
Google :
Results/page : 10 => 30
-> +500 ms
-> -20% pages seen
Amazon :
+100 ms
-> -1 % sales
estimated annual cost : $160M

8. Load tests : why ?
Load tests
=
know your app & infrastructure
No load tests
=
potential problems in production
=
angry users

9. Methodology
Clicking everywhere/Selenium = not a load test !
● You expect more than 1 user
● "Seems fast enough." => Not a metric !

10. Methodology
Comes after proper local perf testing
● Client/Network optimization: minification, javascript,
sprites, caching...
○ Tools : Developer Tools, YSlow, Google PageSpeed
Insights...
● Application debugging: 50 SQL queries/page = BUG!
○ Tools : VisualVM, Yourkit, JProfiler, MAT...

11. Methodology
● Analyze = Data = Monitoring
● Fix hotspot
● Iterate!

12. Tooling
● Load injector: Gatling, JMeter, Locust.io...
● JVM monitoring: JMXtrans, Yammer Metrics...
● System monitoring: Nagios
● Network monitoring
● Database monitoring
● Dashboard: Graphite, Ganglia
● Webapp mock: H. Gomez's basic perf webapp

13. Types of load tests
● Capacity Test
● Stress Test
● Endurance Test

14. Capacity Test
Goal :
Determine how much load your system can hold
How :
Repeat the same scenario over and over, but add virtual
users every time until performance starts degrading.

15. A response time under 1 s is expected
-> user cap : 1500 users

16. Stress Test
Goal :
Study system behavior in case of heavy load, during AND
after
How :
Find the max load your system can handle, and then run
the scenario with a heavier load.

17. 10k users for 1 min, then 1k :
webapp struggles, but stabilizes

18. Endurance Test
Goal :
Validate system behavior after a long period of activity.
How :
Run the scenario with a manageable load, but for a long
period of time (several hours at least).

19. Fast memory leak :
Runs fine for 2 minutes until heap's full...

20. Ramps
Start virtual users progressively, because that's what real
users do !
Ramps also help to warm up your system.

21. Reports
The purpose of load injectors : stress your app and
produces reports
● Meaningful reports help developers analyze stress tests
results and what to make of them
● Something shiny to give to your boss

22. The good, the bad and the ugly
metrics
Every metric can be useful, but some less than others...
● Response time min/max = worst case/best case
● Mean can be biased in case of extreme values
Response time is a physical phenomenon
=> Statistically distributed

24. Percentiles to the rescue
nth percentiles
=
n % of users' response time

25. Part of the development process
Like any other functional test, load testing should be :
● integrated early in the development cycle
● automated
● versioned

26. Gatling

27. Yet Another Stress Tool
JMeter, Grinder, Tsung,
LoadUI,
LoadRunner, Neoload…

28. Issue #1
High Performance
http://www.shopfbparts.com/catalog/nal-19201331_w.jpg

29. 1 user = 1 thread

30. With 50 threads on a JVM

31. With 2000 threads on a JVM

32. Blocking I/O

33. Threads? Waiting…

34. … and
sleeping

35. Is that a real problem?

36. Can you trust your results?
JMeter 2.8 perf test, expecting 300 tr/sec

37. Issue #2
Usability

38. Graphical User Interface
Listen, it's not
that
complicated...

39. Issue #3
Maintainability

40. What was this change about?

41. Gatling can
change all
that!
http://static.lexpress.fr/medias/15/mai-68_124.
jpg, copyright by AFP

42. Say hello to my little friend…
Version 1.4.1
Released January 2013

43. Be asynchronous,
embrace the actor model

44. Use non-blocking I/O
● Async HTTP Client
● Netty

45. Scenario
= Code
(Scala)
= DSL

46. Easy

47. Use the rich DSL…
Checks
● regex / css / xpath / jsonPath
● find / findAll / count
● is / in / not / whatever
Structures
● doIf / repeat / during / asLongAs
● randomSwitch / roundRobinSwitch
Error handling
● tryMax / exitBlockOnFail
Feeders
● csv / tsv / jdbc

48. … or write your own Scala
code…

49. …or use the Recorder

50. Integrations
● Maven Plugin
● Maven archetype (run in IDE)
● Jenkins plugin
● Graphite live reporting

51. Coming soon…
● Websockets, JDBC…
● Clustering

52. Demo

53. Gatling at Ezakus

54. Ezakus Architecture

55. Metrics
110 M Http hits by day
Peak : 3 000 req/s

56. Gatling usage - Simulations

57. Gatling usage - Example

58. Gatling usage - Results

59. Gatling usage - Results

60. Conclusion

61. Really efficient?
Jmeter perf test
run with Gatling,

62. http://gatling-tool.org
https://github.com/excilys/gatling
@GatlingTool
https://github.com/slandelle
@slandelle