More Related Content Similar to Stress test your backend with Gatling (20) More from Andrzej Ludwikowski (7) Stress test your backend with Gatling4. Performance tests - why so hard?
Simulate production as close as possible:
● hardware
○ CPU, RAM, storage, ...
● software
○ OS, Virtualization, DBs, …
● load
● isolation
6. Performance tests - why so hard?
● Your performance intuition is wrong!
1. collect the data (monitoring, logging, profiling)
2. find the bottleneck (based on data)
3. fix the bottleneck
4. collect the data and check the assumptions
5. go to 1.
7. Performance tests - why so hard?
Lies, damned lies, and statistics:
● arithmetic mean = 2.9
● median = 1
● standard deviation = 6 (only for normal distribution)
8. Performance tests - why so hard?
● Anscombe's quartet
http://bravenewgeek.com/tag/coordinated-omission/
Property Value
Mean of x 9
Sample variance of x 11
Mean of y 7.50
Sample variance of y 4.125
Correlation between x and y 0.816
Linear regression line y = 3 + 0.5x
Coefficient of determination of
the linear regression
0.67
9. Performance tests - why so hard?
Lies, damned lies, and statistics:
● arithmetic mean = 2.9
● median = 1
● standard deviation = 6 (only for normal distribution)
Use:
percentiles
50th = 1
70th = 1
90th = 2.9
95th = 11.45
99th = 18.29
Check percentiles implementation!
10. Performance tests - why so hard?
● Coordinated omission problem by Gil Tene
http://bravenewgeek.com/tag/coordinated-omission/
20. Coordinated omission problem
total time 60 s
max 30 s
99th 1 s
time in % for max 50%
expected time in % for 99th 50% - 1% = 49%
real time for 99th 60 s * 49% = 29.4 s
22. Performance tests - why so hard?
● Coordinated omission vs Gatling
http://bravenewgeek.com/tag/coordinated-omission/
27. Why Gatling?
● non-blocking, asynchronous stack (scala, akka, netty)
● scala !!!111oneoneone (maven, sbt support)
● DSL
● recorder
● math is good
● reports
● integration & performance tests
28. Gatling DSL
class BasicSimulation extends Simulation {
val httpConf = http
.baseURL("http://computer-database.gatling.io")
.acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
.acceptEncodingHeader("gzip, deflate")
.acceptLanguageHeader("en-US,en;q=0.5")
.userAgentHeader("Mozilla/5.0 (Macintosh; X 10.8; rv:16.0) Gecko/20100101 Firefox/16.0")
val firstScenario = scenario("First Scenario Name")
.exec(http("Request name").get("/"))
.pause(7)
setUp(firstScenario.inject(atOnceUsers(1)).protocols(httpConf))
}
29. Gatling DSL
class FirstScenarioV2 extends MySimulation {
val firstScenario = scenario("First Scenario Name")
.exec(http("Go to root page").get("/"))
.pause(7 seconds)
setUp(firstScenario.inject(atOnceUsers(1)).protocols(httpConf))
}
30. Gatling DSL
import io.gatling.core.Predef._
import io.gatling.http.Predef._
class FirstScenarioV2 extends MySimulation {
val firstScenario = scenario("First Scenario Name")
.exec(http("Go to root page").get("/"))
.pause(7 seconds)
setUp(firstScenario.inject(atOnceUsers(1)).protocols(httpConf))
}
31. Gatling DSL
class ComplexScenario extends MySimulation {
val complexScenario = scenario("Complex demo scenario")
.exec(http("request_1").get("/")).pause(7)
.exec(http("request_2").get("/computers?f=macbook")).pause(2)
.exec(http("request_3").get("/computers/6")).pause(3)
.exec(http("request_4").get("/")).pause(2)
.exec(http("request_5").get("/computers?p=1")).pause(670 milliseconds)
.exec(http("request_6").get("/computers/new")).pause(1)
.exec(http("request_7")
.post("/computers")
.formParam("name", "MyComputer").formParam("introduced", "2012-05-30").formParam("company", "37"))
setUp(complexScenario.inject(atOnceUsers(1)).protocols(httpConf))
}
32. Gatling DSL
class ComplexScenarioV2 extends MySimulation {
val complexScenario = scenario("Complex demo scenario")
.exec(goToRootPage).pause(7)
.exec(searchFor("macbook")).pause(2)
.exec(positionAt(6)).pause(3)
.exec(goToRootPage).pause(2)
.exec(goToPage(1)).pause(670 milliseconds)
.exec(openNewComputerForm).pause(1)
.exec(addNewComputer)
setUp(complexScenario.inject(atOnceUsers(1)).protocols(httpConf))
}
33. Gatling DSL
class ComplexScenarioV3 extends MySimulation {
val search = exec(goToRootPage).pause(7)
.exec(searchFor("macbook")).pause(2)
.exec(positionAt(6)).pause(3)
val addComputer = exec(goToRootPage).pause(2)
.exec(goToPage(1)).pause(670 milliseconds)
.exec(openNewComputerForm).pause(1)
.exec(addNewComputer)
val complexScenario = scenario("Complex demo scenario").exec(search, addComputer)
setUp(complexScenario.inject(atOnceUsers(1)).protocols(httpConf))
}
34. Gatling DSL - checks
scenario("DSL demo")
.exec(http("go to page")
.get("/computers")
.check(status.is(200))
.check(status.in(200 to 210))
35. Gatling DSL - checks
scenario("DSL demo")
.exec(http("go to page")
.get("/computers")
.check(regex("computers")
.find(1)
.exists)
https://www.pinterest.com/pin/491666484294006138/
36. Gatling DSL - checks
scenario("First Scenario Name")
.exec(http("Request name")
.get("/computers")
.check(jsonPath("$..foo.bar[2].baz").ofType[String].notExists)
.check(xpath("//input[@id='text1']/@value").is("test"))
.check(css("...").transform(_.split('|').toSeq).is(Seq("1", "2")))
37. Gatling DSL - checks
scenario("DSL demo")
.exec(http("Authorize")
.get("/auth")
.check(regex("token").find(1).exists
.saveAs("authorizationToken")))
.exec(http("Authorized resource")
.get("/authorized_resource?token=${authorizationToken}"))
38. Gatling DSL - looping
repeat(5, "i") {
exec(goToPage("${i}".toInt))
.pause(1)
}
● repeat
● foreach
● during
● asLongAs
● forever
https://blog.hubspot.com/blog/tabid/6307/bid/32019/Why-Every-Marketer-Needs-Closed-Loop-Reporting.aspx#sm.0005lrqj811waf3ntmn1cul3881gr
39. Gatling DSL - polling
exec(
polling
.every(10 seconds)
.exec(searchFor("thinkpad"))
)
http://www.firmus-solutions.com/terms-conditions.html
40. Gatling DSL - conditions
doIf(session => session("user").as[String].startsWith("admin")) {
exec(goToAdminPage)
}
● doIf
● doIfEquals
● doIfOrElse
● doSwitch
● doSwitchOrElse
● randomSwitch
https://en.wikipedia.org/wiki/Decision_tree
41. Gatling DSL - error management
exec(sendMoney)
.tryMax(10){
exec(checkIfMoneyReceived)
}
● tryMax
● exitBlockOnFail
● exitHereIfFailed
Alice Bob
Kafka
42. Gatling DSL - setup
setUp(myScenario
.inject(
nothingFor(4 seconds),
atOnceUsers(10),
rampUsers(10) over (5 seconds))
.protocols(httpConf))
.maxDuration(10 minutes)
● constantUsersPerSec
● rampUsersPerSec
● splitUsers
● heavisideUsers
43. Gatling DSL - setup
setUp(myScenario
.inject(atOnceUsers(10))
.protocols(httpConf))
.assertions(
global.responseTime.max.lt(50),
global.failedRequests.percent.is(0)
)
http://englishthroughlaxas.blogspot.com/2015/07/531-24-expression-of-assertion-emphasis.html
44. Gatling DSL - setup
setUp(myScenario
.inject(atOnceUsers(10))
.protocols(httpConf))
.throttle(
reachRps(100) in (30 second),
holdFor(1 minute),
jumpToRps(50),
holdFor(2 hours)
)
45. Gatling DSL - feeders
val companies = List("apple", "lenovo", "hp")
val feeder = Iterator.continually(
Map("company" -> companies(Random.nextInt(companies.size))))
val searching = scenario("Searching")
.feed(feeder)
.exec(searchFor("${company}"))
● RecordSeqFeederBuilder
● CSV
● JSON
● JDBC
● Sitemap
● Redis
● … http://favim.com/orig/201104/23/Favim.com-22725.jpg
46. Gatling DSL - resource inferring
val httpConf = http
.baseURL("http://computer-database.gatling.io")
.acceptHeader("...")
.acceptEncodingHeader("...")
.acceptLanguageHeader("...")
.inferHtmlResources()
.userAgentHeader("...")
48. Gatling DSL - other goodies
● Custom validators
● HTTP
○ SSL
○ SSE (Server Sent Event)
○ basic cookies support
● WebSocket
● JMS
● Pluginable architecture:
○ cassandra plugin
○ kafka plugin
○ rabbitMQ
○ AMQP
49. Gatling DSL - logging/debugging
● logging ALL HTTP request and responses
<logger name="io.gatling.http.ahc" level="TRACE" />
<logger name="io.gatling.http.response" level="TRACE" />
● logging ONLY FAILED HTTP request and responses
<logger name="io.gatling.http.ahc" level="DEBUG" />
<logger name="io.gatling.http.response" level="DEBUG" />
50. Gatling DSL - reporting
================================================================================
---- Global Information --------------------------------------------------------
> request count 10 (OK=10 KO=0 )
> min response time 40 (OK=40 KO=- )
> max response time 177 (OK=177 KO=- )
> mean response time 55 (OK=55 KO=- )
> std deviation 41 (OK=41 KO=- )
> response time 50th percentile 42 (OK=42 KO=- )
> response time 75th percentile 43 (OK=43 KO=- )
> response time 95th percentile 117 (OK=117 KO=- )
> response time 99th percentile 165 (OK=165 KO=- )
> mean requests/sec 0.909 (OK=0.909 KO=- )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms 10 (100%)
> 800 ms < t < 1200 ms 0 ( 0%)
> t > 1200 ms 0 ( 0%)
> failed 0 ( 0%)
================================================================================
53. Gatling DSL - distributed
● manually
● Gatling FrontLine
● Flood.io
IT system