O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Stress test your backend with Gatling

189 visualizações

Publicada em

slides from https://www.meetup.com/Scala-by-the-Lagoon/events/242338243/

Publicada em: Software
  • Seja o primeiro a comentar

  • Seja a primeira pessoa a gostar disto

Stress test your backend with Gatling

  1. 1. Performance tests with Gatling Andrzej Ludwikowski
  2. 2. About me ➔aludwikowski.blogspot.com ➔github.com/aludwiko ➔@aludwikowski
  3. 3. Performance tests - why so hard? Simulate production as close as possible: ● hardware ○ CPU, RAM, storage, ... ● software ○ OS, Virtualization, DBs, … ● load ● isolation
  4. 4. Performance tests - why so hard? Necessary to have: ● infrastructure ● monitoring ● logging
  5. 5. Performance tests - why so hard? ● Your performance intuition is wrong! 1. collect the data (monitoring, logging, profiling) 2. find the bottleneck (based on data) 3. fix the bottleneck 4. collect the data and check the assumptions 5. go to 1.
  6. 6. Performance tests - why so hard? Lies, damned lies, and statistics: ● arithmetic mean = 2.9 ● median = 1 ● standard deviation = 6 (only for normal distribution)
  7. 7. Performance tests - why so hard? ● Anscombe's quartet http://bravenewgeek.com/tag/coordinated-omission/ Property Value Mean of x 9 Sample variance of x 11 Mean of y 7.50 Sample variance of y 4.125 Correlation between x and y 0.816 Linear regression line y = 3 + 0.5x Coefficient of determination of the linear regression 0.67
  8. 8. Performance tests - why so hard? Lies, damned lies, and statistics: ● arithmetic mean = 2.9 ● median = 1 ● standard deviation = 6 (only for normal distribution) Use: percentiles 50th = 1 70th = 1 90th = 2.9 95th = 11.45 99th = 18.29 Check percentiles implementation!
  9. 9. Performance tests - why so hard? ● Coordinated omission problem by Gil Tene http://bravenewgeek.com/tag/coordinated-omission/
  10. 10. Coordinated omission problem http://bravenewgeek.com/tag/coordinated-omission/ system.exit(0)
  11. 11. Coordinated omission problem http://bravenewgeek.com/tag/coordinated-omission/ system.exit(0) samples 31 50 th 1 70 th 1 90 th 1 95 th 1 99 th 12.89 99,9 th 28.30 99,99 th 29.82 avg 1.93
  12. 12. Coordinated omission problem http://bravenewgeek.com/tag/coordinated-omission/ system.exit(0) samples 10001 50 th 1 70 th 1 90 th 1 95 th 1 99 th 1 99,9 th 1.15 99,99 th 28.01 avg 1.03
  13. 13. Coordinated omission problem http://bravenewgeek.com/tag/coordinated-omission/ system.exit(0)
  14. 14. Coordinated omission problem http://bravenewgeek.com/tag/coordinated-omission/ system.exit(0) samples 60 50 th 1 70 th 12.6 90 th 24.3 95 th 27.2 99 th 29.45 99,9 th 29.95 99,99 th 29.99 avg 8.25
  15. 15. Coordinated omission problem
  16. 16. Coordinated omission problem total time 60 s max 30 s 99th 1 s
  17. 17. Coordinated omission problem total time 60 s max 30 s 99th 1 s time in % for max 50%
  18. 18. Coordinated omission problem total time 60 s max 30 s 99th 1 s time in % for max 50% expected time in % for 99th 50% - 1% = 49%
  19. 19. Coordinated omission problem total time 60 s max 30 s 99th 1 s time in % for max 50% expected time in % for 99th 50% - 1% = 49% real time for 99th 60 s * 49% = 29.4 s
  20. 20. Coordinated omission problem http://bravenewgeek.com/tag/coordinated-omission/
  21. 21. Performance tests - why so hard? ● Coordinated omission vs Gatling http://bravenewgeek.com/tag/coordinated-omission/
  22. 22. Performance tests - why so hard? ● Tests…
  23. 23. Performance tests - why so hard? ● Tests…
  24. 24. Gatling
  25. 25. Gatling
  26. 26. Why Gatling? ● non-blocking, asynchronous stack (scala, akka, netty) ● scala !!!111oneoneone (maven, sbt support) ● DSL ● recorder ● math is good ● reports ● integration & performance tests
  27. 27. Gatling DSL class BasicSimulation extends Simulation { val httpConf = http .baseURL("http://computer-database.gatling.io") .acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8") .acceptEncodingHeader("gzip, deflate") .acceptLanguageHeader("en-US,en;q=0.5") .userAgentHeader("Mozilla/5.0 (Macintosh; X 10.8; rv:16.0) Gecko/20100101 Firefox/16.0") val firstScenario = scenario("First Scenario Name") .exec(http("Request name").get("/")) .pause(7) setUp(firstScenario.inject(atOnceUsers(1)).protocols(httpConf)) }
  28. 28. Gatling DSL class FirstScenarioV2 extends MySimulation { val firstScenario = scenario("First Scenario Name") .exec(http("Go to root page").get("/")) .pause(7 seconds) setUp(firstScenario.inject(atOnceUsers(1)).protocols(httpConf)) }
  29. 29. Gatling DSL import io.gatling.core.Predef._ import io.gatling.http.Predef._ class FirstScenarioV2 extends MySimulation { val firstScenario = scenario("First Scenario Name") .exec(http("Go to root page").get("/")) .pause(7 seconds) setUp(firstScenario.inject(atOnceUsers(1)).protocols(httpConf)) }
  30. 30. Gatling DSL class ComplexScenario extends MySimulation { val complexScenario = scenario("Complex demo scenario") .exec(http("request_1").get("/")).pause(7) .exec(http("request_2").get("/computers?f=macbook")).pause(2) .exec(http("request_3").get("/computers/6")).pause(3) .exec(http("request_4").get("/")).pause(2) .exec(http("request_5").get("/computers?p=1")).pause(670 milliseconds) .exec(http("request_6").get("/computers/new")).pause(1) .exec(http("request_7") .post("/computers") .formParam("name", "MyComputer").formParam("introduced", "2012-05-30").formParam("company", "37")) setUp(complexScenario.inject(atOnceUsers(1)).protocols(httpConf)) }
  31. 31. Gatling DSL class ComplexScenarioV2 extends MySimulation { val complexScenario = scenario("Complex demo scenario") .exec(goToRootPage).pause(7) .exec(searchFor("macbook")).pause(2) .exec(positionAt(6)).pause(3) .exec(goToRootPage).pause(2) .exec(goToPage(1)).pause(670 milliseconds) .exec(openNewComputerForm).pause(1) .exec(addNewComputer) setUp(complexScenario.inject(atOnceUsers(1)).protocols(httpConf)) }
  32. 32. Gatling DSL class ComplexScenarioV3 extends MySimulation { val search = exec(goToRootPage).pause(7) .exec(searchFor("macbook")).pause(2) .exec(positionAt(6)).pause(3) val addComputer = exec(goToRootPage).pause(2) .exec(goToPage(1)).pause(670 milliseconds) .exec(openNewComputerForm).pause(1) .exec(addNewComputer) val complexScenario = scenario("Complex demo scenario").exec(search, addComputer) setUp(complexScenario.inject(atOnceUsers(1)).protocols(httpConf)) }
  33. 33. Gatling DSL - checks scenario("DSL demo") .exec(http("go to page") .get("/computers") .check(status.is(200)) .check(status.in(200 to 210))
  34. 34. Gatling DSL - checks scenario("DSL demo") .exec(http("go to page") .get("/computers") .check(regex("computers") .find(1) .exists) https://www.pinterest.com/pin/491666484294006138/
  35. 35. Gatling DSL - checks scenario("First Scenario Name") .exec(http("Request name") .get("/computers") .check(jsonPath("$..foo.bar[2].baz").ofType[String].notExists) .check(xpath("//input[@id='text1']/@value").is("test")) .check(css("...").transform(_.split('|').toSeq).is(Seq("1", "2")))
  36. 36. Gatling DSL - checks scenario("DSL demo") .exec(http("Authorize") .get("/auth") .check(regex("token").find(1).exists .saveAs("authorizationToken"))) .exec(http("Authorized resource") .get("/authorized_resource?token=${authorizationToken}"))
  37. 37. Gatling DSL - looping repeat(5, "i") { exec(goToPage("${i}".toInt)) .pause(1) } ● repeat ● foreach ● during ● asLongAs ● forever https://blog.hubspot.com/blog/tabid/6307/bid/32019/Why-Every-Marketer-Needs-Closed-Loop-Reporting.aspx#sm.0005lrqj811waf3ntmn1cul3881gr
  38. 38. Gatling DSL - polling exec( polling .every(10 seconds) .exec(searchFor("thinkpad")) ) http://www.firmus-solutions.com/terms-conditions.html
  39. 39. Gatling DSL - conditions doIf(session => session("user").as[String].startsWith("admin")) { exec(goToAdminPage) } ● doIf ● doIfEquals ● doIfOrElse ● doSwitch ● doSwitchOrElse ● randomSwitch https://en.wikipedia.org/wiki/Decision_tree
  40. 40. Gatling DSL - error management exec(sendMoney) .tryMax(10){ exec(checkIfMoneyReceived) } ● tryMax ● exitBlockOnFail ● exitHereIfFailed Alice Bob Kafka
  41. 41. Gatling DSL - setup setUp(myScenario .inject( nothingFor(4 seconds), atOnceUsers(10), rampUsers(10) over (5 seconds)) .protocols(httpConf)) .maxDuration(10 minutes) ● constantUsersPerSec ● rampUsersPerSec ● splitUsers ● heavisideUsers
  42. 42. Gatling DSL - setup setUp(myScenario .inject(atOnceUsers(10)) .protocols(httpConf)) .assertions( global.responseTime.max.lt(50), global.failedRequests.percent.is(0) ) http://englishthroughlaxas.blogspot.com/2015/07/531-24-expression-of-assertion-emphasis.html
  43. 43. Gatling DSL - setup setUp(myScenario .inject(atOnceUsers(10)) .protocols(httpConf)) .throttle( reachRps(100) in (30 second), holdFor(1 minute), jumpToRps(50), holdFor(2 hours) )
  44. 44. Gatling DSL - feeders val companies = List("apple", "lenovo", "hp") val feeder = Iterator.continually( Map("company" -> companies(Random.nextInt(companies.size)))) val searching = scenario("Searching") .feed(feeder) .exec(searchFor("${company}")) ● RecordSeqFeederBuilder ● CSV ● JSON ● JDBC ● Sitemap ● Redis ● … http://favim.com/orig/201104/23/Favim.com-22725.jpg
  45. 45. Gatling DSL - resource inferring val httpConf = http .baseURL("http://computer-database.gatling.io") .acceptHeader("...") .acceptEncodingHeader("...") .acceptLanguageHeader("...") .inferHtmlResources() .userAgentHeader("...")
  46. 46. Gatling - launching ● gatling:test ● gatling:testOnly x.y.z.GetUserSimulation
  47. 47. Gatling DSL - other goodies ● Custom validators ● HTTP ○ SSL ○ SSE (Server Sent Event) ○ basic cookies support ● WebSocket ● JMS ● Pluginable architecture: ○ cassandra plugin ○ kafka plugin ○ rabbitMQ ○ AMQP
  48. 48. Gatling DSL - logging/debugging ● logging ALL HTTP request and responses <logger name="io.gatling.http.ahc" level="TRACE" /> <logger name="io.gatling.http.response" level="TRACE" /> ● logging ONLY FAILED HTTP request and responses <logger name="io.gatling.http.ahc" level="DEBUG" /> <logger name="io.gatling.http.response" level="DEBUG" />
  49. 49. Gatling DSL - reporting ================================================================================ ---- Global Information -------------------------------------------------------- > request count 10 (OK=10 KO=0 ) > min response time 40 (OK=40 KO=- ) > max response time 177 (OK=177 KO=- ) > mean response time 55 (OK=55 KO=- ) > std deviation 41 (OK=41 KO=- ) > response time 50th percentile 42 (OK=42 KO=- ) > response time 75th percentile 43 (OK=43 KO=- ) > response time 95th percentile 117 (OK=117 KO=- ) > response time 99th percentile 165 (OK=165 KO=- ) > mean requests/sec 0.909 (OK=0.909 KO=- ) ---- Response Time Distribution ------------------------------------------------ > t < 800 ms 10 (100%) > 800 ms < t < 1200 ms 0 ( 0%) > t > 1200 ms 0 ( 0%) > failed 0 ( 0%) ================================================================================
  50. 50. Gatling DSL - distributed
  51. 51. Gatling DSL - distributed IT system
  52. 52. Gatling DSL - distributed ● manually ● Gatling FrontLine ● Flood.io IT system

×