Things always go wrong in production, but this doesn’t have to be a bad thing. It’s an opportunity to learn about your system and the real world with which it interacts. Rouan will show you how to build a set of feedback mechanisms that help you find out about issues as they happen and ship fixes quickly. Using real examples, he’ll walk you through how they use metrics, logging, auditing and alerting to support tes.com and move fast. He’ll show you which tests to throw away and what to put in their place. He’ll guide you to finding the right balance of pre-production and production quality practices, so that you can gain a more realistic and holistic understanding of the quality of your system.

1. QA in Produc,on.
The tests we never wrote and
the produc1on monitoring we
use instead.
Rouan Wilsenach
@rouanw
rouanw.github.io
. . .. . . . .

2. Zero produc,on
defects.

3. Most produc,on
defects are surprises.

4. What you thought you
knew was wrong.

5. Try again. Fail again. Fail
be@er.
Samuel Becke>, 1983

6. h>ps://mar1nfowler.com/ar1cles/qa-in-produc1on.html

7. What should we
monitor?
$$$

8. Talk to your humans.

9. Track metrics.
. . .. . . . .

10. Dashboards.

11. Alerts.

13. If you liked it, then you
should have put a
metric on it.
#beyonceMonitoring
Dan Abel, 2016
@twicezer0

14. Simple start.
4s 500 0

15. Something a li@le
unexpected.

18. Playing detec,ve.

19. Logs.

20. Empty logs.
.

21. Audi,ng.
——————————>

23. Are your tests slowing
you down?

24. User interface tests.
....................................

25. Performance tests.

26. Are you ready for QA in
Produc,on?

27. Yes, but…

28. How fast can you ship a
fix?
....................................

29. How fast do you need
to ship a fix?
.$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$.....

30. Finding the right
balance.
....................................

31. Rouan Wilsenach
@rouanw
rouanw.github.io
QA in Produc,on.
....................................