QA challenge accepted edition

1. Viktor Slavchev &
The Testing troll
Siteground
The “worst” practices of software testing

2. About me
• Senior QA @ Siteground Hosting
• Wooden philosopher at Pragmatic
(part time)
• Blogger @ mrslavchev.com(long
time ago in a galaxy far far away)
• True believer that testing is a
human centric activity

3. • A mythical testing creature
• Graduated the technical
university of Trolandia
• Certified as SHGMCNGTT -
Super Hyper Giga Mega
Certified Ninja Guru Testing
troll
• Book author
About The testing troll

4. And the book:
• Absolute bestseller
• Sold more copies than “50
shades of gray” in Trolandia
• “Absolutely mind-blowing”
New York Times
• “The troll is strong with this
one”
Darth Vader

5. Best practice#1 - Rely on requirements
and documentation

6. Worst practice #1:
Learn about the real testing oracles
instead
Testing oracles can benefit from:

7. Feelings and mental models
• It starts as a feeling
• Not sufficient for a
bug report

8. Artifacts: documents, designs,
requirements, etc.

9. Experience in:
• The domain
• General testing
• Software industry

10. Conference
• Developers
• Other testers
• Clients
• Anyone who
matters

11. Example 1: Reference oracle
• External application
• Other platform
• Correctness presumed

12. Example 2: Self-verifying data as an
oracle
Desired response: Credit card #:
000: Approved 4470330769941000
010: Partially Approved 4658512425423010
110: Insufficient Funds 4488282659650110

13. Example 3: Constraints oracle
Example: VAT number
BG123456789
• Length
• Country index
• Spaces
Source:The Oracle Problem and the Teaching of Software
Testing

14. Best practice#2 - Do regression
testing by executing your old tests
Regression testing != repetitive testing

15. Repetitive testing is the reason why
“others” see testing like this...

16. Worst practice #2:
Regression checks and focus on tests
that reveal new information
Regress - verb
• to move backward; go back.
• to revert to an earlier or less advanced
state or form

17. Regression bug:
• Bug causing regress in
quality
Regression testing:
• Testing to expose
regression bugs

18. Regression check:
• Building part of a test
• Definite answer
• Small and easy to
manage
Focus on information

19. Best practice#3 - Automationion testing is
the best, “manual” testing sucks

20. Quote from J. Kohl - Tap into mobile
application testing
Ideas that will produce
bug-free software:
• C++ STL
• OOP
• Agile
• TDD
• Test automation

21. Quote from J. Kohl - Tap into mobile
application testing
“Each of these were also used to tell me
that testers would no longer be
needed on software projects. So far, I
am fifteen years in and I haven’t seen
anything bug free yet.”

22. Can a testing tool produce better
testers?
Answer:

23. How to make efficient automation that
provides value to testing?
The answer is:

24. Worst practice #3:
Use automation as a tool
• To extend your abilities
• Not to replace you
completely
• To ask the right
questions

25. Automation is great for:
• Confirmation /
Verification
• Low level checking
• Precision
• Functional correctness
• Solving machine
problems

26. But automation sucks when dealing
with:
• Exploration
• Unanticipated risks
• Qualitative evaluation
• Open ended questions
• Solving human
problems
Source:Things Could Get Worse-
Michael Bolton

27. Source: The non-manual, unautomated tester

28. Best practice#4 - Assure quality
Quality is not a resource
Quality != testing

29. Worst practice #4:
Provide information about risks
• Risk based testing -
focus testing to explore
potential risks

30. “Nightmare newspaper headings”
• You missed a critical
bug in production
• It caused your
company a huge
financial loss
• What would it be?

31. • Pre-mortem - imagining a failure and
working backwards in order to
determine the possible risks.
Benefits:
• Evaluation of risks
• Business perspective
• Prioritization

32. Best practice#5 - Follow best practices

33. Worst practice #5:
Be always alert and aware of your
context
• What would be your
approach in 3 different
contexts?

34. Context #1: Weather app
• You’re an outsourced
testing expert
• Working for a start-up
• Low on budget
• Behind schedule

35. Context #2: Medical software
• High precision is a must
• Human lives might
depend on it
• Regulatory compliance
of testing

36. Context #3: Court case
• Bug caused an injury
• You are independant
expert
• On a case for negligence
in testing

37. Questions:
• How exhaustive will be
your testing in each
context?
• Will bug-hunting be your
highest priority?
• Will you have the same
strategy of testing?

38. Conclusion
• No best practices!
• “We are not into the
belief business” J.Bach
• In order to be a good
tester, you have to be a
professional sceptic

39. Thank you very much!
… And don’t forget to Rock ‘n Troll

40. Image credits:
Too lazy to post them all.
Image credits go to the almighty internet, I don’t own
any of the images included in this presentation, except
the ones created by me, linked to my blog.

41. STIONS?
ANY