Sharing on Software Testing after attending the ISTQB Software Tester Foundation course

1. Software Testing
(Part I)
A Brief Presentation By
Disney E. Lai

2. Outline
● Fundamentals of Testing
● Testing in the Software Life Cycle
● Discussion

3. Fundamentals of Testing
WHY IS TESTING NECESSARY?

4. Fundamentals of Testing
The Threats of Bugs
● Damaged Reputation ● Pollution/Waste
● High/Unpredictable ● Lost jobs
maintenance cost ● Lost lives
● Unexpected delays ● Lost rights
● Lack of confidence ● Lost missions
● Lawsuits ● Lost Wars
and many more ....

5. Fundamentals of Testing
ARIANE 5 Flight 501 (1996)
● Pride of the European Space Agency
● Exploded 37 seconds after launched
● Cause of failure – Inertial Reference
System software failed to compute
64-bit float number. Software reused
from Ariane 4 only uses 16-bit.
● Cost of Failure - $8 billion

6. Fundamentals of Testing
Patriot Missile Bug (1991)
● Patriot Missile System
● Designed to intercept and destroy
incoming missile
● Early version failed to respond to
incoming SCUD missiles
● Cause of failure – software rounding
error incorrectly calculate time
● Cost of Failure – 28 lives lost and
many injured.

7. Fundamentals of Testing
Mars Climate Orbiter (1998)
● Designed to stay on Mars' low orbit
to study Mars' climate.
● Failed to stabilize in orbit. Destroyed
by Mars atmospheric pressure
● Cause of failure – software system
use imperial units rather than metric
● Cost of Failure – ~$330 million,
lost mission

8. Fundamentals of Testing
Soyuz TMA-10 (2007)
● Return module carrying 3 person
including Malaysian Angkasawan
● Computer 'glitch' caused a steeper
ballistic descent
● Extreme G-force
● Cost of Failure – ~ 200KM off target,
'fried Angkasawan'
(almost)

9. Fundamentals of Testing
Universiti Sains Malaysia (2009)
● Online Application System
● Misinformed all 8,173 prequalified
applicants of successfull entry rather
than only 3,599 applicants.
● Caused frustration and anger.
● Cost of Failure – ~ lost of reputation?

10. Fundamentals of Testing
City Bus Terminal First Day (2006)
● RM4.5 Million City Bus Terminal
Inanam
● Computerised ticketing system failed to
function due to computer glitch – Manua
alternative
● Caused frustration and anger.
● Cost of Failure – ~ lost of reputation?
Lost service

11. Fundamentals of Testing
KD Tunku Abdul Rahman (2010)

12. Fundamentals of Testing
BUG? WHAT BUG?

13. Fundamentals of Testing
Where Do Bugs Come From?
●
People  Errors  Bugs (defects)  Failures
● Requirements and design specifications
● Code (Business Logic and UI)
● Documentation (electronic and hard copy)
●
Failures  People  Dissatisfaction with quality

14. Fundamentals of Testing
Where Do Bugs Come From?
● Bugs Occur Due to....
● Programmer, Analyst, and other individuals contributing
(including tester!)
● Complexity of code, infrastructure or problem
● Changing and Meshing of Technologies
● Time Pressure
● Many system interactions

15. Fundamentals of Testing
Where Do Bugs Come From?
● Failures Occur Due to bugs and....
● Environmental conditions – dust, humidity...
● Misuse (deliberate and accidental)

16. Fundamentals of Testing
What does 'QUALITY' means to you?
● Fitness for use vs conformance to requirements.
● TESTING AND QUALITY
● Important Quality characteristics? OR
● Testing, Quality Assurance and Quality Improvement

17. Fundamentals of Testing
WHAT IS TESTING?

18. Fundamentals of Testing
Testing Objectives
● Find bugs – information programmers need to fix important
bugs
● Increase Confidence Level – Quality
● Prevent defects – early detection
● Help management understand system quality
● And so on....

19. Fundamentals of Testing
TESTING
OBJECTIVES
PLANS
? ACTIONS

20. Fundamentals of Testing
Test Phases
● Unit/Component Test
● Integration Test
● System Test
● Acceptance Test/Pilot Test
● Maintenance Test
● Operational Test

21. Fundamentals of Testing
Effective Efficient

22. Fundamentals of Testing
Testing Vs. Debugging
● Testing - Find failures that caused by bugs
● Debugging – Fix defect
● Identifies root cause of a bug
● Repairs the code
● Checks that defects is fixed correctly
● Different responsibilities:
● Testers Test
● Programmers Debug

23. Fundamentals of Testing
TESTING PRINCIPLES

24. Fundamentals of Testing
Testing Reveals Presence of Bugs
Testing can reveal the presence of
bugs, but cannot prove their absence

25. Fundamentals of Testing
Mission Impossible: Exhaustive Testing
Exhaustive testing (all combinations of inputs and
preconditions) is NOT possible
Exhaustive testing as a way to prove the software works is a
misexpectation
Bad Expectation create problem for test pro and teams
● Unachievable high demands on test group
● Perception of incompetence

26. Fundamentals of Testing
Benefit of Early QA and Testing
● Cost of bug tends to increase as the project progresses....
Defect Clustering
● Bugs tend to appear in clusters
Pesticide Paradox
● One technique alone is not effective to find defects
● Use new and different techniques to find defect

27. Fundamentals of Testing
Absence-of-Errors Fallacy
● Finding and fixing many bug does not guarantee satisfaction
● Successful projects balance – features, schedule, budget
and quality

28. Fundamentals of Testing
FUNDAMENTAL TEST PROCESS

29. Fundamentals of Testing
Testing Processes
I ON
EXECUTION
NTAT
G
S
CLOSURE
NNIN
LYSI
IGN
EME
DES
ANA
PLA
EVALUATING
EXIT CRITERIA
IMPL
REPORTING
TEST RESULT
CONTROL

30. Fundamentals of Testing
Planning and Control
● Determine Test Scope, Risks, Objectives, Strategies,
Resource
● Scheduling
● Implement test strategies
● Determine test exit criteria
● Measure and analyze results
ING
● Monitor and document
N
● Initiative corrective actions
PLAN
● Make decisions
CONTROL

31. Fundamentals of Testing
Analysis and Design
● Review the test basis
● Identify and prioritize
● Evaluate testability
● Design and prioritize combinations of test data
● Identify test data needed for test condition
● Design test environment
● Identify infrastructure tools
EXECUTION
ION
G
S
NTAT
CLOSURE
NNIN
LYSI
N IG
EM E
DES
EVALUATING
ANA
PLA
EXIT CRITERIA
IMPL
REPORTING
TEST RESULT
CONTROL

32. Fundamentals of Testing
Implementation and Execution
● Develop, implement and prioritize test cases
● Create test harnesses, scripts
● Organize test suites, and sequences of test procedures
● Verify the test environment
● Execute test cases (manual or automated)
● Log test results
● Compare actual and expected
EXECUTION
● Report and analyze incidents
ION
G
● Repeat corrected and/or updated
S
NTAT
CLOSURE
NNIN
LYSI
N IG
EM E
DES
EVALUATING
ANA
PLA
test EXIT CRITERIA
IMPL
REPORTING
● Run confirmation and/or
TEST RESULT
regression test CONTROL

33. Fundamentals of Testing
Exit Criteria, Reporting, and Closure
● Check test logs against exit criteria
● Assess if more tests are required
● Write test summarry report
● Confirm test deliverables
● Finalize and archive testware, test environment
● Deliver testware to maintenance org.
● Perform restrospective to capture
EXECUTION
improvements for future releases
ION
G
S
NTAT
CLOSURE
NNIN
LYSI
N IG
EM E
DES
EVALUATING
ANA
PLA
EXIT CRITERIA
IMPL
REPORTING
TEST RESULT
CONTROL

34. Fundamentals of Testing
PSYCHOLOGY OF TESTING

35. Fundamentals of Testing
Good Tester Attributes
● Curiosity
● Professional Pessimism
● Critical Eye
● Attention to detail
● Good communication skills

36. Fundamentals of Testing
Defining Tester Skills
● Reading
● Writing
● Technology
● Project
● Testing skills

37. Fundamentals of Testing
Differing Mindsets
● Testing and Reviewing vs Development
● Developers – author bias

38. Fundamentals of Testing
Degrees of Independence
● Independent tester – effective at finding
failures
● Levels of test independence

39. Fundamentals of Testing
● Clear Objectives
● Focus
● Constructor or Destructor?
● Bad News or Bad Guy?

40. Testing In The SW Life Cycle
''V'' MODEL

41. Testing In The SW Life Cycle

42. Testing In The SW Life Cycle
TEST TYPES

43. Testing In The SW Life Cycle
Testing
Static Dynamic
Static
Review White box Black box
Analysis
Non
Functional
Functional

44. Testing In The SW Life Cycle
Other Test scopes
● Functionality
● Security
● Interoperability
● Performance and Reliability
● Stress, Capacity and Volume
● Maintenance and Maintainability
● Usability and UI
● Configuration and Portability
● ...

45. Testing In The SW Life Cycle
ISO 9126 Quality Standard for Software

46. ISTQB Certification RoadMap

47. Discussion
Q&A
Software Testing in JPKN