Презентация доклада Knud Hangaard на конференции SQADays-14 English Day, Львов 7 ноября 2013

1. AGILE DEVELOPMENT IN SAXO BANK
HOW TO MANEUVER AS TESTER
TO GET THE RIGHT QUALITY
BY KNUD HANGAARD

2. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
FACTS ABOUT ME
Master of Science, Electronics and
Computer Engineering
Certifications:
ISTQB, PRINCE2, ACI Dealing/Trading
My working sectors:
• HW/SW production industry
• Banking industry
2
My current role:
Front Office test lead:
• Release test responsible
• Test environments
• Development and test processes
Contact:
knud@hangaard.dk
LinkedIn:
www.linkedin.com/in/khangaard
SKYPE: khangaard

3. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
PRESENTATION FOCUS AREAS
Introduction to Saxo bank and the software
Agility challenges and finding the “Right” quality
What we have done to become more agile: process, tools, people
What to bring home:
o Key focus areas in the test process for achieving right level of quality
o Key skills that has been necessary to manage testing in a hyper agile organisation
3

4. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
INTRODUCTION TO SAXO BANK
Established 1992
Employees >1300 (700 in IT)
Offices 25 countries
FX trades daily 170,000
Daily av. turnover 20 billion USD
Countries with retail clients 190
Financial instruments > 25,000
Received prices daily 5-6 billion
Development in Denmark and India
The “Front Office” in Saxo Bank headquarter.
4

5. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
HUGE SOFTWARE PLATFORM
o Software components: 350+
o 11 million lines of code

6. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
THE DEVELOPMENT CONDITIONS
o FOCUS ON LEADING ROLE, DELIVERY AND COST:
o BE PRODUCTIVE
• Ongoing business development, large
project portfolio - demand for new
functionality (DEVELOPING THE BANK)
o BE GOOD
• Ongoing technical and performance
improvements – minimum number of
bugs in Live. (RUNNING THE BANK)
o BE ECONOMICAL
• Cost awareness: (off shore
development and test)
6

7. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
SO WE WANT TO BE AGILE…
7

8. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
SAXO BANK DEVELOPMENT MODEL
(Prince2)/Scrum/Kanban
Project
testers
Scrum/Kanban
Release
testers
Project 1
Projects
Project 2
Quality gate
Merge to main
Project N
Platform
Quality gate
Production
Defects
Incidents from Production
Project Test
Environments
8
Release Test
Environment
Simulation
Production

9. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
DEVELOPMENT TRENDS IN SAXO BANK,
BECOMING MORE AGILE
Delivery cycle has been reduced over time
o
Idea is less changes per cycle
o
No forcing of last minute changes (follow the process!)
Development has been divided into sectors, aligned with business areas
o
More focus on “the customer”
9

10. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
CHALLENGES FOR QUALITY ASSURANCE AND TEST
o
o
o
o
o
o
o
10
Huge code base
Huge amount of components
Test basis (documentation) is limited
Limited time for writing test cases
Limited time for testing
Limited time for writing test reports.
What would we really like to test - test coverage?

11. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
SO WHAT IS THE RIGHT QUALITY?
Live
Incidents
and roll
backs (y)
y=v*x where v: development velocity (throughput)
Low quality (many new
changes, huge amount of Live
incidents)
“Right” quality (enough number of new
changes, reasonable number of
incidents in Live)
High quality (less new
changes, only few Live
incidents)
Deliveries/changes to Business (x)
Right Quality:
o Happy Business (more competitive products, earnings – less problems in
Live, losses)
o Happy Customers (more products, opportunities – less bugs found)

12. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
BECOMING MORE AGILE
KEY POINTS COVERED:
o The Process
o The Tools
o The People
12

13. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
PROCESS
DEVELOPMENT AND TEST PROCESS – KEY POINTS
Time boxed development and time boxed testing (backlogs for each area)
Incremental approach:
o release in small steps - backward compatible - possible to roll back
Risk based testing:
o Spend more time on high risk changes (both functional and regression)
Flexible development and test process:
o Quality gates/ verification points can be decided on low level
Cross functional teams:
o Shared test responsibility between developers and testers
13

14. PROCESS
AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
RELEASE TEST CYCLE (TIME BOXED)
o
o
o
o
o
o
Monday: Baseline
Tuesday: Deploy DB changes and verify
Wednesday: Deploy new services and verify
6 days for test and reporting
Next Thursday: Go/No meeting
Friday: Follow-up and planning next release
Monday
Tuesday
Wednesday
•Baseline of
test
environment
•DB release
candidates
•Apps og
Services rel.
candidates
Monday
•Release
Test
Tuesday
•Release
Test
Wednesday
•Release
Test
•Reporting
Thursday
Friday
•Release
Test
•Release
Test
Thursday
•Go/No Go
meeting
•Release to
Simulation
Friday
•Planning
next
release
test
Saturday
Sunday
•DB
Releases
to Live
Saturday
•Release
to Live
Sunday
Backup of
Live

15. PROCESS
AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
RELEASE DOCUMENTATION BUILT-UP
A tree structure of work items:
o Release item: Parent for the full release
Release, week x
Change (Component 1)
Delivery (Project A)
User Story a
o Change items: One for each Component –
release request for production
o Delivery items: One from each team or project.
User Story b
Bug Fix y
Delivery (Project A)
User Story c
o User Story items: The functional changes
Change (Component 2)
o Bug items: The bug fixes.
Delivery (Project C)
…
15

16. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
STATE TRANSITIONS AND TEST FLOWS
Development User Story states (Kanban):
• Not Done, Active, Resolved and Closed
Independent Test stages (Test Flows):
• NA, Not Done, Done
Test flows at 3 stages:
• development phase,
• integration phase,
• system integration phase (release test)
Auto generated test report
16
PROCESS

17. TOOLS
AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
THE TOOLS – ALL IN ONE (Team Foundation Server along
with Microsoft Test Manager)
Bugs
Test
Execution
Plan
Test
Results
17
Test Cases/
Test
Scripts
Test
Reports
Code
User
Stories/
Features
Tasks

18. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
TOOLS
THE TEST ENVIRONMENTS
o Test environments that simulates Live (Production) environment
• Release and project test environments
o Baseline of test environments with Live data every two weeks
o Intensive scripting for adapting Live data to test environment (object)
o One-click deployment of new software
18

19. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
PEOPLE
AGILITY - IMPACTS ON QUALITY ASSURANCE THINKING
o Adapt to the situation and plan very much based on RISK
o Find the “right” test depth (effort) – use your time optimal
o Share results early and adjust when you know more
• early feedback both ways is important
o Find the right level of test documenting and reporting
• so you get the message out but do not overdo
19

20. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
PEOPLE
A NEW TESTER ROLE
o Test certifications help as a basis - but is not sufficient in “the field”
o Be able to handle many different stakeholders (developers, project
managers, delivery managers, business owners, business support)
o Be technical well founded on multiple areas (less time for study, understand
fast, react fast)
o Be RISK focused (find the serious bugs in the beginning)
o Be Proactive - ask questions!
o Make plans but expect them to change and look for the new test opportunities
o Spend more time on exploratory testing instead of just running the same test
cases
o Do early and precise test reporting (only few lines to explain the problem)
20

21. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
EFFECTS OF THE APPROACH
21

22. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
EFFECTS
Business Effects:
• More releases going to Live
• Less high severity bugs found in Live (roll back)
Process Effects:
• Better transparency and traceability
• Effective documenting and reporting
• The flexible dev. model makes it easy to adapt to
new situations
• Benefit from time difference: Build late in
Denmark - test early in India
Test Effects:
• ”Exploratory testing” generates more bugs than
executing test cases prepared in advance
22

23. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
LESSONS LEARNED
23

24. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
KEY FOCUS AREAS IN THE TEST PROCESS FOR ACHIEVING
RIGHT LEVEL OF QUALITY
Simple, clear and flexible process
o Clear responsibilities
o Clear expectations
Teamwork
o Daily scrums with testers and
developers
o Avoid being dependent on other
team’s priorities.
Effective and clear communication
o To avoid time wasted
o To avoid bugs being overseen
24
Effective documentation
o Find the ”right” level of details
o Be focused on key points – make it
easy for the reader
Effective tooling
o Easy planning, test execution and
reporting
o Easy setup of test environments

25. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
KEY SKILLS NECESSARY TO MANAGE TESTING
IN A HYPER AGILE ORG.
o Great communication skills - be able to communicate with many different
stakeholders
o Knowledgeable
o Need to have both good business and technical/development knowledge
o Need to know the ”Go to Guys” to get more knowledge quickly
o Great team player - share knowledge – get knowledge
o Open mindset - be ready to change plans quickly
o Tester’s mindset – explore, ask questions
o Responsible mindset – Be the one that follows-up
25

26. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
CULTURAL DIFFERENCES BETWEEN OFF-SHORE TEAMS
Different thinking:
Communication path - hierarchical vs. lateral
Control - centralized vs. decentralized
Planning - detailed vs. less detailed
Different communication form:
Direct vs. indirect (verbally)
Direct vs. indirect (in writing – sugar coating)
language barriers
Different job goals:
Grow hierarchical vs. grow expert wise
26

27. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
CHALLENGES FOUND AND HOW TO OVERCOME
o Culture differences
o Discuss the cultural differences for better understanding and learning
o Engage with team members (non-work related)
o Travel often both ways
o Unclear goals/expectations
o Ensure clear communication (combine verbal and writing)
o Team SPOCs both places for ”translating” the messages
o Be very concrete on goals and expectations
o Regularly status meetings retrospectives and feedback
o Having the feeling of being far away
o Electronic boards for status sharing
o Video conferencing a must
27

28. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
NEXT PLANS FOR SAXO BANK
o Continuous delivery
o Even more effective processes
o Even more automation
o Even more auto generated reports
28

29. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
CONCLUSION – IMPORTANT FOCUS AREAS IN AGILE TEAMS
Clear Communication
Effective Processes
Effective Tools
Team Player Mindset
Risk Mindset

30. AGILE DEVELOPMENT IN SAXO BANK. HOW TO MANUVER AS TESTER TO GET THE RIGTH QUALITY
THANK YOU FOR LISTENING!
Feel free to contact me:
LINKEDIN: WWW.LINKEDIN.COM/IN/KHANGAARD
30