The document provides an overview of a presentation on retrospectives. It defines retrospectives as rituals held at the end of projects to learn lessons and plan for future improvements. The presentation covers planning retrospectives, conducting them, and following up after to ensure identified issues are addressed. It provides tips on various stages, such as inviting the right stakeholders, using facilitators, and establishing ground rules. Examples of lessons learned include the need for action plans and management support to implement identified changes.

1. Presented by:
Betty Schaar, CSQA, CSM
BenchmarkQA
Date:
March 03, 2011

2. BenchmarkQA helps project teams deliver higher
quality software through:
Quality Assurance Consulting
Contract & Permanent Staffing
QA/Test Training & Seminars
Local Outsourcing Test Lab
QA IS ALL WE DO!
© 2011 BenchmarkQA

3. Agenda
Introductions
Retrospective Defined
Planning for a Retrospective
At the Retrospective
After the Retrospective
Lessons Learned
Let s Try One!
Q&A
Closing Remarks
3/3/2011 © 2011 BenchmarkQA Slide 3

4. Top 10 Fashion Trends We Wish Had a Retrospective
(So They Won t Be Repeated)
3/3/2011 © 2011 BenchmarkQA Slide 4

5. Retrospective Defined
3/3/2011 © 2011 BenchmarkQA Slide 5

6. Retrospective Defined
retrospective (rèt re-spèk-tîv)
a ritual held at the end of a project to learn from the
experience and to plan changes for the next effort.
Inspect and Adapt opportunity for process and team
What went well?
What could be better?
What do we want to change?
3/3/2011 © 2011 BenchmarkQA Slide 6

7. Retrospective Defined cont d
Attendees and Roles
No one knows the whole story of the project. Each person has a piece of
the story. The retrospective ritual is the collective telling of the story and
mining the experience for wisdom.
- -Norman Kerth
Include everyone who has a piece of the story to tell; all disciplines on
the project should be represented
Make sure there is a facilitator, preferably someone who was not
involved with the project, to lead the retrospective
A designated scribe is helpful
3/3/2011 © 2011 BenchmarkQA Slide 7

8. Retrospective Defined cont d
When to hold a retrospective?
At periodic intervals such as at the end of every delivery cycle
When a significant change or disruption to the project occurs
At the conclusion of the project
3/3/2011 © 2011 BenchmarkQA Slide 8

9. Retrospective Defined cont d
Why have a retrospective?
It s something that every true learning organization has as part of its
culture, and it s one of the best ways to grow project by
project into a smarter and increasingly successful organization.
- -Norman Kerth
To consider ways to improve overall performance
To address ongoing obstacles to productivity
For continuous improvement
Also
To acknowledge and CELEBRATE successes
3/3/2011 © 2011 BenchmarkQA Slide 9

10. Planning For A Retrospective
3/3/2011 © 2011 BenchmarkQA, Inc. Slide 10

11. Planning For A Retrospective
Invite right players; make sure adequate representation
Decide who will facilitate
Best if someone outside the team so entire team can participate
If from within the team, rotate so not always the same person
Want someone with good facilitation skills
Facilitator determines questions to be asked and if will solicit
input/feedback at the session or ahead of time
If ahead of time, decide input mechanism electronic or hard
copy
3/3/2011 © 2011 BenchmarkQA Slide 11

12. Planning For A Retrospective cont d
Determine how much time for the session
Collect defect info for top 1 or 2 classes of defects
For review and correction of assigned root cause
Team to consider if root cause assignments indicate trends to
address
Organize supplies needed white board, large paper, sticky
notes, markers, tape, treats!
3/3/2011 © 2011 BenchmarkQA Slide 12

13. At The Retrospective
3/3/2011 © 2011 BenchmarkQA, Inc. Slide 13

14. At The Retrospective
Review the objective; suggest:
Reflect on last delivery cycle to identify what went well, what didn t
go so well in order to determine what can be done in future delivery
cycles to improve on project execution and teamwork.
Set the stage
Regardless of what we discover, we understand and truly believe that
everyone did the best job they could, given what they knew at the
time, their skills and abilities, the resources available, and the
situation at hand.
- - The Prime Directive (Norman Kerth)
3/3/2011 © 2011 BenchmarkQA Slide 14

15. At The Retrospective cont d
We know more at the end than when we started
This is wisdom to be celebrated!
Session rules:
Respectful communications, single speaker, all opinions count
Esther Derby and Diana Larsen approach:
Focus On Focus Off
Inquiry Rather Than Advocacy
Dialogue Rather Than Debate
Conversation Rather Than Argument
Understanding Rather Than Defending
3/3/2011 © 2011 BenchmarkQA Slide 15

16. At The Retrospective cont d
Review top defects for proper root cause classification and for
problematic process trends to address
Identify and group into categories what went well and what
could be done better
Or review pre-submitted items grouped by facilitator
Vote or rate; approaches:
1 to 3 or 1 to 5 rating item of highest significance gets highest
value
3 voting dots each member can place their 3 votes on different
items or all on one item
3/3/2011 © 2011 BenchmarkQA Slide 16

17. At The Retrospective cont d
For the top vote getters or highest rated items (recommend
no more than top 2 or 3):
Decide who will own the top item(s)
Team identifies possible root causes; use 5 why s
Team brainstorms ideas to experiment with that they think will
improve the issue
Team selects one or two ideas to try during next delivery cycle
Define action steps for the chosen solutions
3/3/2011 © 2011 BenchmarkQA Slide 17

18. After The Retrospective
3/3/2011 © 2011 BenchmarkQA, Inc. Slide 18

19. After The Retrospective
Keep the results of the retrospective in a prominent spot for
the team
Plan the selected action steps into the next delivery iteration
along with product work
Follow up on the experimented actions at the next
retrospective to see if had desired affect
Use the previous retrospective results as a starting point for
the next retrospective process
3/3/2011 © 2011 BenchmarkQA Slide 19

20. Lessons Learned
3/3/2011 © 2011 BenchmarkQA, Inc. Slide 20

21. Contributors
Sheila Conway Steve Anderson
Denise Fitzgerald Warren McLeod
Pat Stearns Aimée Willoz
Nirmal Chaudhari Shankar Jayavelu
Mark Nelson Santosh George
Susan Chapman Betty Schaar
3/3/2011 © 2011 BenchmarkQA Slide 21

22. Lessons Learned: General Thoughts
Try something different to achieve a different result.
A retrospective allows aggrieved team members to let off steam and
not carry it over to the next round of work.
It also helps make the case for new ideas with past experiences to
justify it rather than have someone from high above bringing about a
change for the sake of change.
I have seen too many retrospectives talk about what went well and
what did not go well; but never end up with implementable process
improvements that individuals have responsibility for.
3/3/2011 © 2011 BenchmarkQA Slide 22

23. Lessons Learned: General Thoughts
Problems observed during initial retrospectives:
1. No one talks, big silence.
2. Sometimes too much discussion on one issue.
3. QA folks are very nervous to talk about the issues.
4. Shutting down another person's views before coming to conclusion.
5. Bringing up the same issue again and again.
6. Very short retrospective.
Retrospectives can be very political where no one wants to be really
honest because they don t want to single any one person out regarding
their bad behavior, code quality, etc. So, a lot can go unsaid yet needs
to be brought out somehow by the facilitator.
3/3/2011 © 2011 BenchmarkQA Slide 23

24. Lessons Learned Planning
It can take time to get the thoughts out of everyone in the group.
Solicit the input ahead of the group session; this can also help with
anonymity.
Retrospectives held too far after the work period are less effective;
people forget.
Hold the Retrospective no more than 1 week after the delivery cycle
Holding two levels of retrospective helps. First on sprint level and
second after release/phase.
If a lot of issues occurred between two teams, split up their
retrospectives.
Retrospectives have been a formal 2 hour meeting with minutes.
3/3/2011 © 2011 BenchmarkQA Slide 24

25. Lessons Learned Planning
It really helps to have an agenda and rules of conduct.
Make sure ground rules are set beforehand. No personal attacks,
attack the process, etc.
If people think they can t speak to the group, send in the comments
to the Project Manager ahead of time.
If people cannot attend the meeting at all, ask for their comments
beforehand and share in the meeting.
I have done surveys as well but I don t think they work as well. It s
more difficult to get peoples perspectives and it s also harder to get
responses.
3/3/2011 © 2011 BenchmarkQA Slide 25

26. Lessons Learned Planning
Ask each team member to bring one thing that went particularly well
and one thing that went poorly.
Often only the recent pain or joy is remembered or only the Big
Stuff gets remembered.
Have a Project Notebook (lessons learned journal) for every member of
the team from the beginning, and capture the good, bad, and ugly along
the way.
Note the resolution for any challenges encountered.
It can be a great way to track what the team does well, even though
they may not realize it. It can be very enlightening to the rest of the
team when sharing individual entries.
3/3/2011 © 2011 BenchmarkQA Slide 26

27. Lessons Learned At/During
Items might come out that the group feels are important to their
success, but not in their control to resolve.
Still capture these, someone should own, but the group should also take
on one or more other items that are in their control.
Without an effort to make process changes, the results of the post-
mortems just don t go anywhere.
An action plan coming out of the effort can help to actually get some of
the suggestions implemented.
Bring in a moderator who knows nothing about the project.
Having a neutral moderator helps keep things on track and not get
steered in a particular direction.
3/3/2011 © 2011 BenchmarkQA Slide 27

28. Lessons Learned At/During
Bring in a scribe not involved in the project.
If the scribe knows nothing of the project, they can focus on taking
notes.
Have food.
It can be nice just to look back and appreciate the amount of effort
that went into a project even if you do not get to fix anything that
gets complained about.
A lot of the issues were discussed in hallway conversations during
the project. As a result, the effort can just turn into a complaint
session with little or no effort to fixing the problems.
3/3/2011 © 2011 BenchmarkQA Slide 28

29. Lessons Learned At/During
Put the one thing that went particularly well for the Sprint and one
thing that went poorly from each team member on a white board.
Get consensus from the team on two of the most critical things that
went well and two of the most critical items that went poorly.
Create a standard checklist or questionnaire which helps team to
stick to the point during retrospective. (e.g. requirements/user
stories, communications, development, unit test, database, QA
testing, data preparation, inter team dependencies, test
environment etc. focused topics).
Ask team members individually if feedback is not received.
3/3/2011 © 2011 BenchmarkQA Slide 29

30. Lessons Learned At/During
Make sure the questions are open ended.
To get people to open up, bring up specific things that happened
during the process. Specific meetings, documents due, process of
getting through the phase, etc.
Any effort to include metrics is helpful as they provide a somewhat
objective perspective on a given aspect of the project. Although,
they can also be political if bug rates, etc. are brought up.
3/3/2011 © 2011 BenchmarkQA Slide 30

31. Lessons Learned At/During
Review defects and root cause and use that for opportunities to
improve the process that results in defects.
I've seen fishbone diagrams and mind maps used to organize the
input, which seems to work well.
Capture action steps to get the most value out of the session.
Action plans should be created for the top items to improve those
that went poorly and standardize the items that went well.
3/3/2011 © 2011 BenchmarkQA Slide 31

32. Lessons Learned After
Find a way to keep the team accountable for trying changes.
Checkpoints mid-release to see if on track with the changes we said
we would make/do.
Without an effort to make process changes, the results of the post-
mortems just don t go anywhere. An action plan coming out of the
effort can help to actually get some of the suggestions implemented.
The most critical aspect of doing a retrospective or postmortem is
having management motivated to actually do something with the
data that's gathered (otherwise it's a waste of effort to gather it).
3/3/2011 © 2011 BenchmarkQA Slide 32

33. Lessons Learned After
Compare retrospective notes from different teams (if multiple
teams) and share that information.
Compare current retrospective to previous retrospective to
understand problems and improvements.
Always share notes immediately after retrospective. Do not wait for
another day.
Document all findings, send them out to the team for review.
Action plans should be assigned as tasks to individuals for the
following Sprint just like all the other tasks required to generate
software Sprint deliverables.
3/3/2011 © 2011 BenchmarkQA Slide 33

34. Let s Try One!
3/3/2011 © 2011 BenchmarkQA, Inc. Slide 34

35. A Retrospective About Retrospectives
The subject of our group retrospective is retrospectives
We ll use collected Lessons Learned for this exercise
Each attendee gets 3 votes to place on any of the items on
the wall place your votes on 1-3 items
We ll take the top vote-getter and work out some possible
root causes, possible solutions, action steps
3/3/2011 © 2011 BenchmarkQA Slide 35

36. References
Project Retrospectives: A Handbook for Team Reviews
Norman L. Kerth Dorset House Publishing ISBN 978-0-932633-44-6 2001
Agile Retrospectives: Making Good Teams Great
Esther Derby, Diana Larsen Pragmatic Bookshelf ISBN 978-0977616640 2006
Agile Testing A Practical Guide For Testers and Agile Teams
Lisa Crispin, Janet Gregory Addison Wesley ISBN 978-0-321-53446-0 2009
http://www.scrumalliance.org/
© 2011 BenchmarkQA Slide 36

37. Questions?
3/3/2011 © 2011 BenchmarkQA, Inc. Slide 37

38. In Closing
3/3/2011 © 2011 BenchmarkQA, Inc. Slide 38

39. Thank You!
For additional information on how BenchmarkQA can be of
assistance, please contact:
Molly Doyle Decklever
952.392.2384
molly.decklever@benchmarkqa.com
© 2011 BenchmarkQA