2. 9/6/19
2
Tasks
1. Plan work incrementally
2. Gain consensus on Just in time (JIT) acceptance criteria
3. Tune process to organization, team and project
4. Release minimal viable products
5. Work in small batches
6. Review often
7. Prioritize work
8. Refactor often
9. Optimized environmental, operational and infrastructure
factors
10. Review and checkpoint often
11. Balance adding value and reducing risk
3
Tasks
12. Reprioritize periodically to maximize value
13. Prioritize nonfunctional requirements
14. Review and improve the overall process and product
4
3. 9/6/19
3
Why Is Value – Driven Delivery?
• Delivery Value Early (Eat your dessert first)
5
Build
Test
DeployPlan
Design
Build
Test
DeployPlan
Design
Envision
High Level
Plan Build
Test
DeployPlan
Design
Release
66
Value
5. 9/6/19
5
ASSESSING VALUE
Assessing Value
• Financial Assessment Metrics
• [T&T] Return on Investment (ROI)
• [T&T] Net Present Value (NPV)
• [T&T] Internal Rate of Return (IRR)
• [T&T] Earned Value Management (EVM)
• [K&S] Agile Project Accounting
• [K&S] Key Performance Indicators (KPIs)
• Managing Risk
• [T&T][K&S] Regulatory Compliance
12
6. 9/6/19
6
Assessing Value
• Financial Assessment Metrics
– Return on investment (ROI)
– Internal Rate of Return (IRR)
– Net Present Value (NPV)
13
Assessing Value
• [T&T] Return on Investment (ROI)
– The ratio of the benefits received from an investment to the money
invested in it, expressed as a percentage
14
7. 9/6/19
7
Assessing Value
• Return on Investment (ROI) (cont.)
– Present Value
15
Assessing Value
• [T&T] Net Present Value (NPV)
– The present value of are venue stream( income minus costs) over a
series of time periods
16
8. 9/6/19
8
Assessing Value
• [T&T] Internal Rate of Return (IRR)
– The discount rate at which the project inflows (revenues) and project
outflows (costs) are equal
17
Assessing Value
9. 9/6/19
9
Assessing Value
• [T&T] Earned Value Management (EVM)
– One tool commonly used to track project spending is an “S-curve.”
19
Assessing Value
• [T&T] Earned Value Management (EVM)
– Constructing an Agile Earned Value Tool
20
10. 9/6/19
10
Assessing Value
• [K&S] Key Performance Indicators (KPIs)
– Rate of progress
• How many features or user stories are getting completed and
accepted by the product owner per week or month?
– Remaining work
• How much work is left in the backlog?
– Likely completion date
– Likely costs remaining
21
Risk Management
• Risk (impediment)
• Managing Risk
22
11. 9/6/19
11
Risk Management
• Risk (impediment)
• Managing Risk
23
Review
(@ Retrospective)
Identify
(@Daily Scrum; Retrospective;
Requirement Workshop; Sprint
Planning; Sprint Review)
Assess (Likelihood, impact,
Response)
Respond
(Avoid, Mitigate, Transfer,
Accept)
343
4
Risk Management
Ø In Agile, risks are considered as anti value
Ø The risk management process is repeated everyiteration
Ø The four steps in risk management cycle are:
Ø Risk Identification
Ø Risk Assessment
Ø Risk Response
Ø Risk Review
Ø The product backlog is continually reviewed and adjusted for the
risks
Ø Risk based spikes are planned for high value risks
12. 9/6/19
12
Assessing Value
• [T&T][K&S] Regulatory Compliance
– Many agile projects operate in a highly regulated environment
and have to deal with regulatory compliance issues
– Typically projects that are subject to regulatory compliance
require special documentation to prove that the required
practices were followed.
– There are two simple approaches for incorporating regulatory
compliance work into agile projects
• Doing the compliance work “as you go” keeps it linked and
relevant to the current work
• Doing the compliance work after product development
doesn’t allow us to tweak our practices on the go, but it
does avoid the issue of rework
• Most organizations adopt a hybrid approach
25
PRIORITIZING VALUE
13. 9/6/19
13
Prioritizing Value
• [T&T] Customer – Value
Prioritization
• [K&S] Prioritization
Schemes
• [T&T] MoSCoW
• [T&T] Relative
Prioritization/Ranking
27
99
Customer Value - Prioritization
14. 9/6/19
14
[K&S] Prioritization Schemes
• CARVER relative to the objective and mission of the project
– Criticality – how important to be done up front
– Accessibility – can work on it immediately? or depends on other work
/ skills?
– Return – ROI / NPV / IRR
– Vulnerability – how easy to achieve the desired results?
– Effect – what are the effects on the project (help moving towards the
goal of the project)?
– Recognizability – have the goals been clearly identified?
29
202
0
[K&S] Prioritization Schemes
15. 9/6/19
15
[K&S] Prioritization Schemes
– Simple Schemes
• rank from high to low (priority 1, 2, 3, ...)
– MoSCoW
– Monopoly Money
• the project budget and ask them to distribute those funds
amongst the system features
31
101
0
[K&S] Prioritization Schemes
20. 9/6/19
20
Minimal Viable Product (MVP)
42
• The minimal product (with just essential features and no
more) that allows can be shipped to early adopters see and
learn from the feedback instantly.
• The concept is somewhat similar to Minimally Marketable
Feature (MMF) in which MVP is the first shippable product
with the first set of MMF.
Minimal Viable Product (MVP)
45
• The minimal product (with just essential features and no
more) that allows can be shipped to early adopters see and
learn from the feedback instantly.
• The concept is somewhat similar to Minimally Marketable
Feature (MMF) in which MVP is the first shippable product
with the first set of MMF.
Minimal Viable Product (MVP)
45
• The minimal product (with just essential features and no
more) that allows can be shipped to early adopters see and
learn from the feedback instantly.
• The concept is somewhat similar to Minimally Marketable
Feature (MMF) in which MVP is the first shippable product
with the first set of MMF.
Minimal Viable Product (MVP)
21. 9/6/19
21
[T&T] Agile Tooling
• Just as the Agile Manifesto values “Individuals and interactions
over processes and tools,” agile teams prefer low-tech, high-
touch tools over sophisticated computerized models.
– Ex: Sophisticated scheduling tools can also exclude team
members from interacting with the tools and discourage
whole-team collaboration
– Let’s look at these problems in more detail
• Data accuracy perception increases
• Barriers for stakeholder interaction are created
44
[T&T] Agile Tooling
• Low Tech, High-Touch Tools
– Instead of using high-tech tools, agile teams prefer to employ
a “low- tech, high-touch” approach to planning and tracking
• Ex: Trello (Demo with Trello in managing user story)
– With low-tech, tangible objects promote communication and
collaboration, which is where learning and knowledge
transfer really occur on a project
45
22. 9/6/19
22
[K&S] Delivering Incrementally
• [T&T] Task/Kanban boards
46
373
7
Ø An "information radiator"
- ensures efficient
diffusion of information
Ø Can be drawn on a
whiteboard or even a
section of wall
Ø Makes iteration backlog
visible
Ø Serves as a focal point for
the daily meeting
Ø Story cards can be
quickly and easily
moved to update status
[K&S] Delivering Incrementally (TASK BOARD)
23. 9/6/19
23
[K&S] Delivering Incrementally
• [T&T] Work in Progress (WIP)
• [T&T] WIP Limits
48
383
8
Delivering Value - Limit WIP
Ø WIP (work in progress) also known as “work in
process”
Ø Includes work that has been started but not
completed yet
Ø Represents money spent with noreturn
Ø Hides process bottlenecks that slow the processes
Ø Represents risk in form of potential risk
Ø Agile processes aim to Limit and optimize WIP
Ø Optimal WIP makes processes effecient
24. 9/6/19
24
Prioritizing Value
• [T&T] Cumulative Flow Diagrams (CFDs)
50
Prioritizing Value
• [T&T] Cumulative Flow Diagrams (CFDs)
– Little’s Law
51
25. 9/6/19
25
Delivering Value - Timeboxing
Ø NG: Student syndrome: a person will only start to apply
themselves to an assignment at the last possible moment
before its deadline
Ø NG: Parkinson’s law: work expands so as to fill the time available for
its completion
Ø GOOD
Ø Timeboxing is setting a fixed time limit to activities
Ø If something cannot be accomplished in a timeboxed period, it
is deferred to the next period
Ø Allows velocity to be determined between iterations and sprints
Ø Applies everything: Scrums, Sprint planning, Sprints and
iterations, riskspikes
393
9
Delivering Value - Quality
Ø Agile embeds quality throughout the project
lifecycle
Ø Quality is “built in” in agile approach
Ø Pair programming
Ø Test Driven Development / Test-First Development
Ø Acceptance Test Driven Development
Ø Collaborative definition of done
Ø Continuous integration
26. 9/6/19
26
404
0
Delivering Value – Continuous Improvement
Ø Daily standup
Ø Sprint demos => Spring Review
Ø Retrospectives
Ø Highest value on quality
Ø Continuous Integration
Ø Process Improvement
484
8
Tracking Value – ReportingProgress
Burndown Chart
Cumulative Flow Diagram
27. 9/6/19
27
[K&S] AGILE CONTRACTING
Agile Contracting
• Agile Constraints and Contracts
• Money for Nothing and Change for Free
• Graduated Fixed-Price Contract
• Fixed-Price Work Packages
• Customized Contracts
57
28. 9/6/19
28
Agile Contracting
• Agile Constraints and Contracts
58
Agile Contracting
• Money for Nothing and Change for Free
– Changes will be free if the total amount of contracted work has not
changed
– “Money for nothing” allows the customer to terminate the project
early when they feel there is no longer sufficient ROI in the backlog to
warrant further iterations
59
29. 9/6/19
29
Agile Contracting
• Graduated Fixed-Price Contract
– both parties share some of the risk and reward associated with schedule
variance
• Fixed-Price Work Packages
• Customized Contracts
60
Verifying and Validating Value
30. 9/6/19
30
Verifying and Validating Value
• [T&T] Frequent Verification and Validation
• [T&T] Testing and Verification in Software Development
• Exploratory and Usability Testing
• [T&T] Continuous Integration
• Test Driven Development
• Acceptance Test-Driven Development (ATDD)
62
Verifying and Validating Value
• [T&T] Frequent Verification and Validation
63
31. 9/6/19
31
Verifying and Validating Value
• [T&T] Testing and Verification in Software Development
64
Verifying and Validating Value
• Exploratory Testing
– Exploratory testing differs from scripted testing that
attempts to exercise all the functional components of a
system; instead , it relies on the tester’s autonomy, skill,
and creativity in trying to discover issues and unexpected
behavior.
• Usability testing
– attempts to answer the question, “How will an end user
respond to the system under realistic conditions?
65
34. 9/6/19
34
Planning Value – Six Levels of Planning
Organization focus
• Strategy: Business goals and roadmaps
agreed by the Executive Leadership
• Portfolio: Selection of the products that
will best implement the vision
• Product: Looking and planning forthe
evolution of released system
Team focus
• Release: Features of each release that
support the Product plan
• Iteration: Tasks needed to transform a
feature request into working, tested
software
• Daily: Daily Scrum and work activities
Planning Value – Product Backlog, Release, Sprint
Product Road Map
Ø Visual overview of product’s releases and its main components
Ø Provides a quick view of primary release points and intended
functionality
Product Backlog
Ø Contains all user stories, themes, and epics.
Ø Product owner prioritizes features, epics and stories on theirvalue
Ø If something is not in the product backlog, it is not in theproduct
Release
Ø Releases are used to support product roadmaps
Ø Product owner selects the items from the backlog that meet thegoals
of a release.
39. 9/6/19
39
Ø How Much?
§ Scrum doesn’t specify
§ We can write as much or as little as we choose.
§ It’s up to the Product Owner and Team to decide
together
Ø Is it useful to create highly detailed written specifications for
every single Product Backlog Item in advance?
§ The time required to write is high
§ The time required to read is high
§ If a requirement is dropped, time will have been
wasted
§ The risk of misunderstanding is high
Written Requirements
Story Cards
Story Cards
Writing Specifications Effectively
User stories
as a very
widely used
format
User Stories are short,
plain-language
descriptions of
features, centered on
the customer and what
they need to do
The 3 C’s: Card,
Confirmation,
Conversation
40. 9/6/19
40
313
1
User Stories
Ø A lightweight mechanism to quickly capture requirements
Ø 3Cs: Card, Conversation and Confirmation
Ø How to Evaluate a good Story INVEST
Ø Acts as an agreement between customers and development
team
Ø Every requirement is a user story
Ø Every story, including technical stories, has a value
Ø Common structure of a user story
Ø Multiple levels - Features, Epics& Stories
As a <user type (actor)>
I <want to/need, etc> goal (usecase) So
that <value>
“As a customer, I want
to pay for the items in
my shopping cart
using a credit card, so
that I can have
them shipped to me”
As a
I want to…
So that…
1. Card – Front of Card
41. 9/6/19
41
2. The Confirmation – Back of Card
“As a customer, I want to pay for the
items in my shopping cart using a
credit card, so that I can have them
shipped to me”
“ The back of the card outlines the
test cases for this feature – how
it’s going to be confirmed.
• Accepts Visa, MC, AmEx and
rejects other types of card
• Rejects invalid card number or
expired card
• Accepts different dollar amounts
• Rejects amount higher than the
card limit
Acceptance Criteria
Notes
Ø The card only covers the most basic information
Ø The next level of detail comes in conversations
between the Product Owner and the Team
Ø When the team starts to understand the user story,
the following questions will make understanding of
the element much easier:
§ Who the user is
§ What the user needs to do
§ Why the user needs to do that
3. The Conversation
42. 9/6/19
42
Example – Online Shopping Portal
As a user,
I can read the reviews
of the books that Iwant
to purchase
As a
shipping
manager , I
can list of
pending
orders
As a user, I can restrict
searches so that I only
see photos of thegoods
for purchase
As a user, I
can return
the goods
purchased
Details added in smaller sub stories
As a user, I can return
the goods purchased
As a premium site member,
I can return the goods
purchased with no penalty
(with in 7 days)
As a non premium site
member, I can return the
goods purchased with 10%
penalty
As a site visitor, I am
emailed a confirmation of
any cancelled request
As a user , I register a
request to return the goods
purchased
43. 9/6/19
43
Ø High Level tests are added to the story
o Can be used to express additional details and expectations
As a user, I can return the goods purchased:
o Verify that a premium member can return the goods without
a penalty if the goods are return in 7 days of purchase
o Verify that a non-premium member is charged 10% for
return of purchased goods
o Verify that an email confirmation is sent
o Verify that the credit card is notified of any cancellation
o Figure out what to do if the user’s card is expired
Details added as tests
User Stories – More details
Before they are
included in a sprint,
user stories are
estimated in story
points.
These are intentionally
rough estimates that
are primarily used to
manage the backlog
and to help prepare for
the sprint.
When a sprint starts,
team will break the user
story down into the
tasks that are required
to implement it.
Team needs to work
with PO in the product
backlog grooming
meeting to determine
which stories need more
data before it can be
broken down into tasks
to estimate the effort.
It should be the basis for
a conversation &
negotiation with your PO
and customers that
continues until the user
story has been finished
and accepted.
Too much detail can
impair the negotiation by
creating an implication
of precision that is not
accurate.
44. 9/6/19
44
Sample User Stories
Story (Business Requirement) Story Points
As a BUYFROMME user, I want to click on a thumbnail photo to see a
full-sized photo of a book
13
As a BUYFROMME user, I want to enter a piece of text and click
“Search books” and then see a list of all books that match any part
of that text (product search)
8
As a BUYFROMME user, I want to select a price range, and see all
the books that are within that price range
5
Total 26
45. 9/6/19
45
Review
• Agile contracting
• Agile project accounting principles
• Agile risk management
• Agile tooling
• Compliance/regulatory compliance
• Cumulative flow diagrams (CFDs)
• Customer-valued prioritization
• Earned value management (EVM)
for agile projects
• Frequent verification and validation
• Incremental delivery
• Managing with agile KPIs
• Minimal viable product (MVP)
– Minimal marketable feature (MMF)
92
• Prioritization schemes
– Kano analysis
– MoSCoW
• Relative prioritization /
ranking
• ROI/NPV/IRR
• Software Development
Practices
– Continuous integration
– Exploratory and usability
testing
– Red, Green, Refactor
– TDD, TFD, ATDD
• Task/Kanban boards
• Value-driven delivery
• Work in progress
– WIP limits
References
• PMI-ACP Exam Prep 2015 By Mike Griffiths, PMI-ACP, PMP
• Many other resources from Internet
93
46. 9/6/19
46
Thank you for your attention!
94
20 Product Prioritization Techniques:
A Map and Guided Tour
48. 9/6/19
48
The Kano Model
98
• Noriaki Kano, a Japanese researcher and consultant,
published a paper in 19843 with a set of ideas and
techniques that help us determine our customers’
(and prospects’) satisfaction with product features.
– Customers’ Satisfaction with our product’s features
depends on the level of Functionality that is provided (how
much or how well they’re implemented);
– Features can be classified into four categories
– You can determine how customers feel about a feature
through a questionnaire
The Kano Model
99
• Satisfaction vs. Functionality
– Kano proposes two dimensions to represent how
customers feel about our products:|
• one that goes from total satisfaction (also
called Delight and Excitement) to total
dissatisfaction (or Frustration)
• and another called Investment, Sophistication
or Implementation, which represents how
much of a given feature the customer gets, how
well we’ve implemented it, or how much we’ve
invested in its development.
49. 9/6/19
49
The Kano Model
100
• The Four Categories of Features
– Performance: the more we provide,
the more satisfied our customers
become.
– Must-be: If the product doesn’t have
them, it will be considered to be
incomplete or just plain bad.
– Attractive: There are unexpected
features which, when presented,
cause a positive reaction.
– Indifferent: Those which their
presence (or absence) doesn’t make
a real difference in our reaction
towards the product
The Kano Model
101
• Determining how customers feel through a
questionnaire
– In order to uncover our customer’s perceptions towards the
product’s attributes, we need to use the Kano questionnaire.
It consists of a pair of questions for each feature we want to
evaluate:
• One asks our customers how they feel if they have the feature;
• The other asks how they feel if they did not have the feature.
• The first and second questions are respectively called the functional
and dysfunctional forms. To each “how do you feel if you had / did not
have this feature”, the possible answers are:
– I like it I expect it
– I am neutral I can tolerate it
– I dislike it
50. 9/6/19
50
The Kano Model
102
• Determining how customers feel through a
questionnaire
2. Quality Function Deployment
103
• a way to help us focus on product features
viewed from different angles, in particular, the
customer and the company
1. Identify customers’ wants and needs
2. Identify the “Voice of the Customer”
3. Identify the How’s (The Voice of the Company)
4. Relationship between “Voice of the Customer”
and “Voice of the Company”
• 9 → Direct and Strong relationship
• 3 → Moderate relationship
• 1 → Weak/indirect relationship
• Blank → No relationship
5. Generate Priorities
6. Examine Priorities
52. 9/6/19
52
4. Buy a Feature
106
• Buy a Feature is a fun innovation game that can be played
collaboratively or individually. Here’s how it works:
– A set of features that need to be prioritized are presented to a
group of buyers (our customers);
– Each buyer gets a budget of play money to spend on features;
– Each feature is priced according to some measure of cost
(complexity, effort, actual cost to develop, etc.) — as long as it’s
the same criteria for all features, you can use any one you
prefer;
– Each player’s budget should be between a third to half of the
total cost for all features;
– It’s possible to play the game in one of two ways:
– Individually — Players are told to use their budget to buy the
features that are most important to them;
– Collaboratively — Using a pricing scale that makes some
5. Story Mapping
107
• The main idea behind Story Maps is that single-list product
backlogs are a terrible way to organize and prioritize the work that
needs to be done. A richer structure is necessary.
– There’s a horizontal axis that represents usage sequence;
– The vertical axis stands for criticality;
– Groups of related user stories can be grouped as Activities
53. 9/6/19
53
6. MoSCoW
108
• The term is an acronym with each letter standing for one of the
possible prioritization categories (with Os added to make it
memorable.) Requirements are thus classified as:
– Must have — these are critical and must be included into the
product.
– Should have — these requirements are important but not
crucial for the release. They’re the first level of “Nice to have”,
and generally share the importance of MUST requirements,
without being so time-sensitive.
– Could have — these requirements are desirable but not
necessary for the release. They’re usually low-cost
enhancements to the product.
– Won’t have — these are considered to be the least-critical or
even not aligned with the product strategy.
6. MoSCoW
109
• The term is an acronym with each letter standing for one of the
possible prioritization categories (with Os added to make it
memorable.) Requirements are thus classified as:
– Must have — these are critical and must be included into the
product.
– Should have — these requirements are important but not
crucial for the release. They’re the first level of “Nice to have”,
and generally share the importance of MUST requirements,
without being so time-sensitive.
– Could have — these requirements are desirable but not
necessary for the release. They’re usually low-cost
enhancements to the product.
– Won’t have — these are considered to be the least-critical or
even not aligned with the product strategy.
54. 9/6/19
54
7. Prune the Product Tree
110
• Here’s how it works:
– Draw a large tree on a whiteboard or sheet of paper;
– Thick limbs represent core product areas and its outermost
branches represent currently available features;
– Write potential new features on some Post-It notes;
– Ask customers and stakeholders to place their desired features
on the tree, thus defining its next phase of growth.
8. Speed Boat
111
• Draw a boat on a whiteboard or large piece of paper;
• This is a speed boat, and it should go really, really fast;
• Unfortunately, it’s being held back by some anchors;
• The boat is the product and anchors are the features that
customers feel frustrated with;
• Ask customers to write on Post-it notes the features
they’re not happy with and how much faster they
estimate the boat could move without those anchors;
• Each anchor and speed estimate will give you a measure
of “pain” which you can later prioritize for improvement.
55. 9/6/19
55
Financial analysis
112
• many organizations require a business case for new
product features
• There are 4 kinds of financial goals we can expect as a
consequence of improving the product in some way:
– New Revenue: new income that is projected to be generated;
– Incremental Revenue: additional income from existing
customers by now being able to charge for an upgrade or
additional services;
– Retained Revenue: income that’s not lost because customer
churn is reduced;
– Cost Savings: any type of operational efficiency that is gained
inside the company.
Financial analysis
113
• 9. Net Present Value (NPV)
• 10.Internal Rate of Return (IRR)
• 11.Discounted Payback Period
56. 9/6/19
56
12. Ian McAllister’s Framework
114
• I don’t think this framework has an official name (hence the uncreative one I’m using.)
1. Define the important themes for the product or business
2. Create a list of the most important themes (e.g. customer acquisition, engagement,
activation, ARPU) and select the top three
3. Prioritize and resource the themes
4. Define the relative priority for each theme and how much resources you want to invest in
each (team members, marketing, etc.)
5. Generate project ideas
• Use projects ideas you already have for each theme and come up with new ones. Keep in
mind the Pareto principle and focus on the 20% of the project that will get you to 80% of
the desired outcome.
• Estimate each project’s potential impact, Estimate each project’s costs
• Work out the impact you expect from each project, in very broad terms (think order-of-
magnitude-similar.)
• With your team’s (and relevant stakeholder’s) help, come up with an estimation for each
project’s costs.
• Prioritize project within each theme
• Set priorities considering the projects with the best impact-to-cost ratios.
13. Impact on Business Goal
115
• Following this line of thinking, Dave McClure introduced the
AARRR metrics for startups9. They’re centered around 5 stages in
the customer lifecycle:
– Acquisition: users come to the site / product;
– Activation: users enjoy 1st visit, signup;
– Retention: users come back multiple times;
– Revenue: users’ activity leads to revenue for the product;
– Referral: users like product enough to recommend to others.