Digital products development: going behind the scene of product development - the benchmarking of digital product development costs - 2015 report by Alpha UX

1. Benchmarking Digital
Product Development Costs
2015 Report

2. IT’S HIGH TIME FOR BENCHMARKS
Clayton Christensen warned companies in the late 90’s of The Innovator’s Dilemma: placing
too much emphasis on customers’ past and current needs rather than anticipating their future
needs. The digital age accelerated the consequences of this scenario, as consumer demands
and expectations rapidly outpaced the ability for many large companies to respond.
Time and time again, nimbler and more customer-focused organizations entered the market
with digital offerings, grabbing market share from established players. The phrase “Kodak
Moment” has now become synonymous in business with tumbling from industry leader into
bankruptcy.
To avoid a similar fate, virtually every single brand today has entered, to at least some degree,
the digital product development business, whether it’s designing an ecommerce shopping
experience for mobile apps, bringing supply chain management capabilities to tablets in a
warehouse, or simply maintaining a website for investor relations. But little is known about
digital product development costs from industry to industry, and across organization sizes.
That’s unfortunate because, as it turns out, building digital products is quite a bit different
than building physical products. The proportional costs of building and planning are reversed,
different skills and capabilities are required, and scaling takes on an entirely different meaning.
Despite seemingly infinite new startups popping up continuously, it’s estimated that more than
90% fail, and mostly because of self-destruction rather than competition.
Before we can discover the best practices for planning and spending with regard to digital
product development, we first need to benchmark costs across industries and organization
sizes. Some of the questions we sought to answer in this report include:
What team members and resources are required to build digital products?
What products, services and methodologies do products teams need?
How do companies budget for new features and user research?
How much does it actually cost and how long does it take to build a digital product?
To answer these questions, and more, we surveyed product managers from numerous
industries and company sizes.
Benchmarking Digital Product Development Costs 2
https://en.wikipedia.org/wiki/The_Innovator%27s_Dilemma
https://s3.amazonaws.com/startupcompass-public/StartupGenomeReport2_Why_Startups_Fail_v2.pdf
http://www.forbes.com/sites/chunkamui/2012/01/18/how-kodak-failed/

3. The Benchmarking Digital
Product Development Costs
2015 survey was conducted by
Alpha UX and MindGrub
between July and October 2015.
The objective of this survey was
to understand the budgeting
process and costs associated
with building digital products
across industries. More than
100 digital product managers
completed this survey.
Alpha UX // Mindgrub Benchmarking Digital Product Development Costs 3
SURVEY METHODOLOGY

4. What do they do?
Who are they?
RESPONDENTS
Benchmarking Digital Product Development Costs 4
<50
employees
51 - 500
employees
501 - 2,000
employees
28%
2,001 - 10,000
employees
> 10,000
employees
27% 18% 11%16%
Other
Health Care
Manufacturing
Services
Technology6%
8%
15%
22%
49%
We actually had more than 20 industries represented in this survey, but simplified many of them
into four encompassing categories. Respondents from technology companies were the most
common, but services and manufacturing made up a sizeable segment, further illustrating the
prevalence of digital product development across industries.
Company Size
Industries
Our respondents hailed
from recently formed
startups to some of the largest
organizations in the world.
For our first foray into benchmarking the costs of digital product development, we
made sure to capture data from companies large and small. We focused primarily
on US companies and USD ($) so as not to overcomplicate the data with varying prices,
exchange rates, and salaries.
Alpha UX // Mindgrub

5. Without industry benchmarks, it’s difficult to outline and determine a reasonable
investment in digital. What are the costs involved? How long does or should it take to
build a product? Should digital products be built in-house or with an agency? And do
these factors vary across company size? Now that you have a better understanding of
the product managers we surveyed, we’ll analyze some of the above questions.
Benchmarking Digital Product Development Costs 5
THE PRODUCT DEVELOPMENT LIFECYCLE
More than half of our respondents spend more than $50,000 per month to build a digital
product. But the aggregate spending may be misleading as we can see when we break down
monthly costs by company size.
Monthly costs
$
$
$
$ $
$
$
$0 - $10,000
11% $
$10,001 - $25,000
16% $
$25,001 - $50,000
22% $
$50,001 - $100,000
24% $
>$100,000
27% $
How much does it cost?
Alpha UX // Mindgrub

6. As you might expect, larger companies generally spend more money on a monthly basis than
startups and smaller companies. Furthermore, as project durations increase, complexity and
therefore costs do as well.
Benchmarking Digital Product Development Costs 6
How long does it take?
>25 months
13-24 months
7-12 months
4-6 months
1-3 months
9%
36%
24%
24%
7%
Project duration
For the overwhelming majority,it takes 4-12 months to builda new product.
Monthly costs breakdown
by company size
Monthly costs breakdown
by duration of project
> 25 Months13-24 Months4-6 Months 7-12 Months
80%
75%
70%
65%
60%
55%
50%
45%
40%
35%
30%
25%
20%
15%
10%
5%
1-3 Months
> $100,000$50,001 - $100,000$10,001 - $25,000 $25,001 - $50,000<$10,000
> 10,0002,001 - 10,00051 - 500 501 - 2,000<50
Alpha UX // Mindgrub

7. Benchmarking Digital Product Development Costs 7
There is a prevalent notion that large organizations cannot innovate as quickly or effectively
as nimble startups due to increased layers of communication and other established processes.
But how does duration vary by company size? Are bureaucratic hurdles insurmountable or can
deep pockets overcome the nimbleness of the more scrappy startups?
Despite what one might expect, there is no strong correlation between company size and time
to market. Next we’ll analyze how spending differs depending on whether an organization
uses in-house development teams as opposed to agencies.
13-24 Months4-6 Months1-3 Months
50%
45%
40%
35%
30%
25%
20%
15%
10%
5%
<50
employees
>10,000
employees
2,001-10,000
employees
501-2,000
employees
51-500
employees
Project duration breakdown by company size
7-12 Months >25 Months
Alpha UX // Mindgrub

8. Benchmarking Digital Product Development Costs 8
Team sizes and makeups for in-house teams varied so drastically, it became impossible to
illustrate them in a single graphic. Instead, we tallied respondents who reported that their
teams had the makeups shown in the graph. Interestingly, although “being innovative” seems
to be an edict from the C-suite, only 11% of product teams have a C-level executive actively
involved. But don’t confuse that with a shortage of resources. 33% of product teams surveyed
have more the one product manager, while 76% have more than two engineers.
11%
c-suite
33%
> 1 PM
40%
QA
47%
UX/UI
76%
> 2 engineers
Who builds it?
14%
84%
Use in-house teams
Use an agency
2%
Blend
80%+ develop
products in-house.
Product Team Contains:Agency vs in-house development
Alpha UX // Mindgrub

9. In terms of costs, you could look at the breakdown in one of two ways. One might interpret this
chart to mean that agencies are used primarily for low-budget, non-critical projects, while
organizations use in-house teams, which are predominantly more expensive, for core product
development. Or, and possibly also true, agencies offer the capability to build products more
cost-effectively.
Project duration however does not appear to differ between in-house teams and agencies.
More research is needed to better understand the types of projects each undertakes, and
the methodologies each uses to limit costs and waste.
15% 5%10%30% 20%25%35%40%45%50%45% 55%50%30% 40%35%25%20%15%10%5%
>25 Months
13-24 Months
7-12 Months
4-6 Months
1-3 Months
>$100,000$50,001 - $100,000$25,001 - $50,000$10,001 - $25,000$0 - $10,000
10% 5%15%20%25%30%35%30% 35%25%20%15%10%5%
Benchmarking Digital Product Development Costs 9
Project duration breakdown by in-house
Monthly costs breakdown by agency Monthly costs breakdown in-house
Project duration breakdown by agency
Alpha UX // Mindgrub

10. Benchmarking Digital Product Development Costs 10
TOOLS & METHODOLOGIES
As digital product development becomes more prevalent, a wealth of tools, services,
and methodologies have emerged to support the discipline. In this chapter, we’ll take
a look at some of the most popular ones.
What exactly are these tools, services and methodologies? How do they budget for product
development?
What tools are used?
Project management, collaboration, and cloud
offerings dominated the list of most commonly
used tools. Two design and prototyping tools,
Axure and Photoshop, also made the list.
Monthly spend on tools
24%
I don't know
15%
>$10,000
12%
$5,001 - $10,000
19%
$2,501 - $5,000
17%
$1,001 - $2,500
13%
$0 - $1,000
Most Commonly
Used Tools
Alpha UX // Mindgrub

11. $0 - $1,000 I don’t know$1,001 - $2,500 $2,501 - $5,000 $5,001 - $10,000 >$10,000
> 10,000
employees
2,001 - 10,000
employees
501 - 2,000
employees
51 - 500
employees60%
55%
50%
45%
40%
35%
30%
25%
20%
15%
10%
5%
< 50
employees
Benchmarking Digital Product Development Costs 11
About half of respondents spend between $1,000 and $10,000 on tools per month.
Interestingly, 24% of product managers reported that they were unsure how much they
were spending monthly on tools. When we looked at the data, this was more prevalent
as companies got larger, which could mean that resources are shared across the organization
for access to tools and services.
Perhaps the most concerning data in the report has to do with how organizations budget
for digital product development. Apparently using many of the holdover methodologies from
budgeting for physical production, more companies either have no defined process, don’t
budget, or use a ‘best guess’ than companies that budget iteratively.
This is unfortunate because one of the many differences between digital products and
physical products is that digital products can be validated and built iteratively, enabling
product teams to gain feedback and fold it into a new version as they go. Being able
to budget iteratively is critical for allocating resources and even being able to kill a
product without wasting resources when it becomes apparent that users don’t want it.
What methods are used for budgeting?
$$$$
in-house
process
agency
quote
we don't
budget
historical
data
no defined
process
I don't
know
iterative engineering
estimate
2% 2% 5% 5% 6% 6% 9% 12% 54%
best
guess
Budgeting process
Monthly tools spend breakdown by company size
Alpha UX // Mindgrub

12. C-Suite
Finance
Training
Legal
Quality Assurance
Sales
Marketing
Departments affected
20%
13%
5%
4%
3%
3%
1%
Benchmarking Digital Product Development Costs 12
RAMIFICATIONS OF A FAILED PRODUCT
We know that most new products fail...but what are the ramifications? What is at risk? Aside
from product teams, what other departments are affected? And what contributes to failure?
One of the most overlooked ramification of
a failed product launch is that product teams
aren’t the only ones impacted. Other
departments, most predominantly marketing
and sales, often dedicate considerable time
and resources as well, so it’s imperative
products teams be diligent in validating and
ensuring that products are built to meet
customer demands.
Note that this is just the percentage that
agreed to speak with us (although many more
may have launched a failed product), and that
a large portion of respondents are working at
established organizations.
How many products fail?
Reported
Failed Product
Launch
Did Not Report
Failed Product
Launch
40%
60%
Who gets impacted?
Reported failed product launches
Alpha UX // Mindgrub

13. “There was nothing that was done to validate the product concept before launching.
The general manager for the business unit came up with the idea and we had to execute
on it, regardless of whether we thought it was a good idea or not. This was the ultimate
reason why the product failed.”
“I practice Lean and use extensive prototyping but my company doesn't formally
embrace this approach; we have an informal agile process and overall product strategy
tends to get lost in lieu of releasing endless features defined by customer or internal
requests. It's the danger of poorly executed Agile brought to life.”
“We were prescribed the product requirements in advance and as a result did not do
proper due diligence in advance of the build.”
Benchmarking Digital Product Development Costs 13
“The failure stemmed from other departments unwillingness to contribute resources
and energy to change outdated practices. It was their lack of inertia that created the
lackluster result.”
There are many factors that can contribute to failure, including an inability to acquire
customers, running out of budget, or building something that people don’t actually want.
Not all of our respondents were willing to talk about their failed products, but a few offered
insight into what went wrong.
What causes products to fail?
https://www.cbinsights.com/research-reports/The-20-Reasons-Startups-Fail.pdf
Alpha UX // Mindgrub