1. BENCHMARKS STILL COUNT
Keys to Effective Use of Contractual Measurement Clauses
By Lawrence Kane, Sr. Leader, IT Infrastructure Strategy, Sourcing, and Asset Management
Execution, Boeing
www.isg-one.com

2. INTRODUCTION
Rapid advances in technology, improvements in operating processes, and
increased competition are all driving declining market prices for IT services. In
this environment, predicting the fair value of services to be provided in the
future can be difficult. Options to ensure that prices for outsourced IT services
remain competitive over time may include negotiating short-term contracts
and frequently re-bidding, or negotiating long-term contracts and
benchmarking competitiveness of prices on a regular basis.
Long-term contracts need to be structured to anticipate expected changes in
technologies and reflect expected efficiencies in forward pricing. While
benchmarks can address this need, many service providers today tell their
customers that benchmarking clauses in outsourcing contracts have outlived
their usefulness. Benchmarks provide limited value, some say, creating
contention, damaging the relationship, and distracting both parties from
more important business issues. Indeed, compared to five or seven years ago,
relatively few contracts today include clauses specifically mandating periodic
reviews of pricing and service quality.
For many businesses, however, executing benchmark clauses that assess
pricing in the context of competitive market standards remains an essential
foundation of an effective outsourcing negotiation, contracting, and
management strategy. Many top-performing global businesses employ
benchmarks as a core element of their operational philosophy, assuring
continuous competitiveness of in-sourced and outsourced operations,
leading-edge contracting practices, and ongoing operational efficiency.
This ISG white paper, guest-authored by a senior IT leader at Boeing,
examines how benchmark clauses can be applied to ensure competitiveness,
drive continuous improvement, gain insight into performance drivers, and
facilitate a positive relationship between customer and service provider.
BENCHMARKS STILL COUNT ■ LAWRENCE KANE 1

3. HIGH SUCCESS RATE
A pioneer in the use of comparative analysis, Boeing first For some organizations and in certain situations, the
employed benchmark reviews of IT outsourcing contracts “soft” approach has proven to be effective. Organizations
more than a dozen years ago. The fundamental objective have had success using the “soft” approach to help frame
was—and remains—to ensure and maintain the their negotiating position, most often when going into a
competitiveness of outsourced services over the life of a sourcing new activity, or renegotiating at the end of an
contract term. Given the dynamic nature of technology existing agreement. Enterprises value insight that
innovation and market trends, Boeing recognized that an benchmarking of (potential) sourced services can
agreement that was competitive on day one could provide, compared to a simple current cost to market
quickly fall out of alignment with rapidly changing market price comparison. Often this type of softer approach can
standards. Periodic and rigorous benchmark reviews be viewed as a “proxy bid.”
were therefore considered the most effective way to For many, however, clearly defined, binding and
validate and, if needed, adjust pricing, service levels, or actionable terms are imperative to a successful
contract Terms and Conditions over time. By enabling benchmark initiative. In Boeing’s view, undertaking a
adjustments, benchmarks were also seen as way to keep benchmark process that does not result in specific
relationships on track and avoid contentious consequences essentially defeats the purpose of the
renegotiations or costly and time-consuming re- initiative. While some argue that a binding benchmark
competitions. clause is likely to produce contention between the two
Since 1999, Boeing has conducted dozens of benchmark parties, Boeing has found that clearly defined terms can,
reviews of IT infrastructure outsourcing, business process in fact, facilitate long-term relationships.
outsourcing, and IT application outsourcing contracts. In By structuring contract negotiations to include
the majority of instances, the analyses resulted in benchmarking milestones, service providers competing
contractual price reductions, cost reductions through for Boeing’s business know the rules of the game
internal process improvements, and/or implementation entering the agreement. While a benchmark analysis
of improved practices. Even after benchmarking each might consume time and resources, both parties benefit
service area in eighteen month to two-year intervals for when independently vetted data either confirms
over a decade, recent studies have yielded approximately competitiveness or points to specific changes that must
a 5 : 1 return in direct cost savings, with indirect benefits be made. Without these studies, it is challenging to pen a
(e.g., improved Service Level Agreements, industry long-term contract with any assurance that IT can
leading practices) far exceeding that. demonstrate a commitment to competitiveness for its
Beyond quantitative benefits, the benchmarking process customers.
inspires confidence by validating how services are being Moreover, an effective service provider can use the
delivered in a comparative context using an independent benchmark results to demonstrate the value they deliver,
third-party benchmarker. In addition, this analysis not just from a pricing standpoint, but in terms of
provides a valuable reality check that assesses additional benefits around service quality and process
contractual Terms and Conditions against the day-to-day improvement. For example, while Boeing and the service
real life of the environment – a perspective that can provider typically share a benchmarker’s fees, some
inform additional improvement both from the service suppliers have voluntarily paid the entire cost of a study
provider and within the customer organization. in order to provide facts and data that could lead to a
renegotiation rather than re-competition at expiration of
BENCHMARK STRATEGIES their contract. Such decisions are by no means
guaranteed, but this behavior not only demonstrates
Organizations take a variety of approaches when goodwill but also faith in the benchmarking process and
employing benchmark clauses. One consideration is credibility of the results.
whether to use the benchmark findings as a non-binding
guide or to make the analysis results “actionable.” In the When benchmarking is characterized by a stringent
former the findings form a basis for further negotiation process that assures participants their competitiveness
whereas in the latter the service provider is contractually will be fairly evaluated, when it yields actionable results,
obligated to adjust pricing anomalies revealed during the
process.
BENCHMARKS STILL COUNT ■ LAWRENCE KANE 2

4. and when all parties share a willingness to use the data high degrees of process discipline along with the
for decision-making, the exercise can be much more than contractual freedom to implement their tools and/or
a “stick” wielded by the client. Rather, it has the best practices, but can pose risks for both the buyer and
potential to become a win/win proposition that delivers seller. From a buyer’s perspective pricing might be front-
inherent value for both parties. loaded, asynchronous with the marketplace, or simply
not competitive. On the seller’s side, profitability could
be at risk if the projected rate of improvements is overly
HOLISTIC VS. GRANULAR optimistic or the state of the current environment
Another consideration in implementing a benchmark insufficiently known.
clause is determining the appropriate level of detail to
A “step-down” pricing approach, while complementary
pursue. Some organizations favor a high-level analysis
to a benchmarking program, has not proven to be an
that presents a general overview of the environment.
adequate substitute. Without the fact-based analysis of a
While this approach is relatively inexpensive and
benchmark, providers can artificially inflate pricing in
oftentimes easier to implement, it typically offers little
year one in order to show a decline over time. Moreover,
insight into where performance issues lie, in part
the step-down pricing can be a numbers game, and even
because ensuring apples-to-apples comparisons where
when suppliers can predict realistic savings, a benchmark
all complexity factors have been accounted for presents
is still needed to quantify improvements and assess
a challenge, but largely because aggregated data tends to
those improvements against the market. Similarly “most
mask underlying problems. The value of such studies,
favored pricing” clauses which assure competitive service
therefore, lies less in stand-alone results and more in
offerings are also a good supplement, but only reflect the
identifying areas to look and “firing orders” for doing so.
service provider’s comparison with itself, rather than
The alternative is a more detailed and rigorous— and with the entire marketplace for like services and service
more costly and time-consuming—analysis that drills levels.
down to identify specific anomalies and root causes of
performance issues at a granular level. This perspective
IMPLEMENTING IMPROVEMENTS
enables an organization to not only understand what’s
causing problems but to formulate specific and The implementation approach must be considered early
appropriate corrective action. Importantly, while an in the procurement process, so that selected suppliers
overall service offering might be competitive, at the know what kind of deal they are getting into. This affects
granular level areas for improvement are almost always how the service provider implements the changes
found when the proverbial onion is peeled back. needed in order to achieve improvements mandated by
the analysis.
For Boeing, the investment a detailed analysis requires
has yielded significant dividends. As a defense In cases of most “utility” IT services, pricing tends to be
contractor, the company must demonstrate pricing paramount and customers are relatively unconcerned
transparency and assure government auditors that rates about the “how” of service delivery. So long as service
paid to service providers are market-competitive. level requirements are met, the provider has significant
Furthermore, a granular approach avoids the trap of leeway to manage the supply. In the majority of such
bundling, whereby highly efficient services subsidize low instances automatic, oftentimes retroactive price
performers. In cases where technology or process adjustments are an appropriate contractual mechanism
investments are needed to reduce risk, improve for implementing the benchmark results.
productivity, or meet service expectations a benchmark
While it’s all about price for utility services, not all deals
can oftentimes help justify any additional expenses.
fall into this category. In some cases the relationship
between customer and supplier is more transformative,
ANNUAL BUILD-DOWN perhaps even a true partnership. In such instances, risk
and market-competitive pricing must be balanced along
As an alternative to regular benchmarks, some
with other factors such as innovation and time-to-market
organizations opt to incorporate an annual mechanism
to ensure that the “shared destiny” of the two parties is
that automatically adjusts pricing to reflect ongoing
not adversely affected. A more complex benchmark
changes in the market, typically building a pre-defined
clause may be appropriate here.
year-over-year reduction percentage into the contract.
This model can be effective when services suppliers have
BENCHMARKS STILL COUNT ■ LAWRENCE KANE 3

5. BENCHMARKS AND MULTI-SOURCING organization being analyzed. Benchmarking consultants
must have superb communication skills and experience
While benchmarks may have grown out of traditional in dealing with customer employees whose jobs may be
single-source solutions, they are even more important in affected and supplier employees whose profitability
multi-sourced arrangements, whether multiple providers might be impacted as a consequence of the benchmark
play in any technology or service area or not. While a results.
multi-vendor approach is oftentimes ideal for leveraging
the unique capabilities of various providers, it also tends While the benchmarker can run the entire process,
to raise retained governance costs for contract Boeing has found that the best results are achieved when
management, integration, and oversight. In such a dedicated, autonomous benchmark “core team” leads
instances, benchmark analysis results can demonstrate the activity as a primary interface with the benchmarker,
not only competitiveness of individual suppliers, but also with customer and supplier service delivery experts
the effectiveness of the entire governance model gathering the data. Up front planning and
(assuming the customer’s retained organization is part of communication is essential.
the analysis).
Multi-sourcing governance assessment plays a key role in CONCLUSION
ensuring not only delivery of services within the context In a successful benchmark, both customer and supplier
of the in-scope agreement, but may also highlight agree that the results are accurate and fair, even if they
weaknesses across an organization’s many sourced do not like them. Customer and supplier executives,
relationships. Hand-off of Change, Incident and other IT management, and employees have a good understanding
Infrastructure Library (ITIL) processes are critical, often of the benchmarking project and process, which is
the cause of real or perceived weaknesses by the end- performed with transparency and proactive
customer in service delivery. Benchmarking can often communication throughout. Results are actionable and
highlight where the initial negotiation may have implemented as planned. In this fashion, the project
weakened critical process interaction. virtually always has a positive impact on the relationship
between customer and supplier.
HIRING AN EFFECTIVE BENCHMARKER Benchmarking has proven to be an effective
Benchmarking is clearly a good thing, but only if management tool for many global organizations, and
implemented “correctly.” Vital to implementation is a central to a culture of continuous improvement. Even as
benchmark service provider with a proven methodology, organizations that benchmark over the long term
strong process discipline, robust database of comparable experience some diminishing returns in terms of hard
peers and skilled consultants who can interpret the savings, the insights gained in terms of soft savings,
findings. An independent benchmarking consultant process improvement and supplier management remain
maintains objectivity and integrity of the results, assures significant.
the use of proven techniques and tools, and provides
access to industry data for comparable companies.
Transparency is paramount. While benchmarks may
commonly be performed without involvement of the
service provider, the process of collecting, validating, and
normalizing data should involve all parties. This
transparency facilitates a sense of fairness and
objectivity, and helps assure that accurate data was used
for the analysis and that all parties will agree with the
results (even if they do not like them).
Benchmarking is mostly a science, but partially an art,
particularly when normalizations or adjustments are
appropriate. A benchmark team must engender trust.
The right personnel will have knowledge, skills,
experience, and a proven ability to achieve actionable
results for organizations that are comparable to the
BENCHMARKS STILL COUNT ■ LAWRENCE KANE 4

6. LOOKING FOR A STRATEGIC PARTNER?
Lawrence Kane is Sr. Leader, IT Infrastructure Strategy, Sourcing, and Asset Management Execution at Boeing. Lawrence
helped develop and execute the IT infrastructure competitiveness strategy in 1999, of which benchmarking was a key
component. Having completed more than sixty studies using a variety of benchmarkers since the program’s inception, he
has acquired vital data that helped shape the company’s sourcing strategy, save hundreds of millions of dollars, and
improve quality and service delivery. He is currently responsible for IT infrastructure strategy, sourcing, and asset
management execution at Boeing.
John Lytle, a Director with ISG, contributed to this report.
For further information, please contact Alex Kozlov, Director of Marketing, Americas, at alex.kozlov@isg-one.com
or +1 617 558 3377
Information Services Group (ISG) (NASDAQ: III) is a leading technology insights, market intelligence and advisory services
company, serving more than 500 clients around the world to help them achieve operational excellence. ISG supports private and
public sector organizations to transform and optimize their operational environments through research, benchmarking, consulting
and managed services, with a focus on information technology, business process transformation, program management services
and enterprise resource planning. Clients look to ISG for unique insights and innovative solutions for leveraging technology, the
deepest data source in the industry, and more than five decades of experience of global leadership in information and advisory
services. Based in Stamford, Conn., the company has more than 800 employees and operates in 21 countries. For additional
information, visit www.isg-one.com.
011113
© Copyright 2013 Information Services Group – All Rights Reserved