1
Collaborative Consulting:
Software Performance
Engineering Services

2
Contents and Overview
Value PropositionAbout Collaborative Services & approach
Performance
Engineering’s high impact
to Client brand and IT Risk
Mitigation
Collaborative has a rich
history of success in
delivering client outcomes
Current and emerging
client services

3
Contents and Overview
Value PropositionAbout Collaborative Services & approach
Performance
Engineering’s high impact
to Client brand and IT Risk
Mitigation
Collaborative has a rich
history of success in
delivering client outcomes
Current and emerging
client services

4
Why Collaborative? Our People, Value and Delivery
Our people
Our people are our strongest asset; we
not only have the most senior
individuals with deep domain expertise,
but our IT technical support team is
second to none in the industry.
Our strongest asset
At the heart of the organization is a
team of committed individuals who are
passionate about solving complex
issues, and dedicated to providing the
highest quality of service possible.
Heart of the organization
Our culture and work process is focused
on pragmatism, problem-solving and
results-oriented action.
Our culture and work process
We understand the “big picture”, as
well as the macro business trends, but
we really thrive on the specific
mechanics and process of the business.
Our understanding
We bring a comprehensive
methodology and approach that allow
us to be confident in our work products,
be rigorous in our approach, and speed
“time-to-value” in all of our projects.
Our comprehensive
methodology
We leverage our proven project
accelerator tools and methodologies
that reduce effort, improve quality and
shorten timelines.
Our proven project accelerator
tools
Our value Our delivery

5
Our solutions approach
Our flexible delivery model allows us to work with clients in the way that makes the most sense
for the situation: as a trusted advisor, in a blended team scenario, or as a dedicated project team.
Flexibility
Enterprise
Architecture
Business Technology
Technology
strategy
Solution
delivery
Business process
optimization
Solution
delivery
A keen
understanding
of enterprise
business issues
and operations,
with
practitioners
across multiple
industries.
A discipline
aimed at
building the
right solution for
each client,
based on their
specific needs
and business
objectives.
Aligning business and technology initiatives enables our clients
to achieve results that deliver business value.

6
Services & Delivery Model
Life Sciences
• Pharmaceutical
• Medical Device
Understand and
envision
Architect and
design
Develop and deploy
Support and
maintain
Consulting Services Domestic Solution Center Services
Financial Services
• Capital Markets
• Insurance
Information
Management
• Master Data
Management
• Data Quality
• Data Integration
• Business Intelligence
• Data Governance
Application
Architecture and
Development
• Architecture
Strategy
• Application
Development
• Systems Integration
• Mobile Enablement
• Application Support
Software Performance
and Quality
• PE Advisory Services
• Application Rescue
• Performance Testing
& Tuning
• Quality Assurance
Strategy
• Software Quality
Testing

7
Contents and Overview
Value PropositionAbout Collaborative Services & approach
Performance
Engineering’s high impact
to Client brand and IT Risk
Mitigation
Collaborative has a rich
history of success in
delivering client outcomes
Current and emerging
client services

8
Performance Engineering Value
PE is the set of roles, skills, activities, practices, tools, and
deliverables to be applied at each phase of the SDLC to ensure that a
solution will be designed, implemented, and operationally supported
to meet the non-functional requirements defined for the solution.
What is PE?
• User experience: PE task and activities in the SDLC promote business and IT alignment
through reliable, scalable, and performing systems. Performance is designed in early.
• Increased revenue: Ensure an organizations’ systems and applications are able to manage
complexity, identify risk, and enable seamless and rapid business growth for increased
revenue.
• Effective use and return on system resources. Capacity planning trending and forecasting
can help manage IT system investments, eliminate budget surprises.
• Manage risk of technology adoption for business value. Performance evaluation and
Proof-of-concept testing can help identify business risk.
Value delivered to the business

9
Benefits and Case for Action
We deliver high value, impactful services, resulting in business stability and
client and customer satisfaction.
• Manage risk and complexity.
• Ensure service level agreement
criteria is met and exceeded.
• Ensure application scales with
business growth.
• Identify the system’s performance
limits.
• Ensure the physical infrastructure
supports the business plan.
• There is a need for application
service assurance and reliability.
• High availability and performance
from enterprise applications are
needed to achieve return on
investment.
• There are multiple configuration
options for distributed applications.
• Multiple integration points in a typical
system installation can lead to
performance and scalability issues.
Key Benefits Case for Action

10
The SPE Body of Knowledge – Key differentiator
Knowledgeareas
Enterprise Software Performance Engineering
Planning, coordination, information sharing, and control
SDLC &
architecture
(SA)
Application
performance
management
(APM)
Problem
detection
and
resolution
(PDR)
Underlying competencies and roles for delivering
Techniques supporting the knowledge areas
Performance
validation and testing
(PVT)
Capacity planning
(CP)
Collaborative is the Performance Engineering Expert

11
PE: Methodology
PE encompasses the set of roles, skills, activities, practices, tools, and
deliverables applied at every phase of the SDLC which ensures that a solution
will be designed, implemented, and operationally supported to meet the
non-functional requirements defined for the solution.
Step-by-step framework for
better software engineering
Collaborative’s PE Methodology is a step-by-step framework designed to
ensure an organizations’ systems and applications are able to enhance
competitive advantage, manage complexity, identify risk, and enable
seamless and rapid business growth.
Focused on business objectives
In short, our methodology helps companies determine if their technology
can accommodate its business objectives.
Based on unique methodology
Our PE Methodology is unique in that it is applicable throughout a system’s
entire life cycle. It also:
• Describes the detailed approach to designing, building, executing and
interpreting performance benchmarks
• Shows how to analyze performance of a system in production that falls
short of service level objectives

12
Contents and Overview
Value PropositionAbout Collaborative Services & approach
Performance
Engineering’s high impact
to Client brand and IT Risk
Mitigation
Collaborative has a rich
history of success in
delivering client outcomes
Current and emerging
client services

13
Contents and Overview
Value PropositionAbout Collaborative Services & approach
Performance
Engineering’s high impact
to Client brand and IT Risk
Mitigation
Collaborative has a rich
history of success in
delivering client outcomes
Current and emerging
client services

14
Performance Engineering Services
Software
Performance
Engineering
Collaborative’s
performance
engineering services
drive improvements
in information
technology and
ensure scalability
and stability for
existing and planned
systems, including
improving overall
quality, reliability,
capacity, security,
and performance.
Advisory Services
Is my current team delivering the right services?
Organizational and operational planning, strategy and design
services for transforming internal capabilities. Establishing PE
Center of Excellence (COE)
Application and
readiness
assessment
Is my application ready for the new volumes?
or is the application response time getting worse over time.
The goal is to technically assess the application to understand
the risks to performance, scalability and the transaction
response times.
Production
Performance Rescue
A proven approach to production performance and stability
issues, reviewing the end to end transaction. Minimizing
disruption and quickly restoring functionality and capability.
Application & Service
Performance
Management
Are my key business transactions being monitored, do I know
of slowdowns before my customers?
Optimizing the performance of specific applications and
services within a system. Define and implement an APM
strategy. Identify key business transactions for monitoring.
Measurement, and trending.
Technology
optimization and
tuning
Is my technical architecture delivering the best end user
response time and transaction throughput?
Review technical components, includes hands on testing and
measurement.
Benchmarks and
Performance testing
Application performance and scalability test design and
execution. Performance test lab definition and testing tool
selection.
We use local and remote team for test execution.

15
1. Advisory Services
• Review the current state of the team; review services provided, interaction
model with development and operations teams, and work products
• Review the services provided to the organization
• Determine the value provided
• Understand what the organization needs from the PE team
• Review the Application Selection Criteria for Performance Engineering and
performance testing
• Engagement Model with the development teams, the operations teams, the
PMO process and the governance process.
Team and Organizational Assessment
• Services roadmap
• Value proposition for the team
• Organization structure, roles and responsibilities
Outcome

16
1. Advisory Services: Organizational Alignment Framework for PE
Communication &
Education
Programs
Communication &
Education
Programs
Communication &
Education
Programs
Communication &
Education
Programs
Competencies
Provide PE processes, skills, and tools,
to excel in the new PE environment.
Resources
Align resource allocation
changes with updates to role
definition and responsibilities.
Assess the alignment of
ongoing initiatives with the
business performance vision.
Business Vision
Manage underlying policies and
conditions that influence
behavior.
Culture Management
Ensure that the plan is successful by
collaborating with key sponsors and leaders to
champion the initiative.
Sponsorship and Change Leadership
Assess the change to existing operations
environment and business areas with regard to
workflow interactions, tools, resource
allocation, responsibilities, and competencies.
Change Impact Assessment

17
1. Advisory Services Scope
Engineering Group Quality Assurance/
Performance Engineering Group
Operations
Platform/
Environment
Validation
Performance
Benchmarking
Performance
Regression
Performance
Integration
Production
Performance
Monitoring
Configuration/
Customization
Design
Implementation Application Testing
Production
Support
Performance Engineering Advisory Scope
Define
service level
objectives
Development Process

18
2. Application Readiness Assessment
Phase 1
Inspection of existing
application and
technical architecture.
Significant
risk
Phase 2
Risk evaluation. Code
inspection and
performance testing.
Assessment report, recommendations,
observations, and next steps.
Call to action:
• Current application is slowing down, user response time is degrading.
• Anticipating significant change in business volume. Will the current system scale?
• Moving to a new technology platform or acquiring a new application. Will it support my business
volumes?
Yes
No

19
2. Application Readiness Assessment
Methodology is iterative and tactical
Review infrastructure &
architecture
Define business activity
profiles & service levels
• Identify risk areas
• Review configuration
settings, topology, & sizing
• Define points of
measurement
• Types & numbers of users
• Business activities &
frequencies
Iterate testing & tuning
• Test data
generation
• Create test scripts
• User & transaction
profiles
• Infrastructure
configuration
Design & build tests
Developing a clear picture of system capabilities helps companies form a strategy to enable
seamless growth.
• Verify accurate inventories of applications and systems, and capabilities and limitations.
• Determine whether certain applications need to be replaced, retired or upgraded.
• Assess and improve performance engineering process and execution

20
3. Production Performance Rescue
Minimize disruption and quickly
restore functionality and capability.
• Accelerated analysis and test design, including system architecture review
• Accelerated iteration of test execution and application and configuration
changes
• Accelerated implementation of performance improvements.
• Mitigation/remediation of issue(s)
• Recommendation and planning of tactical short-term improvements and
strategic longer-term enhancements

21
4. Application & Service Performance Management
Align business and IT through proactive management of service levels
across IT and multiple lines of business.
Performance
management data
repository
Performance repository
administration:
• Create forecasts for
new applications
• Modify existing
forecasts
Performance data sources by:
Tier, Application, Server
Platform Application Tier
Performance dashboard views: Canned Reports:
J F M A M J J F M A M J J F M A M J
Legend: Performance Details Captured Performance Management Data Published
Combine monitoring and management tools with proven Performance Engineering
experience and methodologies to link people, information and decision-making
processes throughout the organization.

22
4. APM: Watch Every Transaction for Every User
WEB
MQ/ESB
DCOM
CORBA
SERVER
APP SVR
PROXY
LDAP
Web
Services
Datacenter

23
5. Technology Optimization and Tuning
• Identify business transactions and
supporting system components
• Evaluate each tier in the technical
architecture
• Leverage end to end transaction
monitoring
• Use deep dive tool for code analysis
and database analysis
Building a better user experience;
and/or reduce resource consumption
• Improved and consistent user experience
• Improved scalability (process more
transactions with the same system)
• Improved stability
• Transfer for knowledge to clients’
technical team
Outcome

24
6. Benchmarks and Performance Testing
What is changing? Where does it need to be validated? What (all) is being
measured?
The project: Why are we testing and what are the success criteria?
Prerequisites:
Business goals and non-functional requirements are defined
Workload and scenarios defined
Environment defined (where are we testing)
Application in a stable state
Configuration management and release management ready
Core performance team identified
Supporting team committed

25
6. Benchmarks and Performance Testing
• Define a custom benchmark for clients, for a
specific workload and technical architecture
• Application Baseline
• Stress and breakpoint
• Duration
• Peak load
• Component(s)
• Failover and Failback (baseline or with load)
• Antagonistic (degraded service times)
• Performance testing and code changes (iteration)
Performance Testing Categories

26
6. Performance test delivery – remote model
Distributed test execution team
Test execution options
• Remote team uses VPM to access client facility and execute tests
• Remote team use Cloud services to delivery load to client application
VPN
Cloud
Client facility
Collaborative’s remote
testing facility
Our performance
engineer is onsite
Test execution
team

27
• CASE
• STUDY Healthcare Prescription Benefits Manager (PBM)
Approach
Objective
•Major business impact due to production disruptions within customer’s PBM System. This resulted in major downtime of the
system.
•No Performance Testing (PT) environment being used. Limited PT execution. Lack of disciple around PT and lack of coverage.
No Performance Engineering occurring!
•Client looking to reduce risk and ensure that there will be no disruption in the future, and that all components of PBM System
will scale and perform for all future releases.
Results
•In an effort to reduce risk and ensure a higher level
of performance testing process, we were initially
engaged to assess the PT and PE maturity across
process, organization, roles and tools to identify and
recommend areas where tactical changes can be
applied to implement PE across the platform.
•Following the assessment phase, presented roadmap
including staffing and cost models to executive
leadership.
•Planned for partial FTEs and On Demand resources
with a model to move towards more FTEs over time.
•Focus has been on the entire Performance
Engineering process across the SDLC, from design
review, test execution, results review and analysis, and
troubleshooting.
•Client has improved the ability to execute disciplined
performance tests and to increase the number of
meaningful tests. Client has been able to execute more
tests for each release.
•By improving the test preparation process and test review
process, Client is able to reduce the number of retests due
to improper configurations and settings.
•Within first 6 months, Client has identified multiple
significant issues, prior to Production deployment, resulting
in immediate ROI, covering the cost of the group for the
first two years.
•Visibility into the performance of the application and
infrastructure has been vastly increased by improving the
monitoring process and increasing the number and type of
metrics being collected.
•Client is better positioned to manage risk and complexity
across their PBM System.

28
• CASE
• STUDY Financial Services Pension Provider
Approach
Objective
Client’s Online Group sought to proactively improve the performance of its systems and demonstrate
previously achieved improvements. Client wants guarantees that the applications will perform
without disruption, and given the frequency of enhancements and modifications being implemented
for the platform, the organization wants to become more proactive and better understand the effect
each enhancement has on the stability of the system before being deployed into production.
Results
•Phase 1: Assessment effort only. Began the
strategic process of developing the PE process
definition and PE roadmap, and PE process
implementation. The assessment phase is a
review of the key applications to assess the
technical challenges of performance
engineering each system. Within Phase 1 this
included:
•Client secure account web application
•Client Public site.
•Phase 2 began the tactical implementation of
the roadmap, based on business priority.
We delivered an assessment including findings and
recommendations, and developed a Roadmap
including several key initiatives. Notably this included
the following work streams:
•Baseline Client secure account web
application and the Client Public site
•PE test environment analysis
•Client secure account web application and
Public Site Production Performance Tuning
•Organization Alignment
•Non-production environment performance
monitoring
•Performance test data management
•Application assessment – Administrative web
application

29
• CASE
• STUDY Retail
Approach
Objective
• “Next generation” pharmacy system to replace legacy application is a highly complex system consisting of multiple
technologies and infrastructure components (2.6M lines of code; 25,000 function points).
• Two PT environments leveraged to accelerate performance testing. Additional complexities and risks are also being identified
(e.g., system stability, level of code completeness) while executing performance testing. Concerns raised over the viability of
aggressive PT timeframe.
• The company looking to reduce risk and ensure that the new solution will scale and perform for pilot and planned rollout to
chain.
Results
•In an effort to reduce risk and ensure a higher level of
performance testing process, Collaborative was initially engaged
to assess the PT plan, process, organization, roles and tools to
identify and recommend areas where tactical changes can be
applied to improve the overall PT process.
•Following the assessment phase, Collaborative was engaged to
manage and execute the large-scale performance testing.
Current team includes a blend of resources from Collaborative,
this company’s off-shore development partner based in India,
and various on-shore support staff. Also working with key tech
vendors (Oracle, IBM, EMC, etc).
•With aggressive program timelines, the team is running multiple
shifts in a 5x24 mode.
•Focus has been on the entire testing process, from design review,
test execution, results review and analysis, and troubleshooting.
•Collaborative technical architects are also working with the
infrastructure team and the off-shore application development
team to provide recommendations and resolve performance
problems with the application and in the environment.
•Collaborative has improved the ability to execute
performance tests and to increase the number of
meaningful tests. By streamlining the testing process,
Collaborative has been able to execute more tests for
each shift during a 24 hour period.
•By improving the test preparation process and test review
process, we were able to reduce the number of retests
due to improper configurations and settings.
•Feedback provided to the test design process has enabled
the test execution team to combine test cases into one
test case. This also allowed the team to be more efficient
with test execution.
•Visibility into the performance of the application and
infrastructure has been vastly increased by improving the
monitoring process and increasing the number and type
of metrics being collected across all the tiers and
components.
•The client is better positioned to manage their risk and
complexity

30
Thank You