Read this informative presentation with contributions from experts on the Future of Software Analysis and Measurement: Dan Galorath, President & CEO of Galorath Inc and Bill Curtis, SVP & Chief Scientist, CAST will have an engaging discussion moderated by David Herron, VP, Knowledge Solution Services, David Consulting Group. These industry veterans will how SAM tools coupled with estimate models can impact organizational performance through increased ROI, customer satisfaction and business value. To view the webinar, visit http://www.castsoftware.com/news-events/event/future-of-SAM?gad=ss

1. The Future of
Software Analysis & Measurement

2. The Future of
Software Analysis & Measurement

3. Daniel D. Galorath
 40+ years Software & Software
Management Experience
 Founder & CEO Galorath Incorporated,
SEER By Galorath
 1980 MBA Management,
California State Universities
 1984 Began working software
estimation, planning & control
 2009 Society of Cost Estimation and
Analysis (SCEA) Lifetime Achievement
award
 2001 ISPA Freiman Award,
lifetime achievement award
 2006 Book: Software Sizing, Estimation,
and Risk Management

4. SEER® Empowering Project Estimation, Planning & Control Since 1988
 SEER parametrically estimates project
cost, effort, duration, cost & risk
 Answer Fundamental Management
Questions
 How much effort is required
to complete an activity?
 How much calendar time is
needed to complete an activity?
 What is the total cost of an activity?
 How reliable will it be?
 What can we do better?
 What If?
www.galorath.com

5. David Herron
 Business Development Manager and VP of
Knowledge Solution Services for David
Consulting Group
 Consulting and coaching services for a variety
of IT organizations throughout the US and
Canada
 Acknowledged authority in the areas of
performance measurement, process
improvement and organizational change
management
 Noted author and lecturer; co-authored several
books on topics relating to IT performance
measurement and Function Point Analysis

6. David Consulting Group
 DCG is an international IT process improvement and measurement
company managing value-driven engagements with companies and
government agencies around the world
 Software Process Improvement
Utilizing CMMI, Six Sigma, Lean and Agile methods
 Software Sizing
Using IFPUG Function point Counting and alternative sizing techniques
 Software Measurement
Providing roadmap planning, estimation models, performance benchmarks
and outsourcing SLA support
 IT Performance Improvement
Improving IT operations through ITIL and IT Governance

7. Dr. Bill Curtis
 Senior Vice President and Chief Scientist
 Industry luminary responsible for influencing
CAST’s scientific and strategic direction
 Best known for leading development of the
Capability Maturity Model (CMM) which has become
the global standard for evaluating the capability of
software development organizations.
 Prior to joining CAST,
 Co-Founder of TeraQuest, the global leader in CMM-
based services,
 Directed the Software Process Program at the Software
Engineering Institute (SEI) at Carnegie Mellon University.

8. CAST Application Intelligence Platform
Most enterprises measure everything but the product delivered to the business
CAST Application Intelligence Platform (AIP) measures the product itself
Process Product
 Robustness
 Performance
Time & Quality  Security CAST AIP
 Planning Duration &Size  Changeability
 Estimation  Transferability
 Scheduling  Size
 Time Tracking
 Cost Tracking  Requirements
Function &
Effort & Budget Scope  Earned Value
 User Acceptance
 Usability

9. QUESTIONS
 What are the single most important business drivers that lead an organization
to embark on a SAM initiative?
 How does software analysis and measurement relate to business outcomes?
 What organizational behaviors are most favorable for a successful SAM
program?
 What is the relationship between SAM and IT governance?
 How can measurement and analysis interact with estimating to provide
improved visibility and information to make the best management decisions
regarding software?
 How can anyone provide a viable estimate software development, software
maintenance, or continuing innovation work and what use it is?
 How can we determine which software projects with have the highest ROI?

10. What are the single most important
business drivers that lead an organization
to embark on a SAM initiative?

11. Human Nature Optimism in Software
HBR Article explains this Phenomenon:
 Humans seem hardwired to be optimists
 We routinely exaggerate benefits and discount costs
Delusions of Success: How Optimism Undermines Executives' Decisions
(Source: HBR Articles | Dan Lovallo, Daniel Kahneman | Jul 01, 2003)
Solution: Temper with “outside view”
Software Analysis & Measurement can temper
Viable estimation can temper
Don’t remove optimism, but balance optimism & realism
11

12. IT Failures are Pervasive: And Even Successful Projects may not be
 Cutter Consortium Software Project Survey:
 62% overran original schedule by more than 50%
 64% more than 50% over budget
 70% had critical product quality defects after release
 Standish Group CHAOS Report
 46% challenged
 19% failed
 35% successful
~$875 billion spent on IT
~$300 billion spent on IT projects
~$57 billion wasted annually
Ever increasing “Technical Debt”
ROI of Applied Software Analysis & Measurement is HUGE

13. Manual Estimates
 Human reasons for error (Software Analysis & Measurement can
help)
 Desire for “credibility” motivates overestimate behavior (80%
probability?)
 So must spend all the time to be “reliable”
 Better approach force 50% probability & have “buffer” for overruns
 Technical pride causes underestimates
 Buy-in causes underestimates
13

14. Important Business Drivers for Structural Measurement
Poor software
1 from suppliers
Embarrassing
2 disasters
High cost of
3 maintenance

15. How does software analysis and
measurement relate to business outcomes?

16. Goal Question Metric Approach
Goal
Organizational Goal
Question
Development
Contractors
Organizations
Metric
 Combine goal-orientation bottoms up, decision-support & other
operational management techniques
 Going to weather.com and deciding to bring an umbrella is decision support

17. How Does Measurement Relate to Business Outcomes
Business Tactical Objectives Application Health
Value Maximize standards compliance Factors
Improve software readability TRANSFERABILITY
Minimize business Reduce vendor lock-in allows new teams to quickly
begin working with an application
risks Reduce cost of ownership
Cost
Reduce injected defects
Reduce application mistakes CHANGEABILITY
makes an application easier and
Minimize IT costs Reduce application rework
quicker to modify
Ease team handoffs
Minimize liquidated damages
Maximize business Maximize application availability ROBUSTNESS
improves application stability &
agility Minimize degraded service
reduces injecting new defects
Reduce learning curves
Risk
Reduce modification effort
Optimize work Accelerate new function delivery PERFORMANCE
productivity Reduces degraded response
Maximize application scalability times and increases scalability
Maximize speed of response
Maximize Maximize information retrieval
SECURITY
customer loyalty Minimize unwanted breaches
affects an application’s ability to
Maximize customer confidence prevent unauthorized intrusions
Maximize information protection

18. What organizational behaviors are most
favorable for a successful SAM program?

19. Key Organizational Behaviors
 Trust but verify
 Viable measurement and analysis
 Answering the right questions
 Having actions based on measurement & analysis
 Measure what can make a difference
 Measure to the audience needs (Goal / Question / Metrics)

20. Generating the Business Value Side of the Equation (Benefits)
 The business owns benefit calculations
 IT should participate
 Exception: projects solely improving internal IT
 Beware of subjectivity translating soft benefits
 Use probability and risk
Increased Revenue
Intangible Benefits Increased Profit
Reduction in cost
Internal benefits (people, processes,
cash out)

21. Conditions that Favor Structural Measurement
Low Fed up
maturity managers
Disciplined
Mature
processes
High Quantitative
maturity management

22. What is the relationship
between SAM and IT governance?

23. The Gap Between IT & Management Needs
Financial
Business/ IT
Organization Customer Vision &
Business Process Governance
Level Strategy
Innovation / Growth
Customer
Focus
CoBIT…
6σ Goals Goal Attainment
Questions Answers IT
GQM Metrics Measurement Governance
Data Collection
PSM
Software Development & Maintenance
(Source: Fraunhofer)

24. How Structural Measurement is Tied to IT Governance
The Evolution of Governance
Govern the Govern the Govern the
Department Project Product
Budget Schedule Structural
headcount defects quality

25. How can measurement and analysis interact
with estimating to provide improved visibility
and information to make the best management
decisions regarding software?

26. What a Parametric Model can Tell You
Firm Fixed
Price?
Feel lucky?
What is
likely to
happen
Understand the risk before you
commit! 26

27. SAM Feeds Estimation and Provides Insight & Choice
Size Effort
S
A SEER-SEM
M Software
O Estimation Schedule
u Complexity Model
t
p
u Risk
t Maintainability/
s
Defect Potential
Reliability
/Defects
People, Process,
Technology Maintenance Effort
Required
Constraints
 Cost, schedule, risk
 Should we update or
Historical redevelop?
Results
(SAM Outputs)  What is the risk of continuing
with software with increasing
“technical debt”?

28. Looking at Maintenance and Technical Debt

29. Software Estimation Basic Model & Associated Metrics
Effective Technology Cte
Development
Technology Legacy,
Maintenance Maintenance/
Specifics & Block Change
Constraints Development
Effort
K People Process and/or Process
Block Changes
As Redevelopment
Staffa Defects
Effective complexity & Count (Qi Qr)
D Constraints
Software Size
Size St Stakeholder Delivered
ReuseDIT Requirements Development (Effective Se
Software & Total St)
Size Se Process
(work units)
Start
Finish
Calendar Time
On-going Iterations of
Effort (ACWP or Spent)
Progress (BCWP or Earned Value)
Defects (Qi Qr )
Growth (Sg)

30. Fundamental Metrics
 Size  Effort
 AKA Volume, Mass  AKA Work, Labor
 Units: Source Lines of Code  Units: Staff Months, Staff Hours
(SLOC); Function Points (FP)
Use Cases  Cost
 New versus rework  AKA Budget, Money
 COTS & Packages  Units: $, other currencies
 Effective Technology  Staffing
 AKA Productivity Potential,  AKA Manpower Loading
Efficiency  Units: FTE People
 Units: none  Defects
 Time  AKA Reliability, Quality
 AKA Duration, Schedule  Units: Defect Count
 Units: Calendar Months,
Calendar Weeks

31. Other Key Metrics Help Track Project Performance
Track defect
discovery and
removal rates
against expected
rates
Heath and Status Indicator
shows status and trends from
the previous snapshot
Thresholds are user definable
Increased defect
reporting rate
shows a
worsening trend

32. When & Why to Collect Data
When to Collect Why People Don’t Want to
1. Up-front: when scoping new Provide Data
project data from completed  They could be proven wrong
projects
 It could be used against them
2. In-Process: During
development for management,  Data often doesn’t exist
to identify issues and progress  Even if processes dictate data
3. Post Mortem: Upon requirements
development completion to  If it exists it may not be clean
improve corporate history
 It may give away corporate
repository
productivity & bid strategy
4. In Service: During
maintenance to continue
learning & improving

33. How can anyone provide a viable estimate
software development, software maintenance, or
continuing innovation work and what use it is?

34. 10 Step System Estimation Process 2011
1. Establish
Estimate Scope
10. Track Project
Throughout
Development
2. Establish Technical 9. Document Estimates
Baseline, Ground and Lessons
Rules, Assumptions Learned
8. Generate a
Project Plan
4. Refine Technical
Baseline Into
Estimable Components 6. Validate Business
Case Costs &
Benefits (go / no
go)
4. Collect data /
estimation inputs 6. Quantify Risks
and Risk Analysis
5. Estimate Baseline Cost,
Schedule, Affordability Value

35. Estimation Organizational Maturity V1.7
Manual effort
Level 0 Informal or no
estimating
estimating
without a
process
Level 1 Direct Task
Estimation
Spreadsheets
Ad Hoc
Process
Simple model
Formal (Size *
Productivity) or Some
Level 2 Sizing (e.g.
function
Direct Task
Estimation informal SEER
Use
measurement
& analysis
Informal
Process
points)
Robust Formalized
Rigorous Parametric
Level 3 Formal
Sizing
Parametric
estimation
(SEER)
Estimate vs.
actual capture
Multiple
Estimate
Process
measurement
& analysis
planning &
Control
Risk
Management
Repeatable
process
Robust Parametric Process
Rigorous
Level 4 Formal sizing
Repeatable
process
parametric
estimating
(SEER)
measurement
& analysis
estimation
with tracking &
control
Risk
Management
improvement
via lessons
learned
Robust Parametric
Rigorous Continuous
Level 5 Formal sizing
Repeatable
process
parametric
estimating
(SEER)
measurement
& analysis
estimation
with tracking &
control
Risk
Management
process
improvement
Why should we care? Maturity is related to estimate viability…
With better estimation process, projects more likely to be successful in execution

36. ESTIMATION & PLANNING: An Estimate Defined
 An estimate is the most knowledgeable statement you can make at a
particular point in time regarding:
 Effort / Cost
 Schedule
 Staffing
 Risk
 Reliability
 Estimates more precise with progress
 A WELL FORMED ESTIMATE IS A DISTRIBUTION
36

37. Avoid Surprises with Estimation Process & Tools
 Challenged projects
 Would you still go forward if you knew
 Schedule would be significantly longer?
 Cost would be dramatically higher?
 Probably: but perhaps more insight could identify mitigation
 Plan functionality differently
 Certainly you could notify stakeholders of real costs
 Ensure staffing is appropriate for the constraints
 Failed Projects
 Would you start a project you knew was unaffordable? Or if schedule was
completely unrealistic?
 If knowing up-front could you do something about it?
 often better to kill project before it begins than waste resources & let the
organization down

38. How can we determine which software
projects with have the highest ROI?

39. Software Projects Must Return Business Value
“Economics is primarily a science of choice… software
economics should provide methods for analyzing the
choices software projects must make.” Leon Levy
“Base choices on those providing the maximum business
value to the organization” Eli Goldratt
Measurement and its uses such as estimating and defect
analysis help this science of choice
 Some say business value is not our problem
 While others generally need to perform benefit analysis
 We need to build systems that optimize the business
 Make IT part of the solution
If IT & measurement don’t generate sufficient profit,
money will go elsewhere

40. Software & IT Systems are about Business Value
Cost
Value

41. An ROI Analysis of an Upgrade: Software Analysis &
Measurement Provides Valve Cost
Can we do better?
Will stakeholders tolerate a loss for 3 years?
What is the risk?

42. Measurement Manifesto for Software & IT
Measurement Perspective Management & Stakeholder Perspective
 We measure to ultimately  Speak so I can understand
produce business value to the
 Give me actionable items
organization
 Don’t just give me problems…
 MEASURED Measurement
Give me solutions
should not ultimately be a cost
 Help me make the best decisions
 The analysis of measurements
so we can produce business
produces decisions that
value
produce business value
Measurement itself is not the answer….
Management decision making, better performance, quality
improvements and better serving stakeholders is

43. L a nmo ea o t A T
er r bu C S
w w c ss f aec m
w .a tot r.o
w
bo .a tot aec m
lgc ss f r.o
w
w w fc b o .o c so q a t
w . e o kc m/a tn u ly
a i
w w sd s aen t a tot ae
w . ie h r.e/ ss f r
l c w
w w t ie.o O Q a t
w . t r m/ n u ly
wt c i