The document discusses managing the evolution of software systems through three main points: processes, metrics, and people. It outlines key maintenance processes like incident handling and work requests. It also discusses the importance of standard processes for communication, quality improvement, and tool usage. High performing organizations are characterized as having clear processes linked to goals and measured performance. The document then covers important maintenance metrics around volume, quality, responsiveness, and efficiency. Finally, it emphasizes that processes and metrics must be properly supported by people.

1. Managing the Evolution of Software Systems
T he Old, the New and W hat’s
Consistently Tr ue:
Managing the Evolution
of Softwar e Systems
By Joe Hessmiller
CA1
I
Computer Aid, Inc. ®

2. Managing the Evolution of Software Systems
Why Focus on Maintenance?
Incident
Handling
End User
Call Support
Work
Requests
Admin.
& Other Costs

3. Managing the Evolution of Software Systems
Maintenance Is Where the
Money Is
• Maintenance and Support Makes Up
50-80% of Software Management
Budgets
• And, 100% of the Software Actually
Used By the Organization
Software
Budget
65%
Maintenance
and Support
Development
Software
Supporting
Business
100%

4. Managing the Evolution of Software Systems
The Old
Software Development and Support Has
Enormous Room for Improvement

5. Managing the Evolution of Software Systems
1969 NATO Report on Software
Engineering
• Documented problems in
– Requirements, Design, Coding
– Estimates, Monitoring
Progress, Communication
– Productivity (26:1), Reliability
(Bugs)
– Hardware Dependencies,
Reuse
– Maintenance Costs
• Sound familiar??

6. Managing the Evolution of Software Systems
A Generation After NATO Report
• 34% of Projects Completed On-Time, On-Budget
• 48% of Required Features Missing in
“Successfully Completed” Projects
• 43% Average Cost Overrun
• 82% Average Schedule Overrun
“So many software projects fail in some major way
that we have had to redefine success to keep
everyone from becoming despondent...”
Source: Tom DeMarco in the book, Controlling Software Projects
Source: Standish Group, Chaos Report, 2003

7. Managing the Evolution of Software Systems
Individual Performance Variation
• Performance Differences
– 10 to 1 Difference in Productivity with
Same Level of Experience
•
•
•
•
•
•
•
Sackman, Erikson and Grant, 1981
Curtis, 1981
Mills, 1983
Demarco and Lister, 1985
Curtis, et al, 1986
Card, 1987
Calett and McGarry, 1989

8. Managing the Evolution of Software Systems
Team Performance Variation
“Productivity of
the 90th percentile
teams is four
times higher
than that of the 15th
percentile teams”
Source: Barry Boehm. Software Engineering
Economics. Englewood Cliffs, NJ: Prentice
Hall

9. Managing the Evolution of Software Systems
What’s Possible?
• Software quality can improve at 15%
to as much as 40% PER YEAR
• Software development productivity
can improve at rates of 5% to 20%
PER YEAR
• Software maintenance
productivity can improve
at rates of 10% to 40% PER
YEAR
- Capers Jones, Calculating the Value of PI, 2000

10. Managing the Evolution of Software Systems
What’s Been Accomplished?
• HP
– 10X reduction in defect rate, average time to fix a defect was cut in
half in one business unit, reduced time to market by 5X over 5 year
period in another business unit
• IBM Toronto
– 10X reduction in delivered defect rates, productivity up by 240%,
rework reduced by 80%
• General Dynamics Decision Systems Division
– rework percent dropped from 23.2% to 6.8%; customer reported
defect rate dropped from 3.20 to 0.19 defects/KSLOC; productivity
rose 2.9X

11. Managing the Evolution of Software Systems
CAI Experience with Productivity
Improvement

12. Managing the Evolution of Software Systems
Productivity Increased in Wide
Variety of Scope-Headcount-Cost
Situations

13. Managing the Evolution of Software Systems
The New
Lessons from Modern Manufacturing for
Application Development and MAINTENANCE

14. Managing the Evolution of Software Systems
Why Manufacturing?
• Importance of Thinking “Outside the
Box”
• Applicability of Modern
Manufacturing Lessons to Software
Development and Maintenance
• Participate in the Continuous
Evolution of “Production” Ideas

15. Managing the Evolution of Software Systems
Manufacturing Productivity
Outpaces Services
Real Output per Full-Time Equivalent Worker, 1977-2001

16. Managing the Evolution of Software Systems
First: Think Out of the Box
The definition of insanity is doing the
same thing over and over and
expecting different results.
- Benjamin Franklin

17. Managing the Evolution of Software Systems
Thinking Outside of the Box
The Fremont Story
• 1982 - GM shutters Fremont assembly plant
– Worst quality and productivity in GM network
– Absenteeism of 20%
• 1984 - GM-Toyota JV (NUMMI)
– Hired 5000 former workers (including union leadership)
– Implements TPS (later called “Lean”)
• 1986 – Fremont Plant is:
– one of the most efficient plants in North America
– producing highest rated car in terms of quality in GM
– Absenteeism of 3%

18. Managing the Evolution of Software Systems
Evolution of Manufacturing
•19th Century – Craft Production
•20th Century – Mass Production
•21st Century – Lean Production
Mass
Production
Craft
Production
Low Cost Per Unit,
Rigid
Flexible,
High Cost Per Unit
Lean
Production
Flexible,
Low Cost Per Unit

19. Managing the Evolution of Software Systems
Software “Shortcut” to Lean
•19th Century – Craft Production
•20th Century – Mass Production
•21st Century – Lean Production
Craft
Production
Flexible,
High Cost Per Unit
Lessons
Lean
Production
Flexible,
Low Cost Per Unit
Mass
Production
Rigid,
Low Cost Per Unit

20. Managing the Evolution of Software Systems
What IS Lean Thinking?
The Five Lean Enterprise
Principles
– Value Focus on What Adds Value for the
Customer
– Value Stream Understand How Value Is
Created
– Flow Maximize Speed to Value
– Pull Work on Just-In-Time Basis
– Perfection Continuously Improve
Performance
Drive Out Waste

21. Managing the Evolution of Software Systems
Lean Makes Powerful Impact in
Administrative Areas
“ We were doubly shocked when we
realized they [144 Japanese firms
benchmarked] were also four times
more productive per person [than peers
in North American companies] in the
administrative area.”
- George Koenigsaecker, CME, 2003

22. Managing the Evolution of Software Systems
Why Lean Works
In reality, companies had come to
recognize that combinations of
efficiency and flexibility allowed
them to meet various and changing
customer needs more effectively than
simply maximizing one dimension.”
- Michael Cusumano, Japan's Software Factories, 1991

23. Managing the Evolution of Software Systems
What’s Consistently True
Process, Metrics and People

24. Managing the Evolution of Software Systems
High Performance Organization
Characteristics
“High performance IT organizations across
all industries share common process
characteristics:
– Processes are clear;
– The link between process and performance
goals is well understood; and
– The performance of processes is well
measured and properly supported.”
Page and Pearson, Transforming the IT Workforce,
Outlook 2004, Number 2

25. Managing the Evolution of Software Systems
Process
Process, Metrics and People

26. Managing the Evolution of Software Systems
Process Should Support Different
Motivation and Management Systems
• Douglas McGregor (1960)
– Theory X
• Average person dislikes work and will avoid it if possible
• People must be forced to work toward organizational goals
• People prefer to be directed and avoid responsibility
– Theory Y
• Effort in work is natural
• People will apply self-control in pursuit of organizational objectives
• People seek responsibility
• William Ouchi (1981)
– Theory Z
• Implicit, Informal Control
• Explicit, Formalized Measures

27. Managing the Evolution of Software Systems
Why Is Standard Process
Important?
Enables …
• Common Language to Communicate
Activity, Status, Issues

28. Managing the Evolution of Software Systems
Why Is Standard Process
Important?
Enables …
• Common Language
• Quality Plans and Improvement Strategies
– Discrete Components for Performance
Specification and Impact Assessment
– Meaningful Process Metrics

29. Managing the Evolution of Software Systems
Why Is Standard Process
Important?
Enables …
• Common Language
• Quality Plans and Improvement Strategies
• Tool Usage Benefit Maximization
– Deployment of Tool Across Organization
– Feedback/Tuning Experience

30. Managing the Evolution of Software Systems
Why Is Standard Process
Important?
Enables …
• Common Language
• Quality Plans and Improvement Strategies
• Tool Usage Benefit Maximization
• Successful Partnering
– Common Expectations
– Clear Interaction Process

31. Managing the Evolution of Software Systems
High Performance Organization
Characteristics
“High performance IT organizations across
all industries share common process
characteristics:
– Processes are clear;
– The link between process and performance
goals is well understood; and
– The performance of processes is well
measured and properly supported.”
Page and Pearson, Transforming the IT Workforce,
Outlook 2004, Number 2

32. Managing the Evolution of Software Systems
What Are the Maintenance
Processes?
Incident
Handling
End User
Call Support
Work
Requests
Admin.
& Other Costs

33. Managing the Evolution of Software Systems
What Are the Maintenance
Processes?
ISO 1764, IEEE 1219

34. Managing the Evolution of Software Systems
Key Process Areas Mapped
Production Incident Process Flow
Work Request (WR) Process Flow
1
52
4
1
WR Received
Time Sheet
Tracking
Preliminary
Estimate
53
a
15
Print Time
Sheets
Prioritize
54
55
Time Entry
20
Print Weekly
Time Audit
36
Development
2
40
Implementation
Documentation
48 Customer Sat.
4
3
5
WR Log-In
Tracking
6
Acknowledge
WR Receipt
7
8
9
Identify
Resources
Create WR
Estimate Task
Assign Est.
Resources
Define
Requirements
16
17
18
User Group
Meeting
Update WR
Priority
Print WR
Status Report
19 Distribute WR
32
Determine
Solutions
Develop
Project Plan
22
Work Request
Scope Change
Update WR
Scope Change
25
26
28
Create WR
Tasks/Estimate
29
30
Notify Memo
14
Estimate
Approval
39
Production
Notification
Application
Bulletin
Application
Documentation
41
42
43
User
W/R Received
Approval
Production
W/R Request
MoveReceived
Move
Source/Object
44
31
WR Design &
Specification
Specification
QC
DB Change
Request
Code/Unit
Testing
Program QC
(System Test)
Update WR
QC/CMPL. Dates
User Acceptance
Test (Date)
34
35
Work Request
Issue Form
45
a
46
Update WR
Issue Tracking
47
2
5
On-Line Program
Feedback Form
13
50 User Returns Cust.
7
Incident Issue
Form
8
14
Sat. Feedback Form
Update Incident
Issue Tracking
23
15
9 Incident Log-In
25
10
Assign
Resources
11
17
6
24
Job Monitor
Production Incident Process
Flow Legend
Legend
27
28
Incident Issue
Form
33
7
Enhancement
Manual Process Function
30
Update Incident
Issue Tracking
29
10
15
No Action
Required
35
31
16
36
87 Time-Entry 7
37
6
Support Issue
Form
Escalation
Process
Prioritize
Incident
16
Corrective
Action
32
Program Chng.
WR Steps (28-45)
17
Update Support
Issue Tracking
11
12
13
Escalation
Process
18
User Approval
(Close)
19
38
P04
39
Close Incident
Update WR
Close. Date
Tracking
User Approval
(Close)
18
Close
Incident
20
21
Work Request Process Flow
Legend
Main Process
Function
Manual Process
Function
Automated Process
Function
8
Incident
Notification
Fill Out Call
Support Ticket
Identify
Resources
Log-In Ticket
Tracking
Support Process Flow
Legend
Assign
Resources
Main Process Function
Update Incident
Tracking
14
Close WR
(Checklist Rev.)
Classify
Support
9
Function
Corrective
Action
Update Incident
Tracking
6
(Work Request)
34
Assign
Resources
Incident
Investigation
Night Support
Analyst
Identify Outside
Resources
Consulting
Automated Process
Program Chng.
WR Steps 28-45
Notify
Resources
Incident Log-In
Tracking
Main Process Function
Escalation
Process
12
Monitor
Production
Identify
Resources
5
3
26
Identify
Resources
Tracking
Log In
Feedback
After Hours
22
Capture Incident
Data
Notify
Resources
4
Business Hours
Batch Program
49 Print Customer Sat.
51
Call Support
Capture Incident
Data
Classify Incident
Program
Turnover
Resource
13 WR Estimate &
P02
Final Estimate
24 Assign Develop
Analysis
12
38
Estimate
27
11 Function -Point
37
Identify
Resources
23 Update WR Final
Report
33
10
21
21
Feedback Form
6
2
Non-ABEND
Incident
1
ABEND/
Job Abort
3
Job Abort
Support Process Flow
20
Incident
Notification
Manual Process Function
Consulting/
Investigation
Automated Process
Function
User Approval
(Close)
Update Ticket
Tracking
Close Call
Support Ticket
Figure X.X
P03
19
Time Entry

35. Managing the Evolution of Software Systems
Work Request Process Mapped
Work Request (WR) Process Flow
52
4
1
WR Received
2
3
5
WR Log-In
Tracking
6
Acknowledge
WR Receipt
7
8
9
Time Sheet
Tracking
Preliminary
Estimate
Identify
Resources
Create WR
Estimate Task
Assign Est.
Resources
Define
Requirements
53
a
15
16
17
18
19
Print Time
Sheets
Prioritize
User Group
Meeting
Update WR
Priority
Print WR
Status Report
Distribute WR
Report
32
Determine
Solutions
33
10
Develop
Project Plan
Work Request
Scope Change
Update WR
Scope Change
54
20
21
22
26
Analysis
28
Create WR
Tasks/Estimate
29
13 WR Estimate &
30
12
Notify Memo
P02
14
Estimate
Approval
37
38
Final Estimate
39
Final
Estimate
25
31
Specification
QC
DB Change
Request
Code/Unit
Testing
Program QC
(System Test)
Update WR
QC/CMPL. Dates
User Acceptance
Test (Date)
Production
Notification
Application
Bulletin
Application
Documentation
48
Implementation
41
42
43
User
W/R Received
Approval
Production
W/R Request
MoveReceived
Move
Source/Object
44
Assign Develop
Resource
WR Design &
Specification
40
Documentation
Identify
Resources
23 Update WR
24
Print Weekly
Time Audit
36
Development
27
11 Function -Point
55
Time Entry
34
35
Work Request
Issue Form
45
a
46
Update WR
Issue Tracking
47
Program
Turnover
Monitor
Production
Close WR
(Checklist Rev.)
Update WR
Close. Date
Work Request Process Flow
Legend
Main Process
Function
Manual Process
Function
Automated Process
Function
Customer Sat.
Feedback Form
49 Print Customer Sat.
Feedback Form
50
51
User Returns Cust.
Sat. Feedback Form
Log In
Feedback

36. Managing the Evolution of Software Systems
Process Step Map

37. Managing the Evolution of Software Systems
Metrics
Process, Metrics and People

38. Managing the Evolution of Software Systems
What Maintenance Management
Measures
• Volume of Service – Measured by the number of
units of services and the number of services
delivered per unit of time.
• Quality – Measured by defect rates, standards
compliance, technical quality, services
availability, and service satisfaction.
• Responsiveness – Measured by time to
acknowledge, time to implement, and services
backlog.
• Efficiency – Measured by cost per unit of
service delivered

39. Managing the Evolution of Software Systems
High Performance Organization
Characteristics
“High performance IT organizations across
all industries share common process
characteristics:
– Processes are clear;
– The link between process and performance
goals is well understood; and
– The performance of processes is well
measured and properly supported.”
Page and Pearson, Transforming the IT Workforce,
Outlook 2004, Number 2

40. Managing the Evolution of Software Systems
Quality Management Performance Over Time

41. Managing the Evolution of Software Systems
Customer Satisfaction Management

42. Managing the Evolution of Software Systems
Quality Management Rework Management

43. Managing the Evolution of Software Systems
Quality Management Defect Management
Identify Incident Causes Data Errors: 34 Hours
Drill Down into Root Causes –
Repeated Data Errors within the Order
Processing System

44. Managing the Evolution of Software Systems
Productivity Management Efficiency Management

45. Managing the Evolution of Software Systems
Productivity Management Effectiveness Management

46. Managing the Evolution of Software Systems
Productivity Objective: More Time Focused
on Value Adding Activities
Analysis ofof Previous MonthHours
Analysis Total Year-to-Date
Hours
Analysis of Total Year-to-Date Hours
Analysis of Previous Month
Hours
Admin, 47
Admin, 57
Work
Requests, 381
Work
Requests, 173
Call Support,
102
Call Support,
113
Incidents, 104
Admin
Call Support
Incidents
Incidents, 229
Work Requests
Admin
Call Support
Incidents
Work Requests

47. Managing the Evolution of Software Systems
People
Process, Metrics and People

48. Managing the Evolution of Software Systems
Motivation is KEY to Productivity
Improvement
• Most productivity studies have found
that motivation has a stronger
influence on productivity than any
other factor.
- Barry Boehm, 1981

49. Managing the Evolution of Software Systems
Achievement and Recognition
Recognition of Achievement is VERY
Important to Developers
Accurate, Objective Metrics Meet People’s Needs.

50. Managing the Evolution of Software Systems
Motivating Developers
•
•
•
•
•
The Work Itself
Achievement
Advancement
Compensation
Recognition
- Cougar, Zawacki, Opperman, MIS Quarterly

51. Managing the Evolution of Software Systems
What Motivates Developers?

52. Managing the Evolution of Software Systems
Getting People to Use the
System
• Threaten Them
• Incent Them
– Make Job
Easier Through
Automation
– Ensure
Recognition
Through
Metrics
How to Herd Cats

53. Managing the Evolution of Software Systems
High Performance Organization
Characteristics
“High performance IT organizations across
all industries share common process
characteristics:
– Processes are clear;
– The link between process and performance
goals is well understood; and
– The performance of processes is well
measured and properly supported.”
Page and Pearson, Transforming the IT Workforce,
Outlook 2004, Number 2

54. Managing the Evolution of Software Systems
Assignment Management

55. Managing the Evolution of Software Systems
Task Management

56. Managing the Evolution of Software Systems
Estimation Management

57. Managing the Evolution of Software Systems
Issue Management

58. Managing the Evolution of Software Systems
Time Management

59. Managing the Evolution of Software Systems
Change (Scope) Management

60. Managing the Evolution of Software Systems
Quality Management – Step
Level

61. Managing the Evolution of Software Systems
How to Implement
Setting Achievable Goals, Objectively
Measuring Performance

62. Managing the Evolution of Software Systems
Improved Capability Level
How They Did It:
Three-Phase Implementation Model
OPTIMIZATION
Streamline & Craft
Work Processes
CONTROL
VISIBILITY
Classify and Capture
Work, Metrics,
Resources and Time
Implement Workflow
Processes

63. Managing the Evolution of Software Systems
Visibility
• To “Shine a Light” on What is Being
Done By Whom and How
• To Capture Performance Metrics;
Baseline, Operational and Systemic

64. Managing the Evolution of Software Systems
Control
• To Produce Meaningful Process and
Product Management Metrics
• To Ensure That Performance Meets
Customer Expectations (and React
Quickly and Effectively To Variation )
• To Provide a Stable Platform for Process
Improvement Experiments and
Improvement Deployment

65. Managing the Evolution of Software Systems
Optimization
• To Identify Efficiency Improvement
Opportunities
• To Identify Effectiveness
Improvement Opportunities

66. Managing the Evolution of Software Systems
What We Need: Maintenance and
the Three Principles
IN
CS
WR
Visibility
What is
causing error?
What problems
are being
experienced?
What are
people working
on?
Control
Are errors
being closed
within SLAs?
Are problems
being resolved
within SLAs?
Are WR being
completed
within
SLAs/Estimates
?
Optimization
What errors
are recurring?
Are the
corrective
actions
working?
What calls are
recurring? Are
corrective
actions
working?
What’s causing
us to miss
estimates? Are
corrective
actions
working?

67. Managing the Evolution of Software Systems
Next Steps
A Best Practice Assessment is Your First
Step Toward Realizing the Benefits of Best
Practices Organization…
– Visibility
– Control
– Optimization

68. Managing the Evolution of Software Systems
Best Practice Assessment
Process
The Best Practice Assessment Helps You
Answer …
Where Are We?
1. Identify Current State
Is Current State = Best Practice?
2. Analyze Gap
Where Do We Want To Be?
3. Develop Future State Vision
How Should We Get There?
4. Prepare Recommendation

69. Managing the Evolution of Software Systems
CAI Profile
World Leader in IT Process and Productivity

70. Managing the Evolution of Software Systems
CAI Company Profile
• Founded 1980
• 2000+ Employees
• Offices in US, Europe, Canada, and Philippines
• 70% of Revenue from Managed Services
Partnerships
• Fortune-1000 Customer Base
• World-Class Solution Centers
• Long-term Strategic Partnership Focused

71. Managing the Evolution of Software Systems

72. Managing the Evolution of Software Systems
CAI Product Of ferings
• Application Support & Maintenance
– Managed Maintenance
• Application Development
– Construction Management
• Desktop Services & Help Desk
– Help Desk, Workstation Support, IMAC
• Staff Augmentation - T&M Consulting

73. Managing the Evolution of Software Systems
The CAI Value Proposition
CAI enables its clients to:
• Track and measure the right activities in the IT function
• Compare IT performance vs. benchmark data
• Optimize IT performance, using data to make decisions
• Achieve double-digit % gains in IT productivity
• Establish objective measures of IT worker performance
This is made possible through the use of:
• A unique metrics-based methodology – “Managed Maintenance”
• The proprietary TRACER® software tool and data repository

74. Managing the Evolution of Software Systems
Closing Thoughts
Embrace Change

75. Managing the Evolution of Software Systems
Thoughts on Change
It is not necessary to change. Survival is not mandatory.
W. Edwards Deming
Beware becoming change-averse change agents.
Barry Boehm, SEI Symposium 2000
In theory, there is no difference between theory and practice.
In practice there is.
Lawrence Peter Berra

76. Managing the Evolution of Software Systems
Excellent IT Reference Site
www.itmpi.org