One of the strengths contributing to the diffusion and adoption during last years of Maturity Models (MMs) such as CMMI and ISO 15504 (aka SPICE) is the evolutionary path towards a continuous improvement they provide, evolving the initial Crosby’s idea. Differently from Performance Management models (PMMs) such as the Balanced Scorecard (BSC), Malcolm Baldridge, EFQM Excellence Model or the JUSE Deming Prize, MMs seems to do not stress in their appraisal criteria the way resources are renewed, redistributing obtained ‘results’ towards the ‘enablers’. Looking at this question from an ‘ecological’ viewpoint, where the current environmental situation urgently asks to adopt renewable resources taking care from an holistic view of the state of the planet, the paper will discuss this issue translating it to the organizational management, proposing possible improvements to the process assessment model (PAM) generic structure of a MM, with the objective to provide a more confident picture of an organization from an appraisal, not overrated, as nowadays it can happen.

1. An Ecological View on Process Improvement
Some Thoughts on Improving Process Appraisals
LuigiBuglione
École de Technologie Supérieure (ETS)
University of Québec
4th World Congress for Software Quality – Bethesda, Maryland, USA – September 2008

2. Outline
• Aim: to propose tips for a more refined and granular
schema for process appraisals when using maturity
models (MM) such as CMMI (Capability Maturity Model
Integration) or ISO/IEC 15504 (aka SPICE) for a process
improvement initiative
• Scope: Organizational and Process Maturity
• Main Issues: Maturity Models; Process Appraisals;
Appraisal Criteria; Renewal; Benchmarking; Process
Improvement ROI
An Ecological View [2] of [26]

3. Agenda
• Introduction
• Maturity Models (MMs)
 a definition
 a short history
 appraisal criteria in PAM
• Performance Management Models and MM
 some well-known models
 main commonalities & differences
• An ‘ecological’ proposal for improving PAM
 Some improvement proposals
• Conclusions & Prospects
An Ecological View [3] of [26]

4. Introduction
Global Warming trend (1884-2006)
Source: NASA
An Ecological View [4] of [26]

5. Introduction
Q: which corrective actions to take for global
warming?
A1: start adopting renewable sources of
energy (e.g. photovoltaic panels, wind
machines, etc…)
A2: modifying the way people to act, working on their
awareness (‘I care’)
An Ecological View [5] of [26]

6. Introduction
Q: and what about the way Maturity Models (MM)
such as CMMI or ISO 15504 evaluate the
management of resources and the allocated
budgets?
Q: Does their PAM (Process Assessment Model)
takes care of such ‘renewal’ or not?
renewal
Q: Are sufficiently comprehensive the current
appraisal criteria (e.g. CMMI GP2.3) in order to
capture the renewal issue?
An Ecological View [6] of [26]

7. Maturity Models (MM)
A definition (tag cloud style)
A Maturity Model is:
“a model the essential elements of
that contains
effective processes for one or more disciplines
and describes an evolutionary improvement path
from ad-hoc, immature processes to disciplined,
with improved quality
mature processes
and effectiveness
(ISO/IEC FCD 24765, Systems & Software Engineering Vocabulary)
An Ecological View [7] of [26]

8. Maturity Models (MM)
A short history  an evolutionary view
An Ecological View [8] of [26]

9. Maturity Models (MM)
Appraisal Criteria in PAM
 Each MM is two-fold:
 PRM (Process Reference Model)  a model comprising
definitions of processes in a life cycle described in terms of process
purpose and outcomes, together with an architecture describing the
relationships between the processes [ISO/IEC 15504-1:2004]
 PAM (Process Assessment Model)  a model describing life
cycle processes, based on good engineering and process
management principles and suitable for the purpose of assessing
process capability [ISO/IEC 15504-1:2004]
 PAM defines the rating process:
 Rating Scale  4-point scale (N/P/L/F)
• CMMI uses quartiles
• ISO 15504 (0-15; 16-50; 51-85; 86-100%) intervals
 Rating Value  the value assigned to:
• a goal/PA; the capability level of a PA; the ML of an organizational
unit
An Ecological View [9] of [26]

10. Maturity Models (MM)
Appraisal Criteria in PAM
CMMI v1.2 ISO/IEC 15504-5
ML/CL GP - Generic Practice PA - Process Attribute
GP 5.2 – Correct Root Causes of Problems PA5.2 - Continuous improvement attribute
5 GP 5.1 – Ensure Continuous Process Improvements PA5.1 - Process change attribute
GP 4.2 – Stabilize Subprocess Performance PA4.2 - Process control attribute
4 GP 4.1 – Establish Quantitatively Objectives for the Process PA4.1 - Measurement attribute
GP 3.2 – Collect Improvement Information PA3.2 - Process resource attribute
3 GP 3.1 – Establish a Defined Process PA3.1 - Process definition attribute
GP 2.10 – Review Status with H/L Management
GP 2.9 – Objectively Evacuate Adherence
GP 2.8 – Monitor and Control the Process
GP 2.7 – Identify and Involve Relevant Stakeholders PAs are subdivided into 36 MP
GP 2.6 – Manage Configuration (Management Practices)
GP 2.5 – Train People
GP 2.4 – Assign Responsibilities
GP 2.3 – Provide Resources
GP 2.2 – Plan the Process PA2.2 - Work product management attribute
2 GP 2.1 – Establish an Organizational policy PA2.1 - Performance management attribute
1 GP1.1 – Perform Base Practices PA1.1 - Process performance attribute
An Ecological View [10] of
[26]

11. Maturity Models (MM)
Appraisal Criteria in PAM: a photo example
• Q: which level of control (granularity) has your PAM on
processes?
(a) (b)
or
An Ecological View [11] of
[26]

12. Maturity Models (MM)
Appraisal Criteria in PAM: a map example
• Q: which level of control (granularity) has your PAM on
processes?
(a) (b)
or
An Ecological View [12] of
[26]

13. Maturity Models (MM)
Appraisal Criteria in PAM
• Q: are the criteria in your PAM placed at the right place?
CMMI v1.2
ML/CL GP - Generic Practice
 An example: Root-Cause Analysis
GP 5.2 – Correct Root Causes of Problems
(RCA)
5 GP 5.1 – Ensure Continuous Process Improvements
 CAR is placed at ML5 (PRM-level)
GP 4.2 – Stabilize Subprocess Performance  GP5.2 is the latest PAM criteria at Level
4 GP 4.1 – Establish Quantitatively Objectives for the 5 (PAM-level)
Process
GP 3.2 – Collect Improvement Information  …but…
3 GP 3.1 – Establish a Defined Process  DAR is a PA at ML3 (but it logically
GP 2.10 – Review Status with H/L Management needs before CAR)
GP 2.9 – Objectively Evacuate Adherence  Indicative equivalence between an ISO
GP 2.8 – Monitor and Control the Process 9001:2000 certified organization and
GP 2.7 – Identify and Involve Relevant Stakeholders CMMI maturity level between ML2-3 
demonstration of ability in RCA is
GP 2.6 – Manage Configuration
mandatory for achieving the ISO
GP 2.5 – Train People certification
GP 2.4 – Assign Responsibilities
GP 2.3 – Provide Resources
 …therefore…
 CAR should be move at ML2 (at least at
An Ecological View
2
GP 2.2 – Plan the Process
GP 2.1 – Establish an Organizational policy
ML3, jointly with DAR) [13] of
[26]
1 GP1.1 – Perform Base Practices

14. Performance Management Models
Some well-known models
An Ecological View [14] of
[26]

15. Performance Management Models
Main Commonalities & Differences with MMs
 Rating Schema: both share an established standard rating schema
in their PAM, allowing comparisons across time and organizations,
based on their PRM
 Entities: Performance Management Models (PMM) rate a criterion to
be mapped against 1+ Quality Management System (QMS)
process(es), MM rate a generic process to be mapped against 1+ QMS
processes
 Rating Criteria: PMM have a % weighted distribution across the
evaluation criteria, while MM appraise single processes by CL
(continuous representation) or groups of processes (staged) by ML
using a NPLF scale
 Intangibles: PMM focus more on intangibles than MM (e.g. EFQM
criterion 3a; 4b) and the way the contribute to the final result (in a BSC
logic), while MM has a more limited scope
 Scope: PMM have a wider scope than MM (‘Enablers’ such as
‘leadership’ or ‘strategy’ are not comparable with CMMI’s GP2.7 and or
GP2.10)
An Ecological View [15] of
[26]

16. Performance Management Models
Main Commonalities & Differences with MMs
• Lessons Learned:
 the way resources are managed in PMM  not simply ‘acquired’
but also renewed, in a continuous lifecycle schema (limited
renewed
resources to be managed, and also this capability should be part of
the evaluation)
 the more attention paid in PMM to intangibles in the evaluation of
results observing ‘objective evidences’
• Potential Side Effects (not applying such lessons):
 process overrating  the less granular the PAM criteria, the less
accurate the representation from appraisal/assessment results of
the organization unit appraised/assessed…
…with particular attention for lower levels (ML2  ML3): this
ability could make a sensible difference between a ‘managed’ and a
‘defined’ organization (tactic  strategy)
An Ecological View [16] of
[26]

17. Improving PAM
An ‘ecological’ proposal
• Q: how to translate this idea in a PAM?
• An evaluation of an organization should analyze results from
improvement actions at the light of its
 MVV (Mission-Vision-Values)
 Value chain, from a causal viewpoint
• The lessons learned from the ‘global warming’ example plus the
way PMM manage resources could be used also for improving
PAM in MMs
 e.g. the ‘People/Employee’ fifth perspective in some ICT BSC
• Some preliminary thoughts:
 investments on people (reskilling, coaching, continuous training, …) 
more than CMMI OT (Organizational Training) or ISO 15504 RIN.2/RIN.3
(Resource Infrastructure)
 relate measurement of tangible + intangible assets  from ‘productivity’ to
‘performance’ before assessing ML4
An Ecological View [17] of
[26]

18. Improving PAM
An ‘ecological’ proposal
Proposal #1: Refine the current PAM criterion on Resources
adding the ‘Renewal’ concept CMMI v1.2
ML/C GP - Generic Practice
Comment: this is the more conservative view.
L
GP 5.2 – Correct Root Causes of Problems
Reinforcing the current criterion would not modify 5 GP 5.1 – Ensure Continuous Process Improvements
a consolidated PAM architecture for benchmarks GP 4.2 – Stabilize Subprocess Performance
and comparisons. 4 GP 4.1 – Establish Quantitatively Objectives for the
Process
GP 3.2 – Collect Improvement Information
 the new way to observe how resources are 3 GP 3.1 – Establish a Defined Process
managed and re-used will lead to more granular GP 2.10 – Review Status with H/L Management
results and to more focused improved actions GP 2.9 – Objectively Evacuate Adherence
GP 2.8 – Monitor and Control the Process

GP 2.7 – Identify and Involve Relevant Stakeholders
the only counterpoint for this option is about GP 2.6 – Manage Configuration
the need to mark – from a benchmarking GP 2.5 – Train People
viewpoint – the appraisals done before and after GP 2.4 – Assign Responsibilities
such modification referring to such criterion and GP 2.3 – Provide Resources
its effects on the overall appraisal results,
whatever the representation type chosen.
GP 2.2 – Plan the Process
2 GP 2.1 – Establish an Organizational policy
1 GP1.1 – Perform Base Practices
An Ecological View [18] of
[26]

19. Improving PAM
An ‘ecological’ proposal
Proposal #2: Introduce a new criterion (‘Renewal’)
Renewal
Comment: this hypothesis will slightly CMMI v1.2
modify the PAM architecture. More ML/C
L
GP - Generic Practice
possibilities about the positioning of such GP 5.3 – Renew Existing Resources
new GP/PA: GP 5.2 – Correct Root Causes of Problems
5 GP 5.1 – Ensure Continuous Process Improvements
 (b1) at CL5, as well as the current GP 4.2 – Stabilize Subprocess Performance
4 GP 4.1 – Establish Quantitatively Objectives for the
GP/PA5.2 positioning for root-cause Process
GP 3.3 – Renew Existing Resources
analysis within the ‘optimizing’ level; GP 3.2 – Collect Improvement Information
 (b2) at CL3, as a further filter from
3 GP 3.1 – Establish a Defined Process
GP 2.11 - Renew Existing Resources
the ‘defined’ and ‘quantitatively GP 2.10 – Review Status with H/L Management
managed’ levels; GP 2.9 – Objectively Evacuate Adherence
GP 2.8 – Monitor and Control the Process
 (b3) at CL2, as a further filter from GP 2.7 – Identify and Involve Relevant Stakeholders
the ‘managed’ and the ‘defined’ levels. GP 2.6 – Manage Configuration
GP 2.5 – Train People
GP 2.4 – Assign Responsibilities
GP 2.3 – Provide Resources
An Ecological View 2
GP 2.2 – Plan the Process
[2] of [xx]
GP 2.1 – Establish an Organizational policy
1 GP1.1 – Perform Base Practices

20. Improving PAM
An ‘ecological’ proposal
Proposal #2a: Introduce a new criterion (‘Renewal’) at CL5
Renewal
CMMI v1.2
Comment: The lower the level it will be
ML/C
L
GP - Generic Practice
introduced, the larger number of GP 5.3 – Renew Existing Resources
organizational units that will consider it during GP 5.2 – Correct Root Causes of Problems
appraisals, producing a more accurate 5 GP 5.1 – Ensure Continuous Process Improvements
picture of the ‘real’ organizational maturity GP 4.2 – Stabilize Subprocess Performance
4 GP 4.1 – Establish Quantitatively Objectives for the
Process
GP 3.2 – Collect Improvement Information
 this positioning will be agreed by a more 3 GP 3.1 – Establish a Defined Process
conservative audience, being included at the GP 2.10 – Review Status with H/L Management
‘optimizing’ level GP 2.9 – Objectively Evacuate Adherence
GP 2.8 – Monitor and Control the Process
 no matter the representation type adopted
GP 2.7 – Identify and Involve Relevant Stakeholders
GP 2.6 – Manage Configuration
(staged/continuous), few organizations would GP 2.5 – Train People
apply for it. GP 2.4 – Assign Responsibilities
GP 2.3 – Provide Resources
GP 2.2 – Plan the Process
2 GP 2.1 – Establish an Organizational policy
An Ecological View 1
[2] of [xx]
GP1.1 – Perform Base Practices

21. Improving PAM
An ‘ecological’ proposal
Proposal #2b: Introduce a new criterion (‘Renewal’) at CL3
Renewal
Comment: it represents a further filter ML/C
CMMI v1.2
GP - Generic Practice
from the ‘defined’ and ‘quantitatively L
managed’ levels. GP 5.2 – Correct Root Causes of Problems
5 GP 5.1 – Ensure Continuous Process Improvements
 here the rationale is that till CL2 a process GP 4.2 – Stabilize Subprocess Performance
consumes resources, but it is not planned 4 GP 4.1 – Establish Quantitatively Objectives for the
Process
their regeneration. From CL3 on the GP 3.3 – Renew Existing Resources
organization should start to adopt a tactical GP 3.2 – Collect Improvement Information
approach to ‘renewal’. When arriving at CL5 3 GP 3.1 – Establish a Defined Process
this ‘way of acting’ should be fully absorbed in GP 2.10 – Review Status with H/L Management
the organizational DNA. its introduction from GP 2.9 – Objectively Evacuate Adherence
Level 3 would explain better the GP 2.8 – Monitor and Control the Process
organizational vision at the mid-long term GP 2.7 – Identify and Involve Relevant Stakeholders
GP 2.6 – Manage Configuration
GP 2.5 – Train People
 placed here, it could be not applied by GP 2.4 – Assign Responsibilities
SMEs with a target on CL2 GP 2.3 – Provide Resources
GP 2.2 – Plan the Process
2 GP 2.1 – Establish an Organizational policy
An Ecological View 1
[2] of [xx]
GP1.1 – Perform Base Practices

22. Improving PAM
An ‘ecological’ proposal
Proposal #2c: Introduce a new criterion (‘Renewal’) at CL2
Renewal
CMMI v1.2
Comment: it represents a further filter
ML/C
L
GP - Generic Practice
from the ‘managed’ and the ‘defined’ levels. GP 5.2 – Correct Root Causes of Problems
5 GP 5.1 – Ensure Continuous Process Improvements
GP 4.2 – Stabilize Subprocess Performance
 separately from GP2.3, the distinct rating 4 GP 4.1 – Establish Quantitatively Objectives for the
Process
for this issue would make aware people about GP 3.2 – Collect Improvement Information
specific corrective/improvement actions. 3 GP 3.1 – Establish a Defined Process
GP 2.11 - Renew Existing Resources
GP 2.10 – Review Status with H/L Management
 the risk, introducing such criterion yet at GP 2.9 – Objectively Evacuate Adherence
Level 2, could be to ask too much evidences
GP 2.8 – Monitor and Control the Process
here, with the consequence of an
GP 2.7 – Identify and Involve Relevant Stakeholders
underestimation, while such level should
GP 2.6 – Manage Configuration
demonstrate the ‘repeatability’ at the project
GP 2.5 – Train People
level .
GP 2.4 – Assign Responsibilities
GP 2.3 – Provide Resources
GP 2.2 – Plan the Process
2 GP 2.1 – Establish an Organizational policy
An Ecological View 1
[2] of [xx]
GP1.1 – Perform Base Practices

23. Improving PAM
Verify & Validate
• V&V:
 compare results from an appraisal performed both with the
traditional PAM rules and with the modified rules (according to one
of the variants above presented)
 determine with ‘picture’ seems to be more adherent to the
observed reality
An Ecological View [23] of
[26]

24. Conclusions & Prospects
 Maturity Models adoption is growing more and more in the ICT
companies from the mid ’90s on at the worldwide level
 Analyzing current PAMs, some thoughts and lessons learned from
Ecology could be adapted & adopted, in particular the attention to
renewal of (whatever) resources, in particular people/employees
 The more granular the appraisal criteria, the more valid the
representational level of the observed entity (organizational unit by its
processes)
 Rethinking the criteria used for rating processes can help in a better
understanding of where (and how) creating new value for the
organization, focusing on the ability to ‘renew’ resources used in a
process and the capability to manage them
 Few hypothesis about where placing this new criteria were presented,
with main pros & cons for its eventual adoption in MMs.
An Ecological View [24] of
[26]

25. Thanks for Your attention!
An Ecological View [25] of
[26]

26. An Ecological View on Process Improvement
Some Thoughts on Improving Process Appraisals
LuigiBuglione
École de Technologie Supérieure (ETS)
University of Québec
luigi.buglione@computer.org