6 sigma LTE release management process improvement

LTE Release Management Process
Application of PMGB tool sets for improvement
PMGB Workbook V34_engl
Application of PMGB tool sets for improvement
06.07.2013Process name : LTE System Release Management 1
6 Sigma Process Management : Jeffrey Gardner

Description of Release Management
System
Acceptance
Quality
MS
Supplier (LTE)
shipment
Integration
and Test
Pilot RolloutI/P
Quality
MS
Quality
MS
Quality
MS
System Development Life cycle
New Release
(N+1)
Start Stop
Preamble
Release Management is a sub process of the Software Delivery Life-cycle (SDLC)
LTE (Long Term Evolution) is a Global standard of 4th Generation Network deployed by Mobile
Operators to provide communications and broadband services.
A centralised Skill Centre owns the SDLC and Release Management approval through a ‘Test-
Release Management process
PMGB Workbook V34_engl 06.07.2013Process name : LTE System Release Management 2
A centralised Skill Centre owns the SDLC and Release Management approval through a ‘Test-
it-once’ approach.
Release Management process is sub-optimal with No. of defects above target level.
Objective is to modify the Process to achieve Operational & Service excellence.
The target outcome is better quality for internal customers and end users/consumers.
The optimised Release Management process is repeatable.

Release Management model – what is the target
performance?
Total Number of defects identified in Lab & Field
No. of
Defects
RFA RFP RRO SAS
Time
Joint System
Test
Pilot Rollout
Improve Test Improve Improve Quality
Target outcome :deal with the root cause and reduce impact of
managing the effects
Reduction in effort expended to identify defects during lab testing and remedial actions in the field.
Measurable improvement in defects detected in Lab compared to those in the field.
RFA
MS
RFP
MS
RRO
MS
SAS
MS
Improve Test
Quality
Improve
Quality
Improve Quality

SIPOC: Release Management
Do we have a common understanding of how the process works today?
Process Name: LTE Release Management
Supplier Input Process Output Customer
START
Supplier of LTE
System/s
System Release
Patch
Execute Patch
Release Testing
Patch under Pilot
Patch passes
Pilot
Patch Rollout
START
Complete Rollout
System Release
Acceptance
Recommendation
to National
Companies
Mobile Network
Operators/multiple
markets
STOP

CTQ / CTB Matrix
What is critical for the customer’s satisfaction?
Critical to Customer (CTQ)
Voice of the Customer Central Topic CTQ / Requirements
I want the minimum number of
SW faults in the Network
Quality/Operational
Each SW release shall be
thoroughly tested prior to
release to Field
Critical to Business (CTB)
SW faults in the Network
release to Field
Faults in the network should be
within acceptable levels
Efficiency
Defect statistics (KPI’s) are
within Target
Minimum delivery time for SW
patch
Quality/Time
Minimum delivery time for SW
patch
Faults are increasing
accumulatively over time
Quality
Problem Reports are as few as
possible
Voice of the Business Central Topic CTB / Requirements
Voice of the Business Central Topic CTB / Requirements
The total effort is increasing over
the lifecycle
Cost efficiency
Effort shall be minimised with no
more than 4 FTE’s allocated at
one time.
The defects may impact
customer experience
CEM/Revenue
Minimise the impact to end
customers

CTQs / CTBs
Output Variable Matrix
Through which output variables would CTQs and CTBs be best described?
CTQ
Each SW release shall be thoroughly
tested prior to release to Pilot
0 3 9 0 0
Fault statistics (KPI’s) are within Target 0 9 9 3 0
Minimum delivery time for SW patch 0 0 3 9 0
Problem Reports are as few as possible 0 3 9 0 0
CTB
CTB
Effort shall be minimised with no more
than 4 FTE’s allocated at one time
(applicable to RN Business Unit)
9 0 0 3 9
Minimise the impact to end customers 3 3 9 0 0
Correspondence 9 strong, 3 medium, 1 low, 0 none

Data Collection Plan
How do we get the data in order to measure the project performance?
Measurement Type of
measureme
nt
(Output/
Process/
Type of data
(continuous/
discrete)
Operational
Definition
Responsible
for data
collection
Method of data
collection
Data /time /
frequency of
measure-
ment
Data source /
place of
measurement
Target
value
(Output/
Process/
Input)
ment
No. Priority
1, 2 PR’s in
Pilot
Output Continuous Fault
removal
efficiency
PR ratio
Lab/Field
Release
Manager
Trouble
Ticket
System
(TIS)
Date/time
stamped
Release
Manager/K
PI
calculation
3:1 Ratio
(Lab to
Field)
No. Priority
1,2 PR’s in
Rollout
Ouput Continuous Quality
Risk
exposure
PR ratio
Lab/Field
Release
Manager
Trouble
Ticket
System
(TIS)
Date/time
stamped
Release
Manager/K
PI
calculation
9:1 Ratio
(Lab to
Field)
Lab/Field
Patch
availability
for Field
Output Continuous Supplier
;First-
in/First-out
Test
Leader
Observed Date/time
stamped
Test Leader </=5 days
Contain
effort to
minimum
Output Continuous FTE run
rate
Release
Manager
Reporting of
actual effort
Monthly ITG/Remaid Max 4 x
FTE

Graphic display of the Output Measurements
How well are we fulfilling customer requirements today?
Diagram
Operational definition of the measurement
No. of Priority PR’s in Pilot : Relationship between defects found in the Lab and defects found in the
field Pilot of the new release. Formulaic based on number of faults generated in the 2 phases.
ResultDiagram
Bar Graph shows the Target results
for Testing efficiencies over the last 4
System Releases.
In all cases the target was exceeded
although the internal customer
experience was unsatisfactory
(internal effort, disruption and
unplanned events).
The customer experience (VOC)
Result
100
50
The customer experience (VOC)
requires reduction in Operational
impact.
Efficiency KPI defined as a measure of
the effectiveness of test assignment:-
Efficiency KPI = No. of faults identified in test phase A
Total No. of faults identified during testing
Efficiency
Release A
Target 75%
Target range 0 – 100 %
80.95
0
96.3
Efficiency
Relese B
Efficiency
Release C
Efficiency
Release D
83.3392.86

Diagram
No. of priority PR’s in Rollout : Relationship between Defects found in the Lab and Defects found in the early
Operational lifecycle of the new Release. Formulaic based on number of faults generated in the 2 phases.
ResultDiagram Result
75
Bar Graph shows the Target results for
Testing efficiencies over last 4 releases.
In all cases the target was not achieved and
the internal customer experience was
unsatisfactory (internal effort, disruption and
unplanned events).
The customer experience (VOC) requires
reduction in Operational impact.
In target achievement the quality is a step
25
50
Quality Risk Exposure
Release A
Target 10%
Target range 5 – 25 %
35.71
0
30.2352.6321.6
Release B
Release C
Release D
In target achievement the quality is a step
improvement to the first Pilot Phase.
Total defects reduce with increase in quality.
Quality risk exposure KPI defined as
adherence to quality targets and value add:-
QRI KPI= No. of unidentified faults during testing
Total No. of faults identified over all testing

Diagram
Effort : Radio Network BU - Adherence to Planning of effort - relationship between planned and actual effort.
Calculation is proportion of planned vs actual effort (Effort = actual/planned)
1
2
Bar Graph shows the Target
achievement for project planning over
the last 4 System Releases.
In 1 out of 4 projects the target was not
achieved due to overrun.
The delay resulted in increased staffing
costs - OPEX estimation.
Total FTE allocation shall be maximum
of 4 FTE’s (planned & actual).
Effort
Release A
Target 1
Target range 0,5 – 1,5
0.98
0
1.34
Effort
Release B
Effort
Release C
Effort
Release D
0.850.85
of 4 FTE’s (planned & actual).
Cost efficiencies are upheld.
Effort KPI defined as ratio of planned vs
actual:-
Effort KPI= Actual Test Effort (person days)
Planned Test Effort (person days)

Diagram
Patch availability : FIFO model (First-in, first-out), time period (days) between receiving patch from
supplier to availability for Pilot/Rollout. Calculation is time based and variable in units of days.
5
10
Bar Graph shows the average
turnaround (Test process cycle) time
for Patch releases (Release D).
Operates on FIFO basis with Lab Test
execution (FTE’s) as sole constraint.
Rapid turnaround time increases fault
likelihood.
Better to define SLA as 5 day
turnaround and offer <5d delivery on a
Yr 2012 Release D
JAN
Target 5d
Target range 2d –5d
0
turnaround and offer <5d delivery on a
per case basis.
Cost efficiencies and Quality are
upheld.
FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC
6.7 2.7 1.7 2.7 2 2 10 3 4 4 3.5 5

Process Function Diagram (‘Swim Lane’)
What does the current process flow look like?
Division Process stepsStart Stop
LTE System
System release
patch available
New patch
required
New patch
required
RN Business
Unit
Operations
(Central)
LTE System
Supplier:
Optimization Confirm
Pilot
Pass
y
N
patch available
VE 1d
required
NVA <40d
required
NVA <40d
Execute Patch
Release
Testing
VA 11d
Patch under
Pilot
VA 10
d
1
2 Patch
Rollout
VA 7d
NOK
3
Confirm as
Rollout gating
NVA 2d
4
Patch Rollout
completion
VE 2d
5
Optimization
Functional
Team
VA Value Adding
VE Value Enabling
NVA Non Value Adding
Confirm
Rollout OK
NVA 0.5d
MNO Market
Units
OK
NOK
System Release
Acceptance
NVA 2d

Value Stream Map
How does material and information flow within the process?
Supplier CustomerSupplier Release ManagerTest Lead
1 FTE – 6 days
effort
1 FTE – 8 days
effort
Delivery to Test Lab
Lab Install
Define Test Template
1 FTE – 1 days effort
Defect
3 FTE - 5 days effort
1 FTE – 0.5 days effort
2.5 FTE – 1 day
System Test Assess Pass/Fail Roll out
1 FTE –
0.5 days effort
Test Report Pilot Readiness
1-7 days (Tues)
3 FTE - 5 days effort
Major Release
5 FTE – 5 days
effort
1 days – 1 FTE’s
Pilot
1 FTE – 7 days
effort
1 FTE – 75 days
effort
2.5 FTE – 1 day
effort

Results of the process analysis
What are the critical factors in the process flow – where have you identified potential problems?
Number Process Step Problem
1
Execute Patch Release
Testing
The Quality of the SW received from Supplier is not ‘error free‘.
Supplier provides Change note information to enable captive Testing to
be scoped within known constraints of environment and manpower.
2
Patch under Pilot The Pilot activity offers the best opportunity to establish defects to
Operations and/or end customers. Depending on Fault (PR) priority the
resulting leadtime to provide a resolution is between 1-40 days.
3
Patch passes Pilot As a quality gate (MS) the Pilot network experience provides data to
support the Ready for Rollout decision.
It does not full fill the following :-
a) Guarantee of a Rollout free from defects
b) A time extension of the Pilot phase
4
Patch Rollout During the rollout phase new defects may occur which often results in a
request for new SW patch/es. This has 2 primary effects :-
a) As SW footprint expands so do the related defects with broader
impact
4 impact
b) If rate of patches through the supply channel exceeds the rollout plan
(patch deploy rate), the network suffers a dip in SW performance.
5
Patch Rollout complete Ideal outcome is one final patch satisfying customer quality expectation.
In practice emerging defects neccessitate further release management
(extension of project into Operations ‘maintenance mode‘) with
extended delays and effort for MNO Business Unit.

Process RACI Matrix
Who is responsible for what in the process?
Process name / description
Process Steps Divison / Area
Input Output CommentsDescription of the
Operations
Optimization
Input Output Comments
Nr.
Description of the
process Step
Supplier
RN
Business
Unit
Operations
Central
Optimization
MNO
Markets
1
Execute Patch Release
Testing
C
R/
A
I I
Release
patch
2 Patch under Pilot A R R I
3 Patch passes Pilot
R/
A
C C I
4 Patch Rollout A R C I
Recommend
Market MNO have
sole authority for
R = Responsible - Person who is competent fof executing the task.
A = Accountable – Person who has the responsability over the task.
C = Consulted – Person who must be consulted about a decision.
I = Informed – Person who must be notified of a decision.
5 Patch Rollout complete A C R R SAS to MNO
Markets
sole authority for
SAS milestone
decision

Failure Mode and Effect Analysis (FMEA)
Where in the process flow can defects occur?
Procedure:Procedure:
1. Determine process steps
2. Determine which defects may occur
3. Determine what the consequences of the defects are
4. Determine the magnitude of the consequences of the defects (Intensity from 1-10)
5. Determine what the possible causes for the defect are
6. Determine the likelihood of occurrence of the identified causes (Occurrence from 1-10)
7. Determine which existing controls are in place
8. Determine the likelihood of detection of the defect causes (Detection from 1-10)
9. Calculate the Risk Priority Number (I x O x D)
9. Calculate the Risk Priority Number (I x O x D)
10. Define and take appropriate measures
11. Calculate/Estimate new Risk Priority Number

Failure Mode and Effect Analysis (FMEA)
“Where can defects occur in the process flow?
Process
step
Potential
defect
possibility
Potential
effect
Intensity
Defect cause
Occurrence
Existing
controls
Detection
RPN
Actions
Intensity
Occurrence
Detection
RPN
Execute
Patch
Release
Testing
Defect/s not
identified in
Lab Testing
Defect
slips to
Pilot.
Operability
and/or end
customer
impact
5 Patch release
too rapid. Lab
not fully
representative of
Field Pilot
(limited
investment
policy).
8 Y: appropriate
allocation of
Resource and
expertise
4 160 Detection; patch
content based decision
that allows appropriate
time for Patch testing.
Drive ‘Release when
ready’ of patch.
5 8 3 120
Patch
under
Pilot
Defect not
identified in
Field Pilot
Defect
slips to
Pilot.
Operability
and/or end
5 Unstable Pilot
network. Patch
related event
data lost in high
number of
8 Y :
Implemented
Operational
Network
surveillance
4 160 Detection; statistically
rule out ‘interferers’ by
implementing network
freeze of Pilot area.
5 8 2* 80
and/or end
customer
impact
number of
network events.
surveillance
unit
Patch
passes
Pilot
Performanc
e reports
(KPI) not
available
No data to
evaluate
Pilot exit
5 Data not
reported by in-
house
Performance
Management
unit
8 Y :
Implemented
Operational
Network
surveillance
unit
3 120 Detection; Source
Performance data from
LTE Supplier
5 8 2* 80
*Preliminary assessment – actual implementation will provide measurement data

Process Management Storyboard – Results & learning’s
During System Introduction Operability and Customer experience are treated with equal importance.
Historically the quality KPI’s measured during Rollout phase are not achieved.
The RN Release Management capability is fit-for-purpose (skill set + manpower) but with Lab/Test Plant constraints
The CTQ’s and Function are acceptable to Central Op’s and thus are mapped to Output Matrix and data collection.
Improvement opportunities in Lab test coverage (scope and time) and Pilot extension are focus of 6 Sigma study.
Partial implementation of improvements in current ‘N’ Release XXYY (Patch turnaround time + Effort).
Full implementation of improvements with benefits expected in ‘N+1’ Release Introduction (AABB).Full implementation of improvements with benefits expected in ‘N+1’ Release Introduction (AABB).
Key Learning’s from describing &
Measuring Process
Analysis of defects found in field - 65% of defects are
reproducible in Lab
Results of key measures since applying
PMGB tools and techniques
Effected extension of Test process cycle
time (5d minimum) with measured increase
in Lab defect ‘capture’ rate
Effort KPI contained to ca 3.75 x FTE’s
(Work force reporting Feb 2012 –Dec 2012)
35%
Not Reproducible
(Work force reporting Feb 2012 –Dec 2012)
Investment uplift in Lab necessary to
increase defect capture rate
35%
65%
Reproducible

Glossary
RN – Radio Networks ; central skill centre tasked
with Release Management
BU – Business Unit
FTE – Full Time EmployeeFTE – Full Time Employee
SLA – Service Level Agreement
PR – Problem Report
KPI – Key Performance Indicator
SW – Software
MS – Milestone
Lab – Representative system for Release Testing
Lab – Representative system for Release Testing
System – Supplier’s aggregate of equipment & SW
LTE – Long Term Evolution : 4th generation 3GPP
standardised system for Mobile communication

Required (‘7+1’)Tools for PMGB Workbook
5S (incl. photos before/after)
CAN DO - Tools
(select one for certification)
SIPOC
Seven MUST DO – Tools
(all are required for certification) ++
5S (incl. photos before/after)
Value Analysis
Time Analysis (incl. process
balancing)
FMEA
Poka Yoke
SIPOC
VoC CTQ (Tool 1)
Output Measurement Variables
(Tool 2)
Data Collection / Measurement
Plan ( incl. Operational Definition)
Graphical display of process
performance
Value Stream Map/Swim-Lane
PMGB Workbook V34_engl Process name : LTE System Release Management 20
Value Stream Map/Swim-Lane
RACI (Responsibility Matrix)
06.07.2013

6 sigma LTE release management process improvement

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (20)

Semelhante a 6 sigma LTE release management process improvement

Semelhante a 6 sigma LTE release management process improvement (20)

Último

Último (20)

6 sigma LTE release management process improvement