1) The document discusses a contingency theory approach to IT service management (ITSM) performance measurement. It presents findings from a study of 6 case organizations on factors influencing their selection of ITSM metrics.
2) Both internal factors (e.g. organizational culture, governance) and external factors (e.g. legislation, industry) influence metric selection. Common internal factors across cases included governance framework, corporate strategy/goals, and IT function structure.
3) The study developed a model of ITSM performance measurement with dimensions of service, function, process and technology, and contingent internal and external factors. Organizations should tailor their metrics based on their unique environment and contingencies.
5. ITSM Performance Measurement
“The process of quantifying the efficiency and effectiveness
of action”
(Neely et al., 2005).
A broad term that “covers both overall economic and
operational aspects” (Tangen, 2005) including measures of
productivity, profitability and quality.
6. Contingency Theory
May lead to a better understanding of ITSM
performance measurement by enabling the
identification of “specific aspects of a performance
measurement system that are associated with certain
defined circumstances”
(Rejc 2004).
Contingency theory has been used in previous studies
that focused on IS.
Saunders and Jones (1992) and Myers et al. (1997).
7. ITIL performance measurement studies
Research approach
(authors)
Underlying theory/ model
Study main purpose
Qualitative (Case study)
SERVQUAL and IT
SERVQUAL
Quality of IT individual
services
Value based management
SERVQUAL, BSC
IT services procurement
Hochstein (2004)
Theoretical (Literature
review), market acceptance
survey
Praeg and Schnabel (2006)
Theoretical (Literature review) BSC, activity diagrams
Donko and Traljic (2009)
Design science
McNaughton et al. (2010)
IS SERVQUAL and IS
Reverse SERVQUAL
Explaining measuring quality
through variation in service
level and business loss
Building a framework for
measuring benefits and value
of ITIL
8. Research Method
Survey of
ITSM benefits
and metrics
• Quantitative and qualitative
analysis of surveys
6 case studies
• Content analysis
of case studies
• Cross case
analysis
9. 6 Case Study Organisations
Service Plan Director,
Performance Manager
Client Services Director
P
u
b
l
i
c
• Health & community
services
• > 10,000 Staff
• Admin - IT MSP
• 200 to 999 staff
$
$5 to $9 Million
$
• IT MSP
• > 10,000 staff
> $150 Million
ITSM 7 years
> $150 Million
$
Service Desk Manager
•Health & community
services
•2,000 to 4,999 staff
•Not for Profit
•200 to 999 staff
$
> $150 Million
ITSM 3 years
> $150 Million
ITSM 4 years
Services Manager,
Reporting Analyst,
Project Office Manager,
Operations Manager
QA & Certification
Manager
$
•Education
•5,000 to 9,999 staff
ITSM 8 years
ITSM 4 years
P
r
i
v
a
t
e
Biz Integration Manager,
Service Level Manager,
Service Manager,
IT Serv. Office Director,
$
> $150 Million
ITSM 1 year
12. Cross Case Analysis Summary
1. Organisation
culture INT
2. ITSM manager
perspective INT
3. Legislation EXT
A
B
4.
F
E
C
1. Senior
management
philosophy
INT
2. Industry
sector EXT
1.
2.
3.
D
INT
governance,
strategy & goals
IS Function
structure
ICT tools in use
13. ITSM Performance Metrics Dimensions
• The ITIL continual service improvement
(OGC 2007a) book classifies metrics into:
Service
Function
Proces
s
Technology
14. ITSM Performance Measurement Model
ITSM Performance
DIMENSIONS & Sample
Metrics
SERVICE
• Customer satisfaction of end-to-end
service
• Response time
• Transaction time
FUNCTION
• Actual spend against budget
• Number of complaints and compliments
PROCESS
•
•
•
•
Number of incidents
Number of successful changes
Incidents caused by changes
Operating level agreement breaches
TECHNOLOGY
• Network bandwidth usage
• Server power consumption
External Contingency Factors
Legislation (5)
Industry sector (5)
ITSM resources (4)
External customers (3)
•
•
•
•
Internal Contingency Factors
Parent Organisation
Governance framework (6)
Corporate strategy & goals (6)
Organisation culture (5)
Senior management philosophy (5)
Internal customers (4)
Corporate performance framework (3)
Senior management needs (3)
CIO influence (3)
•
•
•
•
•
•
•
•
IS Organisation
•
•
•
•
•
•
•
IS function structure (6)
ITSM & ICT tools in use (6)
IS manager perspective (5)
IS function size (4)
IS goals (3)
IS function maturity (3)
IT operations staff influence (3)
Based on Saunders and Jones (1992) and Myers et al. (1997).
Selection of
ITSM
Performance
Metrics
15. Conclusion
3 categories of factors influence selection of ITSM
metrics – external environment, parent organisation
and IS organisation.
4 factors common to 6 cases: governance framework,
corporate strategy and goals, IS function structure and
ITSM and ICT tools in use.
Practitioners should tailor ITSM performance metrics to
the organisation’s environment
16. Links to publications
available at:
http://eprints.usq.edu.au/view/peo
ple/Gacenga=3AFrancis=3A=3A.h
tml
THANK YOU
Francis Gacenga
gacenga@usq.edu.au
Notas do Editor
Why measure the performance of ITSM – significant investment in IT and ITSM, problematic for theory and practice, Understanding factors influencing the selection of ITSM performance measurement will contribute to addressing this problem.
USQ researchers Prof. Cater-Steel, Prof. Toleman, Dr. Tan and myselfPartners – ARC, QH and itSMF
Why we conducted the studyHow we conducted the studyWhat we learnt from the study
ITSM is an approach to customer oriented IT functionsITSM focuses on service creation, design, delivery and maintenance.Typically adopting ITSM involves consultant, training, ITIL books, itSMF, ITSM software
Measurement is one of the underlying elements of service science.This paper focuses on the selection of ITSM performance metrics and contingency theory provides a theoretical foundation.
Value co-creation of services requires we understand how the environment influences the management of IT as a serviceCT - loosely organised propositions that endorse view that there are no universally valid rules of organisation and managemement (Luthans 1973, Rejc 2004)CT enables researcher to relax assumptions underlying theoretical propositions and systematically introduce factors to explain phenomena (Umanath 2003)----- Meeting Notes (23/11/11 16:00) -----`
Methods: Case studies, LRs, DSTheory/Model – SERVQUAL BSCFocus: Quality, benefits and value evaluationWe advance this area by considering the factors that influence the selection of ITSM performance metrics
Survey – 2085 ITSMFA members in 2009Case Study – 2 hour interviews at 6 organisationsContent Analysis – 4 researchers, interview transcripts, survey responses, organisation documentsCross-Case Analysis – used to aggregate case study findings
Org A: MSP Undergoing organisation structure changes. Main influence organisation cultureOrg B: Health: Series of changes in CIOs, BSC tapered off. CIO influence main factorOrg D: EDU: Changing over the ITSM tool. CIO influence and to an extent legislation were influencesOrg C: MSP: Used a combination of frameworks & stds. IS function structure and external customers main influenceOrg E: Charity: No specific framework for measuring performance. Internal customers were key influence.Org F: Community: Early stages of implementing ITSM. ITSM Manager perspective, tools and IS goals key influence.
Environment variables are factors beyond the control of the organistations resource managers (Luthans and Stewart 1977)Internal environment factors – within the organisation but outside the control of the ITSM manager (Luthans and Stewart 1977)Organisations: A to FFont size represents the frequency of factor across the organisation.
External factors – outside the organisational system
Case study findings suggest that selection of ITSM performance metrics is influenced by a range of internal and external environment factors.Case A and C the two MSPs both had external customers as an influencing factorThe was variation in the range of influencing factors. Case E reported 16 environmental factors and Case D reported 6 external factors
The case study organisations primarily measured performance usingtechnology, process and function metrics. Few end to end (service) metricsTechnology e.g. Network bandwidth usageProcess e.g. Number of incidentsFunction e.g. Actual spend against budgetService e.g. customer satisfaction
The selection of ITSM performance metrics is influenced by internal and external environment factorsNew internal contingency factors such as ITSM and ICT tools and corporate performance measurement frameworkOf the external factors identified in previous studies only industry sector and resourcesOrganisations using a corporate performance framework (ACD) had a clearly defined set of metrics