SlideShare uma empresa Scribd logo
1 de 33
Metricon 4.0: The importance of context Security measures as a dependent variable Data reveals foundational practices that optimize security and operations  Kurt Milne Managing Director, IT Process Institute [email_address] Gene Kim CTO, Co-Founder, Tripwire, Inc. [email_address] , @RealGeneKim
Where Did The High Performers Come From?
Agenda ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Common Traits of the Highest Performers Source: IT Process Institute  Change management Causality Compliance and continual reduction of operational variance Culture of… ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Mission  - advancing the science of IT management Research Prescriptive Guidance Benchmarking  IT Management Leadership Foster sustainable change  Enable industry comparison Link practice to results
Data driven management - spectrum of influence Intuitive  Analytical  Academic Study Population Sample Population Sample Academic Study Population Sample Population Sample Academic Study Population Study Benchmark ROI Story Best Practice Peer Recommendation Anecdote Prior Experience ITPI Focus Vendors Consultants Industry Analyst Tradeshows Industry Press Blogshere
Vision: Quality Systems Approach ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Prescriptive Guides ,[object Object],[object Object],[object Object],[object Object],[object Object]
Surprise #1: Higher Performing IT Organizations ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Source: IT Process Institute, May 2007
Operations And Security Already Don’t Get Along ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Words often used to describe information security: “hysterical, irrelevant, bureaucratic, bottleneck, difficult to understand, not aligned with the business, immature, shrill, perpetually focused on irrelevant technical minutiae…”
Surprise #2: Three Controls Predicts 60% Of Performance ,[object Object],[object Object],[object Object],[object Object],Source: IT Process Institute, May 2006
Study #1: 2006: ITPI IT Controls Performance Study ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],N = 98 IT Employees IT Budget Average 483 $114 million Min 3 $5 million Max 7,000 $1,050 million
Study #1: 2006: Design Survey: Pick IT Controls Source: COBIT, IT Governance Institute/ISACA We selected the 6 leading BS15000 areas within ITIL that are conjectured to be “where to start.” These were  Access, Change, Resolution, Configuration, Release, Service Levels 1 We then selected 63 COBIT control objectives within these areas. 2 Source: IT Infrastructure Library (ITIL) / BS 15000
Study #1: 2006: The 63 IT Controls The resulting controls that we selected were in the following control categories: •  Access Controls : 17 controls •  Change Controls : 13 controls •  Configuration Controls : 7 controls •  Release Controls : 6 controls •  Service Level Controls : 8 controls •  Resolution Controls : 12 controls
Study #1: 2006: Performance Differences ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Source: IT Process Institute, May 2006 High performers also have 3x higher budgets, as measured by IT operating expense as a function of revenue
2006: Control Differences ,[object Object],[object Object],[object Object],Source: IT Process Institute, May 2006 Foundational Controls: High vs Medium Foundational Controls: Medium vs Low
2006: Three Clusters Of Respondents Each wedge in the pie represents one of the foundational controls. Each bar represents the percentage of the cluster members that responded ‘yes’ to that control.  2 Almost all of the members of the high performing cluster had all of the foundational controls. 3 Almost all of the members of the low performing cluster had no controls, except for access and resolution. 4 Source: IT Process Institute, May 2006 Low Performer Medium Performer High Performer The ITPI identified 23 “foundational controls” and used cluster analysis techniques to identify the relationship between the use of Foundational Controls and performance indicators of the companies studied Three clusters emerged. 1
High Performers Can Bound Maximum MTTR ,[object Object],Source: IT Process Institute, May 2006
MTTR For Large Outages
First Fix Rate
Percentage Of Outages Fixed Within SLA Limits
Change Success Rate
2007: Larger Repeat Benchmark With Even More Fascinating Results ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Source: IT Process Institute/Institute of Internal Auditors (May 2007) N = 350 IT Employees IT Budget Average 587 $236 million Min 2 $1 million Max 3,500 $15 billion
2007: Larger Repeat Benchmark With Even More Fascinating Results ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Source: IT Process Institute/Institute of Internal Auditors (May 2007)
2007: Overall Performance vs. Control Use In Type I, 3 foundational controls explain 60% of performance In Type II organizations, 9 controls explain 60% of the performance ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Source: IT Process Institute/Institute of Internal Auditors (May 2007)
2007: Surprise #1: Type 1 Organizations:  3 Foundational Controls ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],These controls seem familiar… The controls indicate a  culture of change management  and a  culture of causality!
2007: Surprise #2: Type 2 Organizations: 3 + 9 Foundational Controls ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Again, these controls seem familiar – They seem to hint that for complex organizations, enforcing handoffs and accountability is required…
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],These are the organizations that detect process exceptions, but do not  enforce consequences! Group 4 = 2.65 Group 5 = 4.68 Group 4 = 7.25 Group 5 = 7.90 Group 4 = 8.76 Group 5 = 8.40 Average number essential foundational controls, based on level of use in count  Which Type 2f organizations are  “Smoking more, but enjoying it less?” These are the organizations that where the number of foundational controls does not contribute at all to performance! Why? Surprise #3: Control Maturity - How you manage exceptions matters!
2007: Summary of Key Findings ,[object Object],[object Object],[object Object],[object Object],[object Object]
2007: Change, Config Release Study ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Statistical Analysis used to: ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
2007: Key Practices That Predict Performance ,[object Object],[object Object],[object Object],[object Object],[object Object]
Closing Thoughts ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Mais conteúdo relacionado

Mais procurados

Fluke Connect Condition Based Maintenance
Fluke Connect Condition Based MaintenanceFluke Connect Condition Based Maintenance
Fluke Connect Condition Based Maintenance
Frederic Baudart, CMRP
 
Unomaly - product presentation
Unomaly - product presentationUnomaly - product presentation
Unomaly - product presentation
Rudi Wynen
 

Mais procurados (20)

Fluke Connect Condition Based Maintenance
Fluke Connect Condition Based MaintenanceFluke Connect Condition Based Maintenance
Fluke Connect Condition Based Maintenance
 
DevOpsDays Chicago 2014 - Controlling Devops
DevOpsDays Chicago 2014 -  Controlling DevopsDevOpsDays Chicago 2014 -  Controlling Devops
DevOpsDays Chicago 2014 - Controlling Devops
 
We Bought Some Tools
We Bought Some ToolsWe Bought Some Tools
We Bought Some Tools
 
IS audit checklist
IS audit checklistIS audit checklist
IS audit checklist
 
Domain 6 - Security Assessment and Testing
Domain 6 - Security Assessment and TestingDomain 6 - Security Assessment and Testing
Domain 6 - Security Assessment and Testing
 
Designing NextGen Threat Identification Solutions
Designing NextGen Threat Identification SolutionsDesigning NextGen Threat Identification Solutions
Designing NextGen Threat Identification Solutions
 
Hipaa checklist - information security
Hipaa checklist - information securityHipaa checklist - information security
Hipaa checklist - information security
 
Its time to rethink everything a governance risk compliance primer
Its time to rethink everything a governance risk compliance primerIts time to rethink everything a governance risk compliance primer
Its time to rethink everything a governance risk compliance primer
 
Information Systems Audit & CISA Prep 2010
Information Systems Audit & CISA Prep 2010Information Systems Audit & CISA Prep 2010
Information Systems Audit & CISA Prep 2010
 
Security results of_the_wqr_2015_16
Security results of_the_wqr_2015_16Security results of_the_wqr_2015_16
Security results of_the_wqr_2015_16
 
Security at velocity dc cap one
Security at velocity dc cap oneSecurity at velocity dc cap one
Security at velocity dc cap one
 
Unomaly - product presentation
Unomaly - product presentationUnomaly - product presentation
Unomaly - product presentation
 
SuprTEK Continuous Monitoring
SuprTEK Continuous MonitoringSuprTEK Continuous Monitoring
SuprTEK Continuous Monitoring
 
Structured NERC CIP Process Improvement Using Six Sigma
Structured NERC CIP Process Improvement Using Six SigmaStructured NERC CIP Process Improvement Using Six Sigma
Structured NERC CIP Process Improvement Using Six Sigma
 
Understanding the Risk Management Framework & (ISC)2 CAP Module 9: Assess Con...
Understanding the Risk Management Framework & (ISC)2 CAP Module 9: Assess Con...Understanding the Risk Management Framework & (ISC)2 CAP Module 9: Assess Con...
Understanding the Risk Management Framework & (ISC)2 CAP Module 9: Assess Con...
 
Incremental Software Engineering
Incremental Software EngineeringIncremental Software Engineering
Incremental Software Engineering
 
6 Tools for Improving IT Operations in ICS Environments
6 Tools for Improving IT Operations in ICS Environments6 Tools for Improving IT Operations in ICS Environments
6 Tools for Improving IT Operations in ICS Environments
 
Why Patch Management is Still the Best First Line of Defense
Why Patch Management is Still the Best First Line of DefenseWhy Patch Management is Still the Best First Line of Defense
Why Patch Management is Still the Best First Line of Defense
 
Security management concepts and principles
Security management concepts and principlesSecurity management concepts and principles
Security management concepts and principles
 
Information Assurance Metrics: Practical Steps to Measurement
Information Assurance Metrics: Practical Steps to MeasurementInformation Assurance Metrics: Practical Steps to Measurement
Information Assurance Metrics: Practical Steps to Measurement
 

Destaque

DevOps Kanban Meet Up 3/22/12
DevOps Kanban Meet Up 3/22/12DevOps Kanban Meet Up 3/22/12
DevOps Kanban Meet Up 3/22/12
Gene Kim
 
Cat 250 slideshare assignment
Cat 250 slideshare assignmentCat 250 slideshare assignment
Cat 250 slideshare assignment
rdsmith86
 
2012년 소셜미디어의 현황과 전망
2012년 소셜미디어의 현황과 전망2012년 소셜미디어의 현황과 전망
2012년 소셜미디어의 현황과 전망
Chaenam, Jun
 
Love me, fuseli.illustrated.final
Love me, fuseli.illustrated.finalLove me, fuseli.illustrated.final
Love me, fuseli.illustrated.final
J. E. Freed
 
секреты научного текста
секреты научного текстасекреты научного текста
секреты научного текста
ekaterina7
 
2012 SxSW When IT Says No by Gene Kim
2012 SxSW When IT Says No by Gene Kim2012 SxSW When IT Says No by Gene Kim
2012 SxSW When IT Says No by Gene Kim
Gene Kim
 
2012 05 corp fin 1c
2012 05 corp fin 1c2012 05 corp fin 1c
2012 05 corp fin 1c
Gene Kim
 

Destaque (20)

The archived Canadian US Patent Competitive Intelligence Database (2014/10/14)
The archived Canadian US Patent Competitive Intelligence Database (2014/10/14) The archived Canadian US Patent Competitive Intelligence Database (2014/10/14)
The archived Canadian US Patent Competitive Intelligence Database (2014/10/14)
 
DevOps Kanban Meet Up 3/22/12
DevOps Kanban Meet Up 3/22/12DevOps Kanban Meet Up 3/22/12
DevOps Kanban Meet Up 3/22/12
 
Cat 250 slideshare assignment
Cat 250 slideshare assignmentCat 250 slideshare assignment
Cat 250 slideshare assignment
 
Social networking business
Social networking businessSocial networking business
Social networking business
 
Problemasdolor
ProblemasdolorProblemasdolor
Problemasdolor
 
Maquines simples
Maquines simplesMaquines simples
Maquines simples
 
2012년 소셜미디어의 현황과 전망
2012년 소셜미디어의 현황과 전망2012년 소셜미디어의 현황과 전망
2012년 소셜미디어의 현황과 전망
 
아이애드
아이애드아이애드
아이애드
 
How fashion stores act on Facebook
How fashion stores act on FacebookHow fashion stores act on Facebook
How fashion stores act on Facebook
 
Tecnologias y sistemas de informacion
Tecnologias y sistemas de informacionTecnologias y sistemas de informacion
Tecnologias y sistemas de informacion
 
Love me, fuseli.illustrated.final
Love me, fuseli.illustrated.finalLove me, fuseli.illustrated.final
Love me, fuseli.illustrated.final
 
секреты научного текста
секреты научного текстасекреты научного текста
секреты научного текста
 
101110-Presentation ICSadviseurs-Building better schools conference
101110-Presentation ICSadviseurs-Building better schools conference101110-Presentation ICSadviseurs-Building better schools conference
101110-Presentation ICSadviseurs-Building better schools conference
 
Produccion antigua
Produccion antiguaProduccion antigua
Produccion antigua
 
[Cicle de subvencions 2010] subvencions de l ajuntament
[Cicle de subvencions 2010] subvencions de l ajuntament[Cicle de subvencions 2010] subvencions de l ajuntament
[Cicle de subvencions 2010] subvencions de l ajuntament
 
Maquines simples
Maquines simplesMaquines simples
Maquines simples
 
Happy Mind Fitness - a workshop presentation
Happy Mind Fitness - a workshop presentationHappy Mind Fitness - a workshop presentation
Happy Mind Fitness - a workshop presentation
 
Exposición proyecto Ensilaje de Maiz
Exposición proyecto Ensilaje de MaizExposición proyecto Ensilaje de Maiz
Exposición proyecto Ensilaje de Maiz
 
2012 SxSW When IT Says No by Gene Kim
2012 SxSW When IT Says No by Gene Kim2012 SxSW When IT Says No by Gene Kim
2012 SxSW When IT Says No by Gene Kim
 
2012 05 corp fin 1c
2012 05 corp fin 1c2012 05 corp fin 1c
2012 05 corp fin 1c
 

Semelhante a Itpi metricon 0906a final

Read the article Security Controls that Work by Dwayne Melancon .pdf
Read the article Security Controls that Work by Dwayne Melancon .pdfRead the article Security Controls that Work by Dwayne Melancon .pdf
Read the article Security Controls that Work by Dwayne Melancon .pdf
sales113
 
Read the article Security Controls that Work by Dwayne Melancon.pdf
Read the article  Security Controls that Work by Dwayne Melancon.pdfRead the article  Security Controls that Work by Dwayne Melancon.pdf
Read the article Security Controls that Work by Dwayne Melancon.pdf
sales113
 
2010 06 gartner avoiding audit fatigue in nine steps 1d
2010 06 gartner   avoiding audit fatigue in nine steps 1d2010 06 gartner   avoiding audit fatigue in nine steps 1d
2010 06 gartner avoiding audit fatigue in nine steps 1d
Gene Kim
 
· Processed on 09-Dec-2014 901 PM CST · ID 488406360 · Word .docx
· Processed on 09-Dec-2014 901 PM CST · ID 488406360 · Word .docx· Processed on 09-Dec-2014 901 PM CST · ID 488406360 · Word .docx
· Processed on 09-Dec-2014 901 PM CST · ID 488406360 · Word .docx
LynellBull52
 
How much does it cost to be Secure?
How much does it cost to be Secure?How much does it cost to be Secure?
How much does it cost to be Secure?
mbmobile
 
2011 09 19 LSPE Dev Ops Cookbook 1a
2011 09 19 LSPE Dev Ops Cookbook 1a2011 09 19 LSPE Dev Ops Cookbook 1a
2011 09 19 LSPE Dev Ops Cookbook 1a
Gene Kim
 
Running head AUDITING INFORMATION SYSTEMS PROCESS .docx
Running head AUDITING INFORMATION SYSTEMS PROCESS              .docxRunning head AUDITING INFORMATION SYSTEMS PROCESS              .docx
Running head AUDITING INFORMATION SYSTEMS PROCESS .docx
joellemurphey
 
2011 09 18 United "Platitudes, reality and promise"
2011 09 18 United "Platitudes, reality and promise"2011 09 18 United "Platitudes, reality and promise"
2011 09 18 United "Platitudes, reality and promise"
Gene Kim
 
Five biggest secrets to an it audit webinar slides
Five biggest secrets to an it audit webinar slidesFive biggest secrets to an it audit webinar slides
Five biggest secrets to an it audit webinar slides
Michelle
 
Building Information System
Building Information SystemBuilding Information System
Building Information System
Rabia Jabeen
 
2010 07 BSidesLV Mobilizing The PCI Resistance 1c
2010 07 BSidesLV Mobilizing The PCI Resistance 1c 2010 07 BSidesLV Mobilizing The PCI Resistance 1c
2010 07 BSidesLV Mobilizing The PCI Resistance 1c
Security B-Sides
 

Semelhante a Itpi metricon 0906a final (20)

Read the article Security Controls that Work by Dwayne Melancon .pdf
Read the article Security Controls that Work by Dwayne Melancon .pdfRead the article Security Controls that Work by Dwayne Melancon .pdf
Read the article Security Controls that Work by Dwayne Melancon .pdf
 
Read the article Security Controls that Work by Dwayne Melancon.pdf
Read the article  Security Controls that Work by Dwayne Melancon.pdfRead the article  Security Controls that Work by Dwayne Melancon.pdf
Read the article Security Controls that Work by Dwayne Melancon.pdf
 
2010 06 gartner avoiding audit fatigue in nine steps 1d
2010 06 gartner   avoiding audit fatigue in nine steps 1d2010 06 gartner   avoiding audit fatigue in nine steps 1d
2010 06 gartner avoiding audit fatigue in nine steps 1d
 
· Processed on 09-Dec-2014 901 PM CST · ID 488406360 · Word .docx
· Processed on 09-Dec-2014 901 PM CST · ID 488406360 · Word .docx· Processed on 09-Dec-2014 901 PM CST · ID 488406360 · Word .docx
· Processed on 09-Dec-2014 901 PM CST · ID 488406360 · Word .docx
 
IT Audit For Non-IT Auditors
IT Audit For Non-IT AuditorsIT Audit For Non-IT Auditors
IT Audit For Non-IT Auditors
 
How much does it cost to be Secure?
How much does it cost to be Secure?How much does it cost to be Secure?
How much does it cost to be Secure?
 
2011 09 19 LSPE Dev Ops Cookbook 1a
2011 09 19 LSPE Dev Ops Cookbook 1a2011 09 19 LSPE Dev Ops Cookbook 1a
2011 09 19 LSPE Dev Ops Cookbook 1a
 
SDLC Control
SDLC ControlSDLC Control
SDLC Control
 
2011 06 15 velocity conf from visible ops to dev ops final
2011 06 15 velocity conf   from visible ops to dev ops final2011 06 15 velocity conf   from visible ops to dev ops final
2011 06 15 velocity conf from visible ops to dev ops final
 
Running head AUDITING INFORMATION SYSTEMS PROCESS .docx
Running head AUDITING INFORMATION SYSTEMS PROCESS              .docxRunning head AUDITING INFORMATION SYSTEMS PROCESS              .docx
Running head AUDITING INFORMATION SYSTEMS PROCESS .docx
 
2011 09 18 United "Platitudes, reality and promise"
2011 09 18 United "Platitudes, reality and promise"2011 09 18 United "Platitudes, reality and promise"
2011 09 18 United "Platitudes, reality and promise"
 
Resilient systems design
Resilient systems designResilient systems design
Resilient systems design
 
Resilient systems design
Resilient systems designResilient systems design
Resilient systems design
 
Unit Iii
Unit IiiUnit Iii
Unit Iii
 
Best Practices for Access Reviews - How to Reduce Risks and Improve Operation...
Best Practices for Access Reviews - How to Reduce Risks and Improve Operation...Best Practices for Access Reviews - How to Reduce Risks and Improve Operation...
Best Practices for Access Reviews - How to Reduce Risks and Improve Operation...
 
Leveraging Change Control for Security
Leveraging Change Control for SecurityLeveraging Change Control for Security
Leveraging Change Control for Security
 
Security audit
Security auditSecurity audit
Security audit
 
Five biggest secrets to an it audit webinar slides
Five biggest secrets to an it audit webinar slidesFive biggest secrets to an it audit webinar slides
Five biggest secrets to an it audit webinar slides
 
Building Information System
Building Information SystemBuilding Information System
Building Information System
 
2010 07 BSidesLV Mobilizing The PCI Resistance 1c
2010 07 BSidesLV Mobilizing The PCI Resistance 1c 2010 07 BSidesLV Mobilizing The PCI Resistance 1c
2010 07 BSidesLV Mobilizing The PCI Resistance 1c
 

Mais de Gene Kim

The Unicorn Project and The Five Ideals (Updated Dec 2019)
The Unicorn Project and The Five Ideals (Updated Dec 2019)The Unicorn Project and The Five Ideals (Updated Dec 2019)
The Unicorn Project and The Five Ideals (Updated Dec 2019)
Gene Kim
 
2019 12 Clojure/conj: Love Letter To Clojure, and A Datomic Experience Report
2019 12 Clojure/conj: Love Letter To Clojure, and A Datomic Experience Report2019 12 Clojure/conj: Love Letter To Clojure, and A Datomic Experience Report
2019 12 Clojure/conj: Love Letter To Clojure, and A Datomic Experience Report
Gene Kim
 
The Unicorn Project and The Five Ideals (older: see notes for newer version)
The Unicorn Project and The Five Ideals (older: see notes for newer version)The Unicorn Project and The Five Ideals (older: see notes for newer version)
The Unicorn Project and The Five Ideals (older: see notes for newer version)
Gene Kim
 
DevOps: Who Will Create $2.6 Trillion In Business Value Per Year?
DevOps: Who Will Create $2.6 Trillion In Business Value Per Year?DevOps: Who Will Create $2.6 Trillion In Business Value Per Year?
DevOps: Who Will Create $2.6 Trillion In Business Value Per Year?
Gene Kim
 
How Can We Better Sell DevOps?
How Can We Better Sell DevOps?How Can We Better Sell DevOps?
How Can We Better Sell DevOps?
Gene Kim
 
Kevin Behr: Integrating Controls and Process Improvement
Kevin Behr: Integrating Controls and Process ImprovementKevin Behr: Integrating Controls and Process Improvement
Kevin Behr: Integrating Controls and Process Improvement
Gene Kim
 
Infosec at Ludicrous Speeds - Rugged DevOps
Infosec at Ludicrous Speeds - Rugged DevOps Infosec at Ludicrous Speeds - Rugged DevOps
Infosec at Ludicrous Speeds - Rugged DevOps
Gene Kim
 

Mais de Gene Kim (20)

Speaker Recording Tips For Virtual DevOps Enterprise (And Why We're Pre-Recor...
Speaker Recording Tips For Virtual DevOps Enterprise (And Why We're Pre-Recor...Speaker Recording Tips For Virtual DevOps Enterprise (And Why We're Pre-Recor...
Speaker Recording Tips For Virtual DevOps Enterprise (And Why We're Pre-Recor...
 
The Unicorn Project and The Five Ideals (Updated Dec 2019)
The Unicorn Project and The Five Ideals (Updated Dec 2019)The Unicorn Project and The Five Ideals (Updated Dec 2019)
The Unicorn Project and The Five Ideals (Updated Dec 2019)
 
2019 12 Clojure/conj: Love Letter To Clojure, and A Datomic Experience Report
2019 12 Clojure/conj: Love Letter To Clojure, and A Datomic Experience Report2019 12 Clojure/conj: Love Letter To Clojure, and A Datomic Experience Report
2019 12 Clojure/conj: Love Letter To Clojure, and A Datomic Experience Report
 
GitHub Universe: 2019: Exemplars, Laggards, and Hoarders A Data-driven Look a...
GitHub Universe: 2019: Exemplars, Laggards, and Hoarders A Data-driven Look a...GitHub Universe: 2019: Exemplars, Laggards, and Hoarders A Data-driven Look a...
GitHub Universe: 2019: Exemplars, Laggards, and Hoarders A Data-driven Look a...
 
The Unicorn Project and The Five Ideals (older: see notes for newer version)
The Unicorn Project and The Five Ideals (older: see notes for newer version)The Unicorn Project and The Five Ideals (older: see notes for newer version)
The Unicorn Project and The Five Ideals (older: see notes for newer version)
 
2019 Top Lessons Learned Since the Phoenix Project Was Released
2019 Top Lessons Learned Since the Phoenix Project Was Released2019 Top Lessons Learned Since the Phoenix Project Was Released
2019 Top Lessons Learned Since the Phoenix Project Was Released
 
Leading A DevOps Transformation: Lessons Learned
Leading A DevOps Transformation: Lessons LearnedLeading A DevOps Transformation: Lessons Learned
Leading A DevOps Transformation: Lessons Learned
 
Keeping The Auditor Away: DevOps Audit Compliance Case Studies
Keeping The Auditor Away: DevOps Audit Compliance Case StudiesKeeping The Auditor Away: DevOps Audit Compliance Case Studies
Keeping The Auditor Away: DevOps Audit Compliance Case Studies
 
2014 State Of DevOps Findings! Velocity Conference
2014 State Of DevOps Findings! Velocity Conference2014 State Of DevOps Findings! Velocity Conference
2014 State Of DevOps Findings! Velocity Conference
 
DevOps: Who Will Create $2.6 Trillion In Business Value Per Year?
DevOps: Who Will Create $2.6 Trillion In Business Value Per Year?DevOps: Who Will Create $2.6 Trillion In Business Value Per Year?
DevOps: Who Will Create $2.6 Trillion In Business Value Per Year?
 
2013 Velocity DevOps Metrics -- It's Not Just For WebOps Any More!
2013 Velocity DevOps Metrics -- It's Not Just For WebOps Any More!2013 Velocity DevOps Metrics -- It's Not Just For WebOps Any More!
2013 Velocity DevOps Metrics -- It's Not Just For WebOps Any More!
 
How Can We Better Sell DevOps?
How Can We Better Sell DevOps?How Can We Better Sell DevOps?
How Can We Better Sell DevOps?
 
Why Everyone Needs DevOps Now: 15 Year Study Of High Performing Technology Orgs
Why Everyone Needs DevOps Now: 15 Year Study Of High Performing Technology OrgsWhy Everyone Needs DevOps Now: 15 Year Study Of High Performing Technology Orgs
Why Everyone Needs DevOps Now: 15 Year Study Of High Performing Technology Orgs
 
Kevin Behr: Integrating Controls and Process Improvement
Kevin Behr: Integrating Controls and Process ImprovementKevin Behr: Integrating Controls and Process Improvement
Kevin Behr: Integrating Controls and Process Improvement
 
SecureWorld - Communicating With Your CFO
SecureWorld - Communicating With Your CFOSecureWorld - Communicating With Your CFO
SecureWorld - Communicating With Your CFO
 
2012 Velocity London: DevOps Patterns Distilled
2012 Velocity London: DevOps Patterns Distilled2012 Velocity London: DevOps Patterns Distilled
2012 Velocity London: DevOps Patterns Distilled
 
PuppetConf2012GeneKim
PuppetConf2012GeneKimPuppetConf2012GeneKim
PuppetConf2012GeneKim
 
United2012 Rugged DevOps Rocks
United2012 Rugged DevOps RocksUnited2012 Rugged DevOps Rocks
United2012 Rugged DevOps Rocks
 
Infosec at Ludicrous Speeds - Rugged DevOps
Infosec at Ludicrous Speeds - Rugged DevOps Infosec at Ludicrous Speeds - Rugged DevOps
Infosec at Ludicrous Speeds - Rugged DevOps
 
When IT Fails The Business Fails...
When IT Fails The Business Fails...When IT Fails The Business Fails...
When IT Fails The Business Fails...
 

Itpi metricon 0906a final

  • 1. Metricon 4.0: The importance of context Security measures as a dependent variable Data reveals foundational practices that optimize security and operations Kurt Milne Managing Director, IT Process Institute [email_address] Gene Kim CTO, Co-Founder, Tripwire, Inc. [email_address] , @RealGeneKim
  • 2. Where Did The High Performers Come From?
  • 3.
  • 4.
  • 5. Mission - advancing the science of IT management Research Prescriptive Guidance Benchmarking IT Management Leadership Foster sustainable change Enable industry comparison Link practice to results
  • 6. Data driven management - spectrum of influence Intuitive Analytical Academic Study Population Sample Population Sample Academic Study Population Sample Population Sample Academic Study Population Study Benchmark ROI Story Best Practice Peer Recommendation Anecdote Prior Experience ITPI Focus Vendors Consultants Industry Analyst Tradeshows Industry Press Blogshere
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13. Study #1: 2006: Design Survey: Pick IT Controls Source: COBIT, IT Governance Institute/ISACA We selected the 6 leading BS15000 areas within ITIL that are conjectured to be “where to start.” These were Access, Change, Resolution, Configuration, Release, Service Levels 1 We then selected 63 COBIT control objectives within these areas. 2 Source: IT Infrastructure Library (ITIL) / BS 15000
  • 14. Study #1: 2006: The 63 IT Controls The resulting controls that we selected were in the following control categories: • Access Controls : 17 controls • Change Controls : 13 controls • Configuration Controls : 7 controls • Release Controls : 6 controls • Service Level Controls : 8 controls • Resolution Controls : 12 controls
  • 15.
  • 16.
  • 17. 2006: Three Clusters Of Respondents Each wedge in the pie represents one of the foundational controls. Each bar represents the percentage of the cluster members that responded ‘yes’ to that control. 2 Almost all of the members of the high performing cluster had all of the foundational controls. 3 Almost all of the members of the low performing cluster had no controls, except for access and resolution. 4 Source: IT Process Institute, May 2006 Low Performer Medium Performer High Performer The ITPI identified 23 “foundational controls” and used cluster analysis techniques to identify the relationship between the use of Foundational Controls and performance indicators of the companies studied Three clusters emerged. 1
  • 18.
  • 19. MTTR For Large Outages
  • 21. Percentage Of Outages Fixed Within SLA Limits
  • 23.
  • 24.
  • 25.
  • 26.
  • 27.
  • 28.
  • 29.
  • 30.
  • 31.
  • 32.
  • 33.

Notas do Editor

  1. When we look beyond the metrics into what they actually do day-to-day, the high performers set themselves apart and three major areas. First, they placed a significant emphasis on their change management process. In fact, high performers see their change management as a key capability that allows them to become a high performers. The next area of difference, is that high performers operate within a culture of causality. In other words, they placed a high amount of value on understanding why it change happened -- not just that it was resolved. These attributes led to a third aspect that was evident within the high performers: they were very rigorous about enforcing the policies and insisted on compliance from their team members. They also use the data they collected to reduce the variance of configurations and an increase consistency of practice within their operating environments. This was achieved through an emphasis on sending the right cultural message within the company, implementing the correct controls to hold people accountable for adhering to policies, and following through with consequences for not doing things the right way.
  2. Need to redraw with three arrows in circle IT Management Demming rolling up a hill
  3. How each side Actively impedes the achievement of each other’s goals.
  4. Key findings indicate that a set of 21 control activities, what the ITPI calls Foundational Controls, have the broadest impact on key performance measures. Top performers had higher performance measures in key operational metrics such as 12 to 37 percent less unplanned work than medium and low performers, 12 to 26 percent higher change success rate, and 2.5 to 5.4 times higher server to system administrator ratio. In addition, the presence of specific Change and Configuration controls, among others, appear to be the primary differentiators of top performing organizations.
  5. (1) Of the universe of potential controls, the researchers identified 23 "Foundational Controls" spanning 6 areas of ITIL (Access, Change, Config, Release, Service Level Mgt, and Resolution). Foundational controls were those controls which had the highest correlation with positive performance indicators (there were ~50 performance indicators spanning Operations, Compliance, and Security). By segmenting the respondents based on the performance indicators, 3 clusters of performers were apparent - high, medium, and low. (2) (See block 2 above) (3) (See block 3 above) (4) (See block 4 above) You'll also notice that the high performers utilize a very balanced approach to implementing controls, while the other groups have unbalanced approaches. We believe that the appropriate balance of controls is essential for success. However, some controls have a much greater impact than others. Let's discuss which ones those are…
  6. Based on analysis of the top performers in this study, our overall conclusion is that focusing ongoing audit, compliance, and best practice related resources on Foundational Control activities will generate a significant return on investment realized through increased performance on a wide range of key performance measures.