O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
Alexey Lukatsky
Security business development manager
ICS Cyber Security
Effectiveness Measurement
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Not Petya / Nyetya Tools
Tactics
• Supply chain and ...
ICS Kill Chain
1
2
3
4
5
6
7
8
Conficker
APT1
Иран vs
США
BE3
HAVEX
Stuxnet
Ukraine
2016
WannaCry
Neytya
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Why we need to
measure our
effectiveness?
• Good sec...
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Rare Remote Possible Likely Very likely
Catastrophic...
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cybersecurity is state of
protection of the interests of
enterprise stakeholders in the
information area, determined by
th...
Efficiency/effectiveness
is the quantifiable
contribution to the
achievement of ultimate
goals
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
What goals can we have?
• Fulfillment of NERC CIP or...
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
• Operational (наиболее привычные)
• Realtime, day-t...
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Tactical metrics examples
• Incidents requiring manu...
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Tactical metrics examples
• % of ICS without known s...
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Tactical metrics examples
• Time between creating an...
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
SMART principle for metrics selection
• SMART – Spec...
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
SMART usage example for ICS Cybersecuirty
Characteri...
How to move from
hundreds of operational
metrics to one or two
strategic?
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
From individual metrics to measurement program
• EPR...
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
From individual metrics to measurement program
Strat...
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
From individual metrics to measurement program
Tacti...
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
From individual metrics to measurement program
Opera...
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Automation tool: EPRI MetCalc
What does the business
think of all these metrics?
Business thinks about cybersecurity, but in its
own way
Reservoir
Pump
Water intake
Water
treatment
plants
Underground
tan...
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
The difference in the perception of top
management a...
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Time
Productivity
0
20
40
60
80
100
А
В
С
ВТ
Т1 Т2 Т...
Let's try to reformulate our goals
Profit increase
Geo expansion
Sales increase
Production optimization
Reduction in logis...
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
From the “for myself” measurement to the
measurement...
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cybersecurity incidents loss types
Productivity
•Dow...
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Impact categories Insignificant Minor Moderate Signi...
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Questions for defining strategic business metrics
of...
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Impact categories Insignificant Minor Moderate Signi...
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
The duration of an cybersecurity incident in terms
o...
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Impact categories Insignificant Minor Moderate Signi...
How to measure
cybersecurity for a
business, but not with
money?
Can compare yourself with competitors?
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
План & бюджет
Организация
Защитные меры
Архитектура
П...
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
5 important metrics
• % of cybwersecurity activities...
© 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Common errors in effectiveness measuring
• Choosing ...
Key Success Factors
• You must understand what you are doing
in the field of information security
• You must understand yo...
ANewLookat
Cybersecurity
Measurement
Thank you!
security-request@cisco.com
ICS Cyber Security Effectiveness Measurement
Próximos SlideShares
Carregando em…5
×
Próximos SlideShares
What to Upload to SlideShare
Avançar
Transfira para ler offline e ver em ecrã inteiro.

1

Compartilhar

Baixar para ler offline

ICS Cyber Security Effectiveness Measurement

Baixar para ler offline

Presentation about ICS Cybersecurity Effectiveness Measurement on Kaspersky ICS Cybersecurity Conference

Livros relacionados

Gratuito durante 30 dias do Scribd

Ver tudo

ICS Cyber Security Effectiveness Measurement

  1. 1. Alexey Lukatsky Security business development manager ICS Cyber Security Effectiveness Measurement
  2. 2. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Not Petya / Nyetya Tools Tactics • Supply chain and victim to victim pivoting • Rapid Infection Spread • Destroyed Countless Systems / Networks Processes • Designed to inflict damage as quickly and effectively as possible. • Appears to be Ransomware, but is purely destructive • Wormable Ransomware • Designed to Spread Internally Not Externally • Leveraged Eternal Blue / Eternal Romance and Admin Tools (WMI/PSExec) • Advanced Actor associated with a Nation State • Destructive Attack Masquerading as Ransomware • Most Expensive Incident in History Description
  3. 3. ICS Kill Chain 1 2 3 4 5 6 7 8 Conficker APT1 Иран vs США BE3 HAVEX Stuxnet Ukraine 2016 WannaCry Neytya
  4. 4. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Why we need to measure our effectiveness? • Good security not visible • We want to show that we work well • Top management often wants to compare itself with others • We want to see the dynamics
  5. 5. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Rare Remote Possible Likely Very likely Catastrophic 6 7 8 9 10 Significant 5 6 7 8 9 Moderate 4 5 6 7 8 Minor 3 4 5 6 7 Insignificant 2 3 4 5 6 Accept (score = 2,3) Monitor (score = 4,5) Manage (score = 6) Avoid / Resolve (score = 7) Urgently avoid/ Resolve (score = 8, 9, 10) “Best practices” for security measurement • Not specifically, not quantitatively, conditionally… Impact Probability
  6. 6. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
  7. 7. Cybersecurity is state of protection of the interests of enterprise stakeholders in the information area, determined by the totality of balanced interests of the individual, society, state, and business Or process? Not important!
  8. 8. Efficiency/effectiveness is the quantifiable contribution to the achievement of ultimate goals
  9. 9. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public What goals can we have? • Fulfillment of NERC CIP or ISA/IEC 62443 requirements • Categorization of all CI objects • Certification of key processes for ISO/IEC 27019 • Reduce the number of ICS cybersecurity incidents to 3 per month • Implementation of secure remote access to ICS for contractors • Reduce downtime from ICS cybersecurity incidents to 2 hour on average • Cost reduction for ICS cybersecurity for 15%
  10. 10. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public • Operational (наиболее привычные) • Realtime, day-to-day • Logs, rules, signatures, etc. • How effective is your security measures? • Tactical • Change control • Scorecards and audits • How effective is your security program? • Strategic • Corporate risk and business alignment • How are we secure? Strategic Tactical Operational Measurements are different
  11. 11. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Tactical metrics examples • Incidents requiring manual cleanups • Mean-Time-to-Fix • Also TTR (Time-to-Recovery) or TTC (Time-to-Contain) • Mean-Time-to-Detect • Mean-Time-to-Patch • Involvement of staff in cybersecurity activities • Mean cost to mitigate vulnerabilities
  12. 12. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Tactical metrics examples • % of ICS without known severe vulnerabilities with CVSS >7.0 • % of changes with security review • % of changes with security exceptions • ICS cybersecurity budget allocation (% of total, IT, cybersecurity, ICS) • Compliance rate • Cost of incidents
  13. 13. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Tactical metrics examples • Time between creating and closing a ticket for an incident • Ratio of open and "closed" incident reports • Ratio of incidents and tickets • Number of repeat incidents • Ratio of communication methods (e-mail / calls / portal) • Number of false positives (non-existent incidents)
  14. 14. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public SMART principle for metrics selection • SMART – Specific, Measurable, Achievable, Relevant, Timely • As concretely as possible, without double interpretations, for the right target audience • The result should be measurable, not ephemeral • Why choose a goal that is unattainable? • Relevance to goals • Timeliness and relevance
  15. 15. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public SMART usage example for ICS Cybersecuirty Characteristic Example of bad metric Example of good metric Specific The number of failed login attempts to the HMI The number of failed login attempts to the HMI for one week for one employee Measurable Income from the implementation of an ICS cybersecurity The employees loyalty level about ICS Cybersecurity Achivebale The absence of cyber security incidents in ICS for the current quarter The number of ICS cybersecurity incidents in the current quarter <5 Relevant The number of opened projects for ICS cybersecurity The number of completed on time projects for ICS cybersecurity Timely The number of patched ICS nodes last year The number of unpatched ICS nodes current year
  16. 16. How to move from hundreds of operational metrics to one or two strategic?
  17. 17. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public From individual metrics to measurement program • EPRI (Electric Power Research Institute) Research Program • Creating Security Metrics for the Electric Sector (Parts I, II, III, IV) • Applicable to a wide range of industrial enterprises outside the electric power industry 3 strategic metrics 10 tactical metrics 45 operational metrics
  18. 18. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public From individual metrics to measurement program Strategic Metric Name Tactical Metric Name Protection Score Network Perimeter Protection Score Endpoint Protection Score Physical Access Control Score Human Security Score Core Network Vulnerability Control Score Core Network Access Control Score Data Protection Score Security Management Score - Protection Detection Score Threat Awareness Score Threat Detection Score Security Management Score - Detection Response Score Incident Response Score Security Management Score - Response
  19. 19. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public From individual metrics to measurement program Tactical Metric Name Operational Metric Name Network Perimeter Protection Score Mean Access Point Protection Score Mean Wireless Point Protection Score Mean Internet Traffic Protection Score Mean Count-M Malicious Email Mean Count-M Malicious URL Mean Count-M Network Penetration Security Management Score - Protection Security Budget Ratio Security Personnel Ratio Cybersecurity Risk Tolerance Score
  20. 20. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public From individual metrics to measurement program Operational Metric Data input to the Formula Mean Access Point Protection Score Number of inbound connections per day Number of dropped inbound connections per day Number of all alerts per day Number of security alerts per day Number of probes per day Number of confirmed DOS attempts per month Чnumber of confirmed intrusion attempts per month Number of confirmed incidents that required human intervention per month
  21. 21. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Automation tool: EPRI MetCalc
  22. 22. What does the business think of all these metrics?
  23. 23. Business thinks about cybersecurity, but in its own way Reservoir Pump Water intake Water treatment plants Underground tank Pump Distribution Cleaning with reagents, ozone and coal Sump Flats / Houses Water meter Smooth operation Correctand uninterrupted bills Smooth operation Continuous diagnosis Telemetry control Continuous monitoring Proper dosing FZ-152 Order №31 CIP Law Water supply process
  24. 24. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public The difference in the perception of top management and cybersecurity / IT / ICS Cybersecurity / IT / ICS • Deep dive to details • Unwillingness to share collected data • Data for data, not for decisions • What? Where? When? Top management • Bird's-eye view • Data for decision making • What will happen? What to do?
  25. 25. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Time Productivity 0 20 40 60 80 100 А В С ВТ Т1 Т2 Т3 D = System failure / disaster R = The possibility of attenuating or mitigating the effect before or during a negative event A = The ability to absorb and degrade В = Lower limit; threshold value ВТ = Lower limit duration С = Ability to return to baseline D → R How does a business see security incidents? Reduce А? Reduce Вт? Reduce С? Reduce Т1, Т2 and Т3?
  26. 26. Let's try to reformulate our goals Profit increase Geo expansion Sales increase Production optimization Reduction in logistics costs Loss reduction X hours of downtime due to ransomware Y hours of process downtime due to DoS/DDoS-attack Z hours of employee downtime due to spam N rubles fine from supervisory authorities Business Cybersecurity
  27. 27. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public From the “for myself” measurement to the measurement for business 75% 55% Q2 Q1 The number if incidents by sources The number of ICS incidents Downtime Incidents dynamics Contracts loss $35M127
  28. 28. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Cybersecurity incidents loss types Productivity •Downtime •Deterioration of the psychological climate Response •Incident forensics •PR-activity •Support Service Replacement •Equipment replacement •Re-entry of information Fines •Legal costs, pre-settlement •Suspension of deals Competitors •Know-how, commercial secrets •Customer churn, overtaking by competitors Reputation •Goodwill •Decrease in capitalization, stock price Other •Rate downgrade •Decrease in profitability
  29. 29. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Impact categories Insignificant Minor Moderate Significant Catastrophic Finance impact of more than $Y $1М $5М $10М $50М $100М Let's be more specific and measure the money • The cost of direct losses from disruption of business operations • Business Transaction Recovery Cost • Decrease in stock prices (dumb indicator, but sometimes also measurable) • Fines • Lost profit (if you can count it) • Decrease in customer loyalty • Replacing equipment or re-entering information • Interaction with affected customers, etc.
  30. 30. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Questions for defining strategic business metrics of cybersecurity • What will stop or slow down operations in your organization? • What will lead to a decrease in profits / revenue / margin / market share of your company? • What will lead to a decrease in the quality of the product / service? • What will lead to a negative impact on the goal of the company / business unit / business project / executive sponsor?
  31. 31. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Impact categories Insignificant Minor Moderate Significant Catastrophic Outage of more than X customers 10 customers 100 customers 500 customers 1000 customers 5000 customers Business operations disruption of >= Z min / hours / days 1 hour 4 hours 8 hours 2 days 5 days Serious injury to >= A people 0 people 0 people 1 person 10 people 50 people Breach of data for >= B customers 100 customers 1000 customers 5000 customers 10000 customers 100000 customers Loss of >= C customers 5 customers 10 customers 25 customers 50 customers 100 customers Loss of market share for D% 0% 0% 1% 3% 7% Productivity loss for E% 0% 1% 3% 5% 10% If you can’t count in money?
  32. 32. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public The duration of an cybersecurity incident in terms of cybersecurity and business § The influence level and price components of an incident changes over time This illustration can be used to estimate recovery time after an attack RPO – Recovery Point Objectives, RTO – Recovery Time Objectives, MAD – Maximum Allowable Downtime
  33. 33. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Impact categories Insignificant Minor Moderate Significant Catastrophic Reduction of power generation by F megawatts Power reduction is acceptable Power reduction is acceptable 100 MW 1000 MW 10000 MW Impact categories Insignificant Minor Moderate Significant Catastrophic Publications in mass media Absent In local consumer print media On local TV or in local industry publications On national TV or in national consumer print media Highlighted broadcasts or reporting on national TV or in national industry print media Industry specific metrics
  34. 34. How to measure cybersecurity for a business, but not with money?
  35. 35. Can compare yourself with competitors? 0 0,5 1 1,5 2 2,5 3 3,5 4 4,5 План & бюджет Организация Защитные меры Архитектура Процессы и операции Осведомленность Реагирование Управление уязвимостями Оценка рисков Корпоративное управление В среднем по отрасли У нас Tricks: instead of comparing with competitors (if there is no data), you can compare yourself in different states (there was - now - in a year - ideal)
  36. 36. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 5 important metrics • % of cybwersecurity activities unlinked to business goals • Number of projects / activities linked to business goals • % of projects / assets / services that are important for business that do not meet cybersecurity requirements • For example, uncontrolled remote access by contractors • % of projects / assets / services that are important for business and whose security measures are inadequate or ineffective • Or for whom during the incident the response plan did not work • The likelihood of providing services during an cybersecurity incident You can still play with the risks ...
  37. 37. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Common errors in effectiveness measuring • Choosing hundreds of metrics instead of focusing on strategic • Measuring what is easier to measure instead of focusing on measurement goals • Lack of business focus • Focus on operational result-oriented metrics instead of evaluating process performance • Lack of context • Cybersecurity price reduction with incidents growth
  38. 38. Key Success Factors • You must understand what you are doing in the field of information security • You must understand your business • You must understand your target audience • You must be able to combine these three elements together • You need to know where the data is • You must be able to code/program
  39. 39. ANewLookat Cybersecurity Measurement
  40. 40. Thank you! security-request@cisco.com
  • tasheva

    Jan. 31, 2020

Presentation about ICS Cybersecurity Effectiveness Measurement on Kaspersky ICS Cybersecurity Conference

Vistos

Vistos totais

626

No Slideshare

0

De incorporações

0

Número de incorporações

10

Ações

Baixados

26

Compartilhados

0

Comentários

0

Curtir

1

×