Why Cybersecurity Needs Big Data
•Transferir como PPTX, PDF•
5 gostaram•1,928 visualizações
The connected world creates a rate and volume of streaming cybersecurity data that is unprecedented, and attacks are increasingly sophisticated and multifaceted. Yet it is unreasonably time-consuming for security personnel to piece together data from multiple systems to assess the true nature of a single threat across an enterprise. Learn how big data and data science teams can help information security teams improve threat detection with machine learning and real-time streaming analytics. You will hear from, Michael Schiebel, cybersecurity strategist and James Sirota, Apache Metron committer and Director of Security Solutions at Hortonworks on how to apply big data technology to prevent cybercrime. Leveraging big data and machine learning, Apache Metron can help detect phishing attacks such as the Yahoo security breach by Russian spies. See how Apache Metron accelerates the process of investigating a phishing attack – slides 16-19. You can also learn more about Apache Metron here: https://hortonworks.com/apache/metron/ and join the community at user-subscribe@metron.incubator.apache.org or dev-subscribe@metron.incubator.apache.org View the on-demand webinar: https://hortonworks.com/webinar/why-cybersecurity-needs-big-data/
Recomendados
Recomendados
Mais conteúdo relacionado
Mais de Hortonworks
Mais de Hortonworks (20)
Último
Último (20)
Why Cybersecurity Needs Big Data
- 1. Why Cybersecurity Needs Big Data & Intro to Apache Metron James Sirota, Director Security Solutions March 2017 Michael Schiebel, Cybersecurity Strategy
- 2. 2 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Michael Schiebel, Cybersecurity Strategy www.linkedin.com/in/michaelschiebel/ James Sirota Director Security Solutions www.linkedin.com/in/jsirota/ Anna Yong Cybersecurity Product Marketing www.linkedin.com/in/4everfusion/
- 3. 3 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Agenda Why Cybersecurity Needs Big Data Intro to Apache Metron How big data experts can help IT security teams Case Study: Accelerating Investigation of a Phishing Attack
- 4. 4 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Why Cybersecurity Needs Big Data
- 5. 5 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Digital World Generates Big Data That Security Teams Need to Process
- 6. 6 © Hortonworks Inc. 2011 – 2016. All Rights Reserved6 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Existing Cyber Security Solutions Don’t Scale to the Challenge 82% of breaches happened in minutes 8 months: Average time an advanced security breach goes unnoticed 70%-80% of breaches are first detected by a 3rd party. 2016 Verizon Data Breach Investigations Report Current security tools installed in the data center can’t handle volume of data & threats from everywhere
- 7. 7 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Cybersecurity Journey Single View into Security Risk Free data from security tools Correlate and discover threats Operational efficiency and governance Predictive insights using machine learning Single unified view of enterprise risk & security posture. Innovate Renovate Single Holistic View Historical Records OPEX Reduction Security Tool Ingest Digital Protection Fraud Prevention Public Data Capture ACTIVE ARCHIVE DATA DISCOVERY PREDICTIVE ANALYTICS Cyber Security Machine Data Risk Modeling
- 8. 8 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Apache Metron at Capital One CapOne uses HDF to ingest log data into their cyber security data lake and uses Apache Metron to detect threats that cannot be detected by traditional cyber security tools https://youtu.be/Nffx8SKn7l4?t=1h37m50s
- 9. 9 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Introduction to Apache Metron
- 10. 10 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Metron Journey Jan 2016 OpenSOC renamed Metron Dec 2016 Accepted into Apache Incubation Oct. 2015 Hortonworks, Mantech, B23 press release Sept 2014 First release of OpenSOC Beta by Cisco April 2014 OpenSOC in production June 2014 OpenSOC Community Edition July 2015 Cisco stops supporting OpenSOC March 2016 First Apache Release
- 11. 11 © Hortonworks Inc. 2011 – 2016. All Rights Reserved DataServicesandIntegrationLayer Search and Dashboarding Portal Security Data Vault Community Analytical Models Provisioning, Management and Monitoring ModulesReal-time Processing Cyber Security Engine Telemetry Parsers Enrichment Threat Intel Alert Triage Indexers and Writers Cyber Security Stream Processing Pipeline Apache Metron: Incubating Project TelemetryIngestBuffer Telemetry Data Collectors Real-time Enrich / Threat Intel Streams Performance Network Ingest Probes / OtherMachine Generated Logs (AD, App / Web Server, firewall, VPN, etc.) Security Endpoint Devices (Fireye, Palo Alto, BlueCoat, etc.) Network Data (PCAP, Netflow, Bro, etc.) IDS (Suricata, Snort, etc.) Threat Intelligence Feeds (Soltra, OpenTaxi, third-party feeds) Telemetry Data Sources
- 12. 12 © Hortonworks Inc. 2011 – 2016. All Rights Reserved How Big Data Experts Can Help Security Teams
- 13. 13 © Hortonworks Inc. 2011 – 2016. All Rights Reserved A Day in the Life of An Analyst: • Too many disparate tools • Too many alerts to process • Too much noise • How to connect the dots of the relevant data points together?
- 14. 14 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Problem Posed (For Existing Tools) Security Information Management System (SIEM) • I am prohibitively expensive • I have vendor lock-in • I can’t deal with big data • I am not open • I am not extensible enough Legacy Point Tools • I was built for 1995 • I am super specialized • I don’t scale horizontally • I have a proprietary format • You need a PhD to operate me Behavioral Analytics Tools (UEBA) • I have a limited # of models • I am not trained on YOUR data • I am built by a small startup
- 15. 15 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Problem Posed (For Bad Guys) Advanced Persistent Threat Script Kiddie • I am very unique in a way I do things • I live on your network for about 300 days • I know what I am after and I look for it, slowly • Your rules will not detect me, I am too smart • I impersonate a legitimate user, but I don’t act like one Apache Metron can take everything that is known about me and check for it in real time Apache Metron can model historical behavior of whoever I am impersonating and flag me as I try to deviate • My techniques are predictable and known • My attack vectors are also known • I fumble around a lot • I set off a large number of alerts • You are not the only person I’ve attacked • I brag about what I did or will do Repeatable Patterns Unique Patterns
- 16. 16 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Case Study: Accelerate Investigation of a Phishing Attack
- 17. 17 © Hortonworks Inc. 2011 – 2016. All Rights Reserved The “Threat Story” the Workflow Told….
- 18. 18 © Hortonworks Inc. 2011 – 2016. All Rights Reserved The Challenges faced by the SOC Analyst to Create this Story… Challenge • The analyst had to jump from the SIEM to more than 7 different tools that took up valuable time. • It took more than 24 hours across 2 SOC shifts to investigate, determine scope, remediate and do further forensics/investigation. • Half of my time was spending getting the context needed for me to create the story • The threat was detected too late. Instead of detecting the incident on 4/9, the threat should have been detected on 3/20 when the attacker spoofed Sonja’s email address Need • Want a Centralized View of my data so I don’t have to jump around and learn other tools Eliminate manual tasks to investigate a case • Need to discover bad stuff quicker • Need the System to create the context for me in real-time • The current static rules in the SIEM didn’t detect the threat. Need smart analytics based on: • User Sonja hasn’t used corp gmail in the last 3 months • User Sonja can’t login from Ireland and Southern Cali at the same time
- 19. 19 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Old School vs. New School Security Controls Email Security Rules Firewall Rules IDS Rules Sandbox Rules DLP RulesOld School -> (1-1) New School -> (1-*) Email Classifier Alerts Triage Malware Family Classifier Network Behavior Classifier UEBA System
- 20. 20 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Apache Metron Resources http://hortonworks.com/apache/metron/ https://metron.incubator.apache.org/ https://www.meetup.com/futureofdata- london/events/237165504/
- 21. 21 © Hortonworks Inc. 2011 – 2016. All Rights Reserved Questions?
Notas do Editor
- Talk Script: We all know that data is one of the factors changing the world along with as-a-service ad the primary IT delivery methodology. Data is growing, and is everywhere – yet protections still use the obsolete gate-and-moat security paradigm. We now need security solutions that follow the data and protect it wherever and whenever it resides. Example: DNC hack. Podesta emails lost to hackers, and the DNC only found out about it when the FBI told them.
- Talk Script: The Cyber Security journey is an iterative one where help our customers identify their current tactical need and show value by: Freeing data from proprietary point security products (Active Archive) Build new value by correlating and visualizing to improve business processes. Enabling the business to leverage predictive analytics to reduce risk and improve efficiency Enable the business to have a single view of the company’s security posture
- 8
- To understand Apache Metron, we have to first start with the origins of the project which emerged from a Cisco Project called OpenSOC. Cisco stopped supporting it in July 2015 and their Chief Data Scientist and founding team leader, James Sirota, joined Hortonworks to build Apache Metron, a next generation cybersecurity analytics application built on top of big data technologies (Storm, Spark, Hbase, SOLR, HDFS) and NiFi
- TALK TRACK The project is called Apache Metron. It’s an incubating Apache project and we would love for anyone interested in be more involved with it. It’s designed to be a comprehensive via of all cybersecurity data, all accessed through a single pane of glass. The data from multiple sources – security endpoints such as Fireeye, Palo Alto, Bluecoat are part of the picture – these companies are doing amazing well, but from a contextual threat perspective they are part of the story. There are also machine logs, network data, threat intelligence feeds – all together this is collected and then processed through a real-time cyber security engine. On the other side – the far right hand size, you see some of the results that enabled by a full contextual view with real-time stream processing – search and dashboarding portal – a single pane of glass as mentioned, shared community analytics models. This allows everyone, the community as a whole to work together to combat cybersecurity threats that are becoming increasing sophisticated and difficult to counter these days.
- These are common tools used by the SOC and what CAP One was investing in before using Apache Metron. There are many challenges associated with each one. I have included examples of some of the most common security tools in a glossary in the Appendix. if you do run into any of these technologies an IT environment, please contact the EP Team
- This slide describes two types of hackers. Script kiddie attack patterns are repeating and the advanced hacker uses unique patterns to attack you. The script kiddie wants the power of the hacker without having skills of their own. They are an unsophisticated criminal who cannot create a custom breach and waits for an attack to be weaponized from a known vulnerability. They also create a large amount of noise for security tools. The advanced hacker has been trained or funded by a state or criminal organization ad this where you want most of your resources dedicated because the sponsored agent is aiming for something very specific and these attacks are frequent and much more difficult to detect as they have no known signatures. Apache Metron speeds up the detection of APTs. Find out more here: http://hortonworks.com/apache/metron/
- TALK TRACK If you want to know more, here’s some great resources to check out. Please join the Apache Metron community if you would like to participate. We will also be hosting meetsup in various cities – if you have a specific interest please let us know!