Got SIEM? Now what? Making SIEM work for you!
Dr Anton Chuvakin
SANS 2010
Security Information and Event Management (SIEM) as well as log management tools have become more common across large organizations in recent years. SIEM and log management have also been a topic of hot debates. In fact, you organization might have purchased these tools already. However, many who acquired SIEM tools have realized that they are not ready to use many of the advanced correlation features, despite promises that "they are easy to use." So, what should you do to achieve success with SIEM? What logs should you collect? Correlate? Review? How do you use log management as a step before SIEM? What process absolutely must be built before SIEM purchase becomes successful. Attend this session to learn from the experience of those who did not have the benefit of learning from other's mistakes. Also, learn a few tips on how to "operationalize" that SIEM purchase you've made.
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Got SIEM? Now what? Getting SIEM Work For You
1. Got SIEM?
Now what?
Making SIEM work for you
Dr. Anton Chuvakin
Security Warrior Consulting
www.securitywarriorconsulting.com
SANS @ Night, San Francisco 2010
2. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
Outline
• Brief: What is SIEM/LM?
• “You got it!”
• SIEM Pitfalls and Challenges
• Useful SIEM Practices
– From Deployment Onwards
• SIEM “Worst Practices”
• Secret to SIEM Magic!
• Conclusions
3. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
About Anton
• Former employee of SIEM and log
management vendors
• Now consulting for SIEM vendors and
SIEM users
• SANS class author (SEC434 Log
Management)
• Author, speaker, blogger, podcaster (on
logs, naturally )
8. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
SIEM Evolution
• 1997-2002 IDS and Firewall
– Worms, alert overflow, etc
– Sold as “SOC in the box”
• 2003 – 2007 Above + Server + Context
– PCI DSS, SOX, users
– Sold as “SOC in the box”++
• 2008+ Above + Applications + …
– Fraud, activities, cybercrime
– Sold as “SOC in the box”+++++
9. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
What SIEM MUST Have?
1. Log and Context Data Collection
2. Normalization and categorization
3. Correlation (“SEM”)
4. Notification/alerting (“SEM”)
5. Prioritization (“SEM”)
6. Dashboards and visualization
7. Reporting and report delivery (“SIM”)
8. Security role workflow (IR, SOC, etc)
10. What SIEM Eats: Logs
<122> Mar 4 09:23:15 localhost sshd[27577]: Accepted password for anton from
::ffff:192.168.138.35 port 2895 ssh2
<13> Fri Mar 17 14:29:38 2006 680 Security SYSTEM User Failure Audit
ENTERPRISE Account Logon
Logon attempt by: MICROSOFT_AUTHENTICATION_PACKAGE_V1_0 Logon
account: ANTON Source Workstation: ENTERPRISE Error Code: 0xC000006
A 4574
<57> Dec 25 00:04:32:%SEC_LOGIN-5-
LOGIN_SUCCESS:Login Success [user:anton]
[Source:10.4.2.11] [localport:23] at 20:55:40 UTC Fri Feb 28
2006
<18> Dec 17 15:45:57 10.14.93.7 ns5xp: NetScreen device_id=ns5xp system-
warning-00515: Admin User anton has logged on via Telnet from
10.14.98.55:39073 (2002-12-17 15:50:53)
12. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
Just What Is “Correlation”?
• Dictionary: “establishing relationships”
• SIEM: “relate events together for security
benefit”
• Why correlate events?
• Automated cross-device data analysis!
• Simple correlation rule:
• If this, followed by that, take some action
13. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
Popular #SIEM_FAIL
… in partial answer to “why people think SIEM sucks?”
1. Misplaced expectations (“SOC-in-a-box”)
2. Missing requirements (“SIEM…huh?”)
3. Missed project sizing
4. Political challenges with integration
5. Lack of commitment
6. Vendor deception (*)
7. And only then: product not working
17. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
What is a “Best Practice”?
• A process or practice that
–The leaders in the field
are doing today
–Generally leads to useful
results with cost
effectiveness
P.S. If you still hate it – say
“useful practices”
18. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
BP1 LM before SIEM!
If you remember one thing from this, let it be:
Deploy Log Management
BEFORE SIEM!
Q: Why do you think MOST 1990s SIEM deployments
FAILED?
A: There was no log management! SEM alone is just not
that useful…
19. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
Graduating from LM to SIEM
Are you ready? Well, do you have…
1. Response capability
– Prepared to response to alerts
2. Monitoring capability
– Has an operational process to monitor
3. Tuning and customization ability
– Can customize the tools and content
21. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
BP2 Evolving to SIEM
Steps of a journey …
• Establish response process
• Deploy a SIEM
• Think “use cases”
• Start filtering logs from LM to SIEM
– Phases!
• Prepare for the initial increase in workload
22. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
Example LM->SIEM Filtering
3D: Devices / Network topology / Events
• Devices: NIDS/NIPS, WAF, servers
• Network: DMZ, payment network (PCI
scope), other “key domains”
• Events: authentication, outbound firewall
access
Later: proxies, more firewall data, web
servers
23. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
“Complianc-y” Approach to SIEM
1. List regulations
2. Identify other “use cases”
3. Review whether SIEM/LM is needed
4. Map features to controls
5. Select and deploy
6. Operationalize regulations
7. Expand use
24. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
“Quick Wins” for Phased Approach
Phased
approach #1
• Collect problems
• Plan architecture
• Start collecting
• Start reviewing
• Solve problem 1
• Solve problem n
Phased
approach #2
• Focus on 1 problem
• Plan architecture
• Start collecting
• Start reviewing
• Solve problem 1
• Plan again
25. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
BP3 SIEM First Steps
First step = BABY steps!
• Compliance monitoring
• “Traditional” SIEM uses
– Authentication tracking
– IPS/IDS + firewall correlation
– Web application hacking
• Simple use cases
– based on your risks
What problems do YOU want solved?
26. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
Example SIEM Use Case
Cross-system authentication tracking
• Scope: all systems with authentication (!)
• Purpose: detect unauthorized access to
systems
• Method: track login failures and successes
• Rule details: multiple login failures followed
by login success
• Response plan: user account investigation,
suspension, communication with suspect user
27. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
10 minutes or 10 months?
Our log
management
appliance can
be racked,
configured and
collecting logs in
10 minutes
A typical large
customer takes
10 months to
deploy a log
management
architecture
based on our
technology
?
29. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
Ultimate SIEM Usage Scenarios
1. Security Operations Center (SOC)
– RT views, analysts 24/7, chase alerts
2. Mini-SOC / “morning after”
– Delayed views, analysts 1/24, review and
drill-down
3. “Automated SOC” / alert + investigate
– Configure and forget, investigate alerts
4. Compliance status reporting
– Review reports/views weekly/monthly
30. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
What is a “Worst Practice”?
• As opposed to the “best
practice” it is …
–What the losers in the
field are doing today
–A practice that generally
leads to disastrous
results, despite its
popularity
31. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
WP for SIEM Project scope
• WP1: Postpone scope until after the purchase
– “The vendor says ‘it scales’ so we will just feed
ALL our logs”
– Windows, Linux, i5/OS, OS/390, Cisco –
send’em in!
• WP2: Assume you will be the only user of the
tool
– “Steakholders”? What’s that?
– Common consequence: two or more
simiilar tools are bought
32. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
Case Study: “We Use’em All”
At SANS Log Management Summit 200X…
• Vendors X, Y and Z claim “Big Finance” as
a customer
• How can that be?
• Well, different teams purchased different
products …
• About $2.3m wasted on tools
that do the same!
33. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
WPs for Deployment
• WP3: Expect The Vendor To Write Your Logging
Policy OR Ignore Vendor Recommendations
– “Tell us what we need – tell us what you have”
forever…
• WP4: Unpack the boxes and go!
– “Coordinating with network and system folks is
for cowards!”
– Do you know why LM projects take months
sometimes?
• WP5: Don’t prepare the infrastructure
– “Time synchronization? Pah, who needs it”
• WP6: Deploy Everywhere At Once
– “We need it everywhere!! Now!!”
34. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
Case Study: Shelfware Forever!
• Financial company gets a SIEM tool after many
months of “evaluations”
• Vendor SEs deploy it
• One year passes by
• A new CSO comes in; looks for what is deployed
• Finds a SIEM tool – which database contains
exactly 53 log records (!)
– It was never connected to a production
network…
35. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
WPs for Expanding Deployment
• WP7: Don’t Bother With A Product Owner
– “We all use it – we all run it (=nobody does)”
• WP8: Don’t Check For Changed Needs –
Just Buy More of the Same
– “We made the decision – why fuss over it?”
• WP9: If it works for 10, it will be OK for
10,000
– “1,10,100, …, 1 trillion –
they are just numbers”
36. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
Case Study: Today - Datacenter,
Tomorrow … Oops!
• Log management tool is tested and deployed
at two datacenters – with great success!
• PCI DSS comes in; scope is expanded to
wireless systems and POS branch servers
• The tool is prepared to be deployed in 410 (!)
more locations
• “Do you think it will work?” - “Suuuuure!”, says
the vendor
• Security director resigns …
37. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
More Quick SIEM Tips
Cost countless sleepless night and boatloads
of pain….
• No SIEM before IR plans/procedures
• No SIEM before basic log management
• Think "quick wins", not "OMG ...that SIEM
boondoggle"
• Tech matters! But practices matter more
• Things will get worse before better.
Invest time before collecting value!
38. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
SIEM Resourcing Voodoo
“Things get worse before they get better”
• Hardware – initial + growth
• Software license fees (CPU, device, EPS,
user, etc, etc)
• Support and integration projects
• Operations Personnel (analysts, developer)
• SIEM Administrator Personnel (SA, DBA,
application admin)
39. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
Conclusions
• SIEM will work and has value … but BOTH
initial and ongoing time/focus
commitment is required
• FOCUS on what problems you are trying
to solve with SIEM: requirements!
• Phased approach WITH “quick wins” is
the easiest way to go
• Operationalize!!!
42. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
Questions?
Dr. Anton Chuvakin
Email: anton@chuvakin.org
Site: http://www.chuvakin.org
Blog: http://www.securitywarrior.org
Twitter: @anton_chuvakin
Consulting: http://www.securitywarriorconsulting.com
43. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
More Resources
• Blog: www.securitywarrior.org
• Podcast: look for “LogChat” on iTunes
• Slides: http://www.slideshare.net/anton_chuvakin
• Papers: www.info-secure.org and
http://www.docstoc.com/profile/anton1chuvakin
• Consulting: http://www.securitywarriorconsulting.com/
44. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
More on Anton
• Consultant: http://www.securitywarriorconsulting.com
• Book author: “Security Warrior”, “PCI Compliance”,
“Information Security Management Handbook”, “Know
Your Enemy II”, “Hacker’s Challenge 3”, etc
• Conference speaker: SANS, FIRST, GFIRST, ISSA,
CSI, RSA, Interop, many, many others worldwide
• Standard developer: CEE, CVSS, OVAL, etc
• Community role: SANS, Honeynet Project, WASC, CSI,
ISSA, OSSTMM, InfraGard, ISSA, others
• Past roles: Researcher, Security Analyst, Strategist,
Evangelist, Product Manager
45. Security Warrior Consulting
www.securitywarriorconsulting.com
Dr. Anton Chuvakin
Security Warrior Consulting Services
• Logging and log management strategy, procedures and practices
– Develop logging policies and processes, log review procedures, workflows and
periodic tasks as well as help architect those to solve organization problems
– Plan and implement log management architecture to support your business
cases; develop specific components such as log data collection, filtering,
aggregation, retention, log source configuration as well as reporting, review and
validation
– Customize industry “best practices” related to logging and log review to fit your
environment, help link these practices to business services and regulations
– Help integrate logging tools and processes into IT and business operations
• SIEM and log management content development
– Develop correlation rules, reports and other content to make your SIEM and log
management product more useful to you and more applicable to your risk profile
and compliance needs
– Create and refine policies, procedures and operational practices for logging
and log management to satisfy requirements of PCI DSS, HIPAA, NERC, FISMA
and other regulations
More at www.SecurityWarriorConsulting.com
Editor's Notes
Got SIEM? Now what? Making SIEM work for you!Anton Chuvakin, Ph.D- Tuesday, November 9 - 7:00pm - 8:00pmSecurity Information and Event Management (SIEM) as well as log management tools have become more common across large organizations in recent years. SIEM and log management have also been a topic of hot debates. In fact, you organization might have purchased these tools already. However, many who acquired SIEM tools have realized that they are not ready to use many of the advanced correlation features, despite promises that "they are easy to use." So, what should you do to achieve success with SIEM? What logs should you collect? Correlate? Review? How do you use log management as a step before SIEM? What process absolutely must be built before SIEM purchase becomes successful. Attend this session to learn from the experience of those who did not have the benefit of learning from other's mistakes. Also, learn a few tips on how to "operationalize" that SIEM purchase you've made.============Only so much advice without knowing your environment/needs$10k consulting project CAN save $500k SIEM budget …Assumed in-sourced SIEM, no cloud, MSSP, co-sourcing, outsourcing, etc
Does everybody need a SIEM?Do you need a SIEM?Are you ready for SIEM?Do you want a SIEM?
CISO thinks that SIEM opportunity cost is too big; spend $100k on SIEM vs spend $100k to solve a dozen problems
No problem is truly solved!!
What is correlation? Different definitions given by different people.Dictionary: “establishing relationships”Why correlate events?Cross-device data analysisWhat else one might want to correlate?Events and …
Buy correlation blog posts http://chuvakin.blogspot.com/search/label/SIEM(*) rarely just a vendor: “there is a sucker born every minute”
Another way to decide is to look at what problem you’re trying to solve with the tool. Over the years, the following areas where SIEM and log management tools can deliver value have emerged: Security, detective, and investigative: sometimes also called threat management, this focuses on detecting and responding to attacks, malware infection, data theft and other security issues. It is very useful to see this as two separate factors: monitoring and detection of security issues vs investigation and forensic analysis of security incidents.Compliance, regulatory (global) and policy (local): this focuses on satisfying the requirement of various laws, mandates and frameworks. Most of the mandates have the intention of helping you improve security, so there is a lot of overlap between this and the previous item.Operational, system and network troubleshooting and administration: specific mostly to log management, this use case has to do with investigating system problems as well as monitoring the availability of systems and applications.
Deploy – use - operationalize – get comfortable with!
Organizations that graduate too soon will waste time and effort, and won't any increased efficiency in their security operation. However, waiting too long also means that the organization will never develop the necessary capabilities to secure themselves. In brief, the criteria are:Response capability: the organization must be ready to respond to alerts soon after they are produced.Monitoring capability: the organization must have or start to build security monitoring capability such as a Security Operation Center (SOC) or at least a team dedicated to ongoing periodic monitoring.Tuning and customization ability: the organization must accept the responsibility for tuning and customizing the deployed SIEM tool. Out-of-the-box SIEM deployments rarely succeed, or manage to reach their full potential. Just like college… Graduation tips:Satisfy the graduation criteriaUse a LM vendors that has a good SIEMDeploy LM and use it operationallyPeriodic log reviews = first step to monitoringLook for integrated capability
First, compile a list of regulations that you have to comply with, focus in particular attention to areas where a SIEM or log management tool can be useful. In many cases, the list will contain only one regulation – but the one you absolutely must handle. Next, if possible, review other possible goals that SIEM can help you achieve. Deciding whether SIEM satisfies a critical business need – such as by as an enabling technology for your SOC– is an essential step. Third, at this point you must decide whether you are prepared to work to make SIEM solve your problem – whether compliance or other. Despite help from the vendor and possibly consultants, there are areas where you have to work to make SIEM work. Now, acquire and implement the SIEM solution. This is where you work jointly with the vendor in order to build your initial implementation for regulatory compliance, such as PCI DSS.Now, start actually using SIEM for both “letter and spirit “ of the regulation. This is the most important step in the approach – one of the biggest mistakes organizations make in this area is thinking that simply owning a SIEM tool makes them compliant. In reality, building daily operational procedures and processes to go with your SIEM is the only way to do that. Sadly, few people remember that PCI DSS prescribes a large set of periodic tasks, from annual to daily (log review being the most well-known example of a daily practice) and not just “having logs.” Finally, expand the use case to beyond compliance. Only at this step you can plan for expanding deployment and solving other problems. The tips for that are provided in the next section. One way to quickly grow your security capability is on the incident response side. This is due to the fact that the easiest and most common security use for log management and SIEM tools - beyond compliance - is related to incident response and forensics.
Consulting Servicesfocused on security product strategy, SIEM / log management as well as PCI DSS and other regulatory compliance (details [PDF] )Technology Vendor ServicesThis section of the services is intended for security vendors and security services providers. The focus is on security and compliance strategy for product planning, development and marketing as well as on content development. Product management and strategyReview security product compliance strategy, PCI DSS strategy and optimize them for the marketPerform market assessment and analysis, competitive analysis, product strategy (build/buy/partner); prepare Market Requirements Documents (MRDs)Help develop and refine security product marketing and positioning messages, focused on compliance and new threatsAugment internal Product Management staff for strategic security and compliance projects, use case analysis, product definition, Product Requirement Documents (PRD) developmentWork with product management team to help define and prioritize product features based on market feedback and compliance requirements.Research and content developmentLead content development for whitepapers, "thought leadership"; documents, research papers and other messaging documents, related to security and regulatory compliance (example whitepaper, recent book on PCI DSS)Review security and compliance marketing materials, site contents and other public- or partner-facing materialsCreate correlation rules, reports as well as policies and procedures and other operational content to make SIEM and log management products more useful to your customersMap regulatory compliance controls such as PCI DSS (key focus!), HIPAA, NERC, FISMA, NIST, ISO, ITIL to security product features and document the use of the product in support of the mandatesDevelop compliance content such as reports, correlation rules, queries and other relevant compliance content for security product.Events and webinarsPrepare and conduct thought leadership webinars, seminars and other events on PCI DSS, log management, SIEM and other security topics (example webinar).TrainingPrepare and conduct customized training on log management, log review processes, logging "best practices," PCI DSS for customers and partners (example training class).Develop advanced training on effective operation and tuning of SIEM and log management tools to complement basic training.End-user Organization / Enterprise ServicesThis section of services menu applies to end-user organizations. The main theme is related to planning and implementing logging, log management and SIEM / SIM / SEM for security and compliance. Log management and Security Information and Event Management (SIEM) product selection - how to pick the right SIEM and logging product?Develop log management or SIEM product selection criteria (related writing)Identify key use cases aligning log management and SIEM tools with business, compliance and security requirementsPrepare RFP documents for SIEM, SEM, SIM or log managementAssist with analyzing RFP responses from SIEM and log management vendorsEvaluate and test log management and SIEM products together with internal IT security teamAdvise on final product selectionLogging and log management policyLogging and log management policy - how to develop the right logging policy? What to log?Develop logging policies and processes for servers and applications , log review procedures, workflows and periodic tasks as well as help architect those to solve organization problemsInterpret regulations and create specific and actionable logging system settings , processes and log review procedures (example: what to log for PCI DSS?)Plan and implement log management architecture to support your business cases; develop specific components such as log data collection, filtering, aggregation, retention, log source configuration as well as reporting, review and validationCustomize industry "best practices" related to logging and log review to fit your environment, help link these practices to business services and regulations (example)Help integrate logging tools and processes into IT and business operationsSIEM and log management product operation optimization - how to get more value out of the tools available?Clarify security, compliance and operational requirementsTune and customize SIEM and log management tools based on requirementsContent developmentDevelop correlation rules, reports and other content to make your SIEM and log management product more useful to you and more applicable to your risk profile and compliance needsCreate and refine policies, procedures and operational practices for logging and log management to satisfy requirements of PCI DSS, HIPAA, NERC, FISMA and other regulationsTraining - how to get your engineers to use the tools best?Provide the customized training on the tools and practices of log management for compliance, IT operations, or security needs (example training conducted)Develop training on effective operation and tuning of SIEM and log management tools to complement basic vendor training.Incident response artifact analysisAnalyze logs and other evidence collected during security incident response