Applying AI/ML in live Cybersecurity environments can be challenging. We share some of our learnings and identify common pitfalls. Bibu Labs is a leading Cybersecurity company leveraging AI to solve complex problems faced by Enterprise clients.

1. tahseen@bibulabs.com

2. Bob

3. 300+
alerts per day
60-75%
noise (false positives)
2-7
days to detect attacks
+ =

4. 300+
alerts per day
60-75%
noise (false positives)
2-7
10Xdevices over next
5 yrs
+ =
days to detect attacks

5. 300+
alerts per day
60-75%
noise (false positives)
2-7
10Xdevices over next
5 yrs
Complexity
of sensor data
+ =
days to detect attacks

7. BibuLabs
7
Hacker
Attacks
Organized
Sophisticated
Targeted
I n c r e a s i n g l y :

8. BibuLabs
8
“Challenges in Applying AI to Cybersecurity”
Tahseen Shabab
Presenter
Large Bank (Canada)
Fortune 60 Telecommunication Firm (US)
Top 3 Financial Services Firm (Canada)
North American Government
More +

9. BibuLabs
9
Bibu Labs Team
Tahseen Shabab
Founder & CEO
Prof. Hassan Khan
Chief Scientist
Prof. Kate Larson
Advisor - AI
Prof. Larry Smith
Advisor - Strategy

10. BibuLabs
10
ACCESSING WATERLOO REGION’S SECURITY ECOSYSTEM
waterlooedc.ca
NOTE: While companies may have a presence in mul�ple categories
of this cluster map, they will only appear in the category that most
directly reflects their business.
CYBERSECURITY
CRYPTOGRAPHY
QUANTUMLAF INC.
FINTECH
Security P2P
BLOCKCHAIN/SECURITY NETWORKS
COMMERCIALIZATION HUBS
01100001 01110100 01100101 01110010 01101100
01101111 01101111 00100111 01110011 00100000
01010011 01100101 01100011 01110101 01110010
01101001 01110100 01111001 00100000 01000101
01100011 01101111 01110011 01111001 01110011
01110100 01100101 01101101 01000100 01100101
01100011 01101111 01100100 01101001 01101110
01100111 00100000 01010111 01100001 01110100
01100101 01110010 01101100 01101111 01101111
00100111 01110011 00100000 01010011 01100101
01100011 01110101 01110010 01101001 01110100
01111001 00100000 01000101 01100011 01101111
01110011 01111001 01110011 01110100 01100101
01101101 01000100 01100101 01100011 01101111
01100100 01101001 01101110 01100111 00100000
RISK ASSESSMENT/THREAT DETECTION
BLOCKCHAIN
RESEARCH LABS AND HUBS
MS2discovery Interdisciplinary
Research Ins�tuteWaterloo Cybersecurity and Privacy Ins�tute
Cryptography, Security, and Privacy Research Group
The Centre for Wireless Communica�ons
Centre for Applied Cryptographic Research
Centre for Computa�onal Mathema�cs in Industry
and Commerce
Waterloo Ar�ficial Intelligence Ins�tute
Waterloo Centre for Automo�ve Research
Communica�ons Security Lab
Waterloo Ins�tute for Nanotechnology
Ins�tute for Quantum Compu�ng
RBC Cybersecurity Lab
Cybersecurity Zone
EMBEDDED SECURITY

11. BibuLabs
11
Enterprise Security

12. BibuLabs
HR
Data Lake
Enterprise Security Simplified
Router
IPS/IDS
End Point
Server
Threat Intel
FW
Decoy
Sensors
SIEM Tool
Attack
Detection
Orchestration
IDS
NAC
Antivirus
FW
Controls
Analysts
APIs
Note: The following is a simplified conceptual diagram

13. BibuLabs
13
Impact
96% 4%19%
Security Solutions (avg) Alerts Not Addressed Alerts Reliable Alerts Investigated
75
Reference: Ponemon Institute

14. BibuLabs
14
Last Line of Defence
Threat Vectors Increasing
Analysts have to constantly keep updated with latest
attack vectors
Deployment of More Sensors with AI
Analysts have to look at individual inference from each sensor
Contextual Knowledge
Analysts have to match their expertise with inference to make decisions

15. BibuLabs
Domain Knowledge Still Required (An Analogy)
Pill Pill.ai
Tool = AI
Surgeon = Cybersecurity
Note: “The following is my opinion” ~ Tahseen Shabab

16. BibuLabs
16
Understanding Tools of the Trade
(Explainable Inference)

17. BibuLabs
White Paper VS Client Impact

18. BibuLabs
18
The Perfect Onboarding
Vendor Provides Expert Analyst
Heavy manual intervention during POC period
Custom Report Curated
Analysts pin point some rare attacks, remove false positives and share report
with client
Clients Suffer After POC
Clients expect product to run by itself after POC period
Image Credit: Hackernoon: How to Attract “Turkers” and Be the Ultimate Mechanical Turk Hero!

19. BibuLabs
19
Lab VS Production Environment
Pill
Lab (Research
Setting)
Production

20. BibuLabs
20
Imbalanced Datasets
~ 0.001% of dataset correlates to hack
Dynamic Environment
Traffic, User Behaviour, Attacker Behaviour
Attack Pattern Not Necessarily Carried Forward
Hackers are getting increasingly targeted
Problems Specific to Cybersecurity

21. BibuLabs
21
Context
Relevance of inference is dependant on context which keeps on changing
Attack Surface
Unique Based on Clients specific IT Environment
Clients Prioritize Attack Vectors specific to risk appetite
Data Quality
Data quality might be the real bottle neck
Challenges With Generic Solutions

22. BibuLabs
22
Red Team VS Data Science Team
Identify Relevant
Attack Vectors
Red Team
Performs Attacks
Data Science Team
Builds Models
1 2
3
Attack Data
Generated
4
Validated Models
Deployed In
Production
5

23. BibuLabs
Cybersecurity - AI Talent?
Cybersecurity AI

24. BibuLabs
Strategic View
AI

25. BibuLabs
HR
Data Lake
Where To Apply AI?
Router
IPS/IDS
End Point
Server
Threat Intel
FW
Decoy
Sensors
SIEM Tool
Attack
Detection
Orchestration
IDS
NAC
Antivirus
FW
Controls
Analysts
APIs
Note: The following is a simplified conceptual diagram
AI (HCI)
AI
AI
AI
AI
AI
AI
AI
AI
AI
AI
AI
AI

26. BibuLabs
26
Adversarial Attacks

27. BibuLabs
27
Hackers take path of least resistance
If a patch has been deployed, hackers will try another route
Adaptive Nature of Hackers (Cat and Mouse Game)
Vulnerability 1
Vulnerability 2
Vulnerability 3

28. BibuLabs
28
Data Distribution Actively
Manipulated

29. BibuLabs
Attack: Data Poisoning

30. BibuLabs
Impact
• Analysts waste time on False Positives
• Illustration*
User Behavior of
Sales Executives
Legitimate deviation from norm
Sophisticated lateral movement
Priority
1. False Positives
2. False Positives
3. False Positives
4. False Positives
5. Sophisticated Attack

31. BibuLabs
web.config
Crown Jewel
Sophisticated
lateral movement
logs
Under The Radar

32. BibuLabs
32
Attack: Induce Specific Output
Add Noise
Classifier Misclassifies Object
Models Learn Differently Than Humans
“Explaining and Harnessing Adversarial Examples”, Ian Goodfellow

33. BibuLabs
33
Attack: Expose Model Attributes
Submit queries, Observe response
- Training Data
- Architecture
"Towards Reverse Engineering Black Box Neural Networks”, Seong Oh
- Optimization Procedures

34. BibuLabs
34
Cost of Error High

35. BibuLabs
35
High Throughput of Data
Analysts Short in Supply
Consequence of missed False Negatives
0.001% Error Rate Could Be Too High

36. BibuLabs
36
Thank You