This document summarizes key findings from Distil Networks' 2017 Bad Bot Report. Some of the main points include: - Bad bots accounted for 19.9% of web traffic in 2016 and target sites with login pages, payment processors, web forms, and pricing information. - Larger sites have a bigger bad bot problem, with bad bot traffic increasing 36.43% year-over-year on large sites. - The US originates more bad bot traffic than all other countries combined, and data centers are responsible for two-thirds of bad bot traffic. - Bad bots lie about their identities and increasingly claim to be mobile browsers. Protecting login pages and blocking old browsers/user agents can help mitigate

1. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
David Monahan
Research Director, Enterprise Management Associates
Twitter: @SecurityMonahan
Distil Networks 2017 Bad Bot Report:
6 High Risk Lessons
for Website Defenders
Rami Essaid
CEO, Distil Networks
Twitter: @ramiessaid

2. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Featured Speakers
David Monahan, Research Director, Risk & Security Management, EMA
David has over 20 years of IT security experience and has organized and
managed both physical and information security programs, including Security and
Network Operations (SOCs and NOCs) for organizations ranging from Fortune
100 companies to local government and small public and private companies.
Rami Essaid, CEO, Distil Networks
Rami is the CEO and co-founder of Distil Networks, the first easy and accurate
way to identify and police malicious website traffic, blocking 99.9% of bad bots
without impacting legitimate users.
With over 15 years in telecommunications, network security, and cloud
infrastructure management, Rami continues to advise enterprise companies
around the world, helping them embrace the cloud to improve their scalability and
reliability while maintaining a high level of security. Follow Rami at @RamiEssaid
Slide 2 © 2017 Enterprise Management Associates, Inc.

3. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Logistics for Today’s Webinar
Slide 3 © 2017 Enterprise Management Associates, Inc.Slide 3 © 2016 Enterprise Management Associates, Inc.
An archived version of the event recording will be
available at www.enterprisemanagement.com
• Log questions in the chat panel located on the lower
left-hand corner of your screen
• Questions will be addressed during the Q&A session
of the event
QUESTIONS
EVENT RECORDING
A PDF of the speaker slides will be distributed
to all attendees
PDF SLIDES

4. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
David Monahan
Research Director of Security and Risk Management
Enterprise Management Associates
dmonahan@emausa.com
@SecurityMonahan
Bad Bot Report:
Six Risky Lessons
for Website Defenders

5. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Bot- a.k.a. “Internet Bot,” “Internet
Robot,” or “Web Robot”
Automated systems using various programs to perform
relatively simple, repetitive tasks on behalf of their owners

6. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Bots are Part of Internet Life
 The web, e-commerce, and bots are here to stay
 Good bots are used by all major web presence
companies:
 Facebook, Google, Microsoft, Yahoo, etc.
 Used to index/manage websites, measure app
performance, and other maintenance tasks
 Bad bots are used by nefarious organizations
worldwide
 Bad bots are created, not born
 Free cloud accounts
 Compromised systems
Slide 6 © 2017 Enterprise Management Associates, Inc.
IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING

7. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
The Good, the Bad, and the Ugly About Bots
 Bots are estimated to be between 40% and 55%
of total Internet traffic
 Bad bots are estimated to be between 19% and
31% of Internet traffic
 Bot control is voluntary without additional
technology
 Robots.txt is the only “integrated” protection method in
html
 Require “tests” or thorough vetting to stop
Slide 7 © 2017 Enterprise Management Associates, Inc.

8. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTINGSlide 8
When Bots Attack (Application and API Flaws)
 Token Cracking
 Carding
 Ad Fraud
 Fingerprinting
 Scalping Obtain
 Expediting
 Credential Cracking
 Credential Stuffing
 CAPTCHA Bypass
 Card Cracking
© 2017 Enterprise Management Associates, Inc.
More at OWASP Automated Threat Handbook
 Scraping
 Cashing Out
 Sniping
 Vulnerability Scanning
 (Distributes) Denial of Service
 Footprinting
 Skewing
 Spamming
 Account Creation
 Account Aggregation

9. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Why Bots can be Tough for Applications to
Detect
 Bots masquerade as users
 Page browsing
 Mouse movement and clicks
 Adaptive content presentation/responses
 Bots masquerading as other devices
 Lies that it is a mobile device
 Lies about its browser engine/version
 Lies about its OS
 Application APIs deliver micro-services,
exposing numerous interfaces to the
Internet
 Net effect: provides opportunity to attack each
micro-service
Slide 9 © 2017 Enterprise Management Associates, Inc.
IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING

10. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Techniques to Stop Application Attacks
 Better application coding practices
 Input filtering
 Safer functions
 HIPS (Human Interactive Proofs)
 (re)CAPTCHA
 Hidden fields
 HOPS (Human Observation Proofs)
 Mouse movement
 Page movement (selection rate, usage patterns)
 Clicks
 Web Application Firewall
 Bots or bad programming – life lesson
Slide 10 © 2017 Enterprise Management Associates, Inc.

11. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Attacks Against Business Logic
Slide 11 © 2017 Enterprise Management Associates, Inc.
 Exploit various facets of operation rather than programming
flaws
 Require a greater understanding of operation than
programming
 No single part of the application or normal Internet filtering has
enough visibility/context
Business logic attacks are not trivial in their consequences and are
successful on even the largest organizations. A few of the large organizations
that fell victim to business logic flaws are Facebook, Nokia, and Vimeo.

12. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
 Modification of authentication flags and privilege escalations
 Business constraint exploitation/modification or business logic
bypass to generate fraudulent transactions
 Requested parameter modification
 Developer’s cookie tampering and business process/logic
bypass
 Exploiting clients’ side business routines embedded in
JavaScript, Flash, or Silverlight
 Identity or profile extraction
 LDAP parameter identification and critical infrastructure
access
Slide 12 © 2017 Enterprise Management Associates, Inc.
IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Attacks Against Business Logic
Examples

13. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Thoughts
Cloud and IoT have done for bots what Paypal and
cryptocurrency has done for ransomware
Bad bots are at epidemic proportions and will continue
expanding if left unchecked
Bot activity will continue to become more invasive and
burdensome to application delivery
Bot sophistication is increasing. Machine learning and AI
will do for bots what they did for malware detection
Automation of Internet attacks will likely have the same
impacts on the hacking industry that it has on other
production line manufacturing (bots replacing humans)
3 © 2017 Enterprise Management Associates, Inc.

14. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Defeating Advanced Bots
Continuous monitoring and prevention are necessary: but with WHAT?!
Must “see” full context
• API and business logic awareness is crucial
• Advanced fingerprinting (sees through the lies)
 More than IP, OS, browser, reputation
 Pull data from client, not rely on push
• Adaptive learning (unsupervised machine learning)
• Behavioral analysis
• Enhanced API authentication
• Dynamic rate limiting to protect API scraping
• Browser validation
Slide 14 © 2017 Enterprise Management Associates, Inc.

15. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Next Steps
Slide 15 © 2017 Enterprise Management Associates, Inc.
IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
• Learn more about bots!
• Take your time in evaluating solutions
• Ask the right questions
(Check out the paper)

16. IT & DATA MANAGEMENT RESEARCH,
INDUSTRY ANALYSIS & CONSULTING
Distil Networks 2017 Bad Bot Report:
6 High Risk Lessons
for Website Defenders
Rami Essaid
CEO, Distil Networks
Twitter: @ramiessaid

17. 2017 Bad Bot Report Methodology
Study based on anonymized data
Hundreds of billions of bot requests
Thousands of domains
Plus 17 global data centers

18. Key Findings
Key Findings

19. Bad Bot, Good Bot, and Human Traffic, 2016
Good
Bots
Humans
Bad Bots
19.9% of Web Traffic Causes Problems

20. The Four Key Website Attributes that Attract Bad Bots
Signup and Login
Payment Processor
Web Forms
Pricing Information
Proprietary Content

21. The Four Attributes By the Numbers

22. The Bad Bot Landscape

23. Size Matters: The Bigger The Site, The Bigger the Bad Bot Problem
Largest sites most
attractive to bad bots
Bad bot traffic on large
sites up 36.43% YOY
Small and tiny sites have
more bots than humans
*Websites grouped by Alexa rank

24. More Bad Bots Than Good on Large and Medium Sites
Small and tiny sites
have more good
bots than bad bots
37.5% more bad
bots than good on
large sites

25. Uncle Sam’s Bot Army
More bad bots come from the US than
the rest of the world...combined
The US originates 5 times more bad bot
traffic than The Netherlands (2nd Place)

26. Countries with the Highest “Bad Bot GDP”
Dominica has 3,348 bad bots per
internet user
Seychelles ranked third, which is
also the alleged home of the owner
of BitTorrent site Pirate Bay
US only 5th on bad bot GDP list
with 446

27. Bad Bots Lie About Their Identity
75.9% of bad bots claim they are
Chrome, Internet Explorer,
Firefox, or Safari
38.61% of bad bots claim they
are Chrome
More bad bots claim to be Safari
Mobile than Safari OSX for the
first time
8% of bad bots claim to be good
bots like search engine crawlers

28. More Bad Bots Claim to Be Mobile
The amount of bad bots claiming to be
mobile browsers jumped 42.78% in 2016

29. Mobile: The Undefended Frontier
9.4% of bad bot traffic
originates from mobile ISPs
T-Mobile and AT&T Wireless
top US based Mobile ISPs for
bad bot traffic
China Mobile third on the list

30. Data Centers are the Biggest Threat
Two out of three bad bots come from a data center
Amazon AWS is responsible for 4x the amount of bad
bot traffic as second place (OVH SAS)

31. Bad Bots Know What They Want

32. You’ve Been Scraped
OWASP AUTOMATED THREAT: SCRAPING
Scraper bot sophistication

33. What Gets Scraped?
Data Scraping Price Scraping
AggregatorsCompetitive Intel

34. Bad Bots Love Login Pages
OWASP AUTOMATED THREATS:
CREDENTIAL CRACKING, CREDENTIAL STUFFING
Account takeover bot sophistication

35. How Credential Stuffing Works
Credential stuffing exploits our
propensity to reuse passwords
across multiple sites.

36. Protecting Your Login Page Is Not Enough

37. Account Based Fraud
OWASP AUTOMATED THREATS:
CARDING, CARD CRACKING, CASHING OUT
Account exploitation bot sophistication

38. Account Takeover Attacks: Why?
Financial fraud
Targets are accounts at financial or
e-commerce services that store
users’ banking details. The attackers
perform unauthorized withdrawal
from bank accounts or fraudulent
transactions using the credit/debit
cards on file.
This includes virtual currency such
as bitcoin, in-game currency, and
rewards programs. This is all worth
real money.
Spam
Spam can appear in any service
feature that accepts user-
generated content, including
discussion forums, direct
messages, and reviews/ratings,
degrading platform integrity and
brand reputation.
Phishing
Attackers can assume a
compromised user’s identity and
launch phishing attacks on
others in his/her social circle to
steal their credentials, personal
information, or sensitive data.

39. Spamming Bots Are Annoying
OWASP AUTOMATED THREAT: SPAMMING
Spamming bot sophistication

40. Application Denial of Service
OWASP AUTOMATED THREAT: DENIAL OF
SERVICE
Denial of service bot
sophistication

41. What’s the Difference Between Application Denial of Service and DDoS?
Application Denial of Service
Attacks the application directly
Hard to spot because it won’t show up
as an anomaly on your firewall and may
not impact load balancer
DDoS
Attacks the ISP hosting your application
Easier to spot because it floods
upstream infrastructure to point where
packets never arrive at the web server

42. All Your Web Analytics Are Wrong
OWASP AUTOMATED THREAT: SKEWING
Sophistication level of bots
that skew analytics

43. Skewed Conversion Tracking
“The number of conversions were
greatly deflated because of bad bot
traffic. Now that we’re filtering bad
bot traffic out, we’re able to see
what the real data is and make
decisions based on real visitors.”
Marty Boos
CIO, StubHub

44. Advice for Web Security Professionals

45. Geofence Your Website from Offending Countries
*Measuring customer block requests by geographical region
China and Russia
accounted for 79.9%
of country-specific
block requests
Dominica, Netherlands,
and Seychelles all
generate more than a
thousand bad bots per
internet user

46. Only Allow Browsers on Your Site
25% of bad bots are simple scripts running
in the command line interface
If you block users that aren’t connecting
with browsers, you will prevent simple bad
bots from attacking your site

47. Block Old User Agents and Browsers
9.45% of bad bots claim to be
browser versions that are 5
years old or older
Blocking old browsers and user
agents will stop bad bots from
reaching your site
The top 10 Oldest Self-Reported Browsers by Bad Bots, 2016

48. Mobile is a Growing Bad Bot Attack Vector
Rate-limit mobile traffic
Consider carefully when IP
blocking within mobile because
it blocks too many real users
Try to generate tokens, in a
secure way, to identify and
rate-limit users

49. Having a login, data, pricing information,
payment processing, and/or forms means you
have bad bots
Take action, don’t just ignore the problem
Don’t do it yourself, because you’ll be stuck in an
endless cycle of IP whack-a-mole
Understand the problem, read the OWASP
handbook on automated threats
Don’t Ignore the Problem

50. What to Look for in a Bot Mitigation Solution
Blocks all automated threats including scraping,
account takeover, spamming, and payment
processor fraud
Uses hi-definition digital fingerprints to ID bad bots,
not just IPs
Enables geofencing from offending nations and ISP
fencing from offending ISPs
Detects scripts, headless browsers, and browser
automation that imitates legitimate users
Applies behavioral analysis using machine learning
Protects APIs

51. Advanced Persistent Bots
APBs
75%
Basic scripts running
in command line
Headless browsers,
more human-like
Browser automation,
most human-like

52. https://resources.distilnetworks.com/whitepapers/2017-bad-bot-report
Download the Report

53. QUESTIONS….COMMENTS
?I N F O @ D I S T I L N E T W O R K S . C O M
OR CALL US ON
1.866.423.0606
www.distilnetworks.com
Thank You for Participating!
To learn more about Distil Networks, visit us at:
http://www.distilnetworks.com
Or contact us at: 415-423-0831