Chat bots been have popping up everywhere for silly things, but what if they can help us make the world more safe and secure? The work of designing secure systems often involves iterating over designs with a team but what if you don’t have a team? What if you could iterate over system design and analysis in a chat window and have a design document with safety constraints as the end product? This talk will present an original chat bot that will do just that

1. January 15, 2017
SAFETY BOT
GUARANTEED
RICH SEYMOUR

2. Who Am I?
 Data Scientist at Endgame
 Got a PhD in Materials Science doing
nanoscale simulations on supercomputers
 Play bass in a band (bleach bones)
 @rseymour on twitter
2

3. Outline
 Bots
 Safety
 Their powers combined!
3

4. Outline
 Bots
 Safety
 Their powers combined!
4

5. BOTS
the hype cycle peaks?

6. Talking about chatbots
 Image and many of these bot
slides courtesy of bot master
and coworker:
 Bobby Filar @filar
6

7. Popular Bots
 Amazon – Alexa
 Apple - Siri
 Facebook - M
 Google - Google Home
 Microsoft – Cortana
7

8. Bots are Changing Lives
8

9. Bots are Changing Lives Pt 2
9

10. Bots Boiled Down
 Bots take words and turn them into actions.
 “Alexa, turn on the lamp” “Ok”
 “Open the pod bay doors HAL”
10

11. Voice Recognition and Speech Synthesis
 We don’t need
these pieces to
have a bot
11
Text to
Speech
bot
Speech
to Text

12. Click-Through vs. Conversational Interfaces
12
+ 6 more steps
vs
Id like to order a cheese pizza.
Great! What size pizza would
you like?
Medium
Okay. I have an order for a
medium cheese pizza. Is that
all?
Yes. That is all.

13. Causes of the Bot Revolution?
 Platform Diversity
• Messaging
• Intelligent assistants
• Team collaboration software
 App Stagnation
• More users on messaging platforms
• Hard to gain traction w/in App Stores
 Ease-of-Use
• Simplifies UI/UX
• Allows user to maintain focus on current
screen/activity
• Provide walkthroughs
• Eliminates need to know query languages
13
Cortana

14. Bot Development Kits (BDKs)
 The rise of bots is largely due to an increase in BDKs
 Companies like Chatfuel, Wit.ai, and API.ai
• Provide simple UI for development
• Closed-domain, Rule-based, goal-oriented bots
• No programming skills required
 How popular have these frameworks become?
• Wit.ai  purchased by Facebook (Jan 2015)
• API.ai  purchased by Google (Sep 2016)
 Major companies pushing out frameworks too!
• Microsoft Bot Framework
• Amazon Alexa Skills Kit
14

15. What bots are made of.

16. GENERATIVE RETRIEVAL
16
OPEN DOMAIN CLOSED DOMAIN
LONG CONVERSATION SHORT CONVERSATION
Types of Bots
Active Research Surprisingly Easy

17. GOOD BOTS
 DoNotPay Bot
• Bot helps people fight
parking tickets
• Wildly successfully in 3
major cities
 Esterbot
• QA bot tied to a LinkedIn
account or resume.
• Recruiters talk to a bot
before they get to you
BAD BOTS
 Microsoft TayBot
• Great idea gone terribly
awry
• Learning from internet
conversations sometimes
leads to being…
the worst.
17
Good Bots and Bad Bots

18. 18
GENERATIVE RETRIEVAL
PROS
• Bots of the future!
• Excellent recall ability
• Ability to learn as exposure to
user increases
• Simple to implement
• Smaller datasets
• No grammatical mistakes
CONS
• Requires a lot of data
• Complex models  large
vocabularies  massive training
time for model building
• Prone to grammatical errors
• Really, really inappropriate…
right TayBot?
• Not intelligent
• Limited “memory”
• Works best in closed-domain, short
conversation scenarios
Which is best?

19. Bot Workflow aka Bot Vocabulary
 Session
 Identity
 Dialogue Script
 Context
 Intents
 Entities
 Action
19

20. Pattern Matching without Context (Eliza 1964-1966)
20

21. Context & Intent
 Context (teal)
• Current state
• Steers conversation
• Ensures all req’d parameters are
collected to perform Action
 Intent (red)
• Maps Utterance (blue)  specific
Action
• What does the user want?
21

22. Entities
 Entities are parameters needed to
satisfy Intents and execute
Actions.
 Entities can be anything
• File hash
• IP address
• Cities
 Permutations of an entity allow for
diversity of vocabulary
• Regular Expressions
• Synonyms
22

23. Training Data
 Few bot developers have necessary
training data
 BDKs provide access to Domains or
pre-defined Knowledge packages:
• Reservations
• Small Talk
• Weather
 BDKs log user interactions to constantly
update training set
23

24. Natural Language Processing & Machine Learning
 BDKs have built in ML/NLP capabilities
 NLP used to handle incoming data
• Tokenization
• Word Boundary & Word Sense
• Synonym generation
• Entity extraction
 ML used to classify user input to an Intent and map
to an output
• Supervised Learning
• Features: n-gram or word-based vectors
• Model constantly updated as bot gets used
24
Input  NLP
Output  ML
features

25. NLP 101  Direct Matching (No ML)
Show flights from Washington to Denver on Monday
Show flights from Washington to Denver on Monday
Show flights from Washington to Denver on Monday
VB NNS IN NNP IN NNP IN NNP
if utterance contains show_vb AND flights_nns:
Utterance
Tokenization
Parts-of-
Speech
Tagging
(POS)
Show flights from Washington to Denver on MondayNamed Entity
Recognition
(NER) Location Location Date
SHOW_FLIGHTS(Washington, Denver, Monday)
Perform Action
25

26. NLP 101  ML-Based
Show flights from Washington to Denver on Monday
Show flights from Washington to Denver on Monday
Utterance
Tokenization
Show flights from Washington to Denver on Monday
Named Entity
Recognition
(NER) Location Location Date
0 1 1 0 1 0 1 0 1 1 0 0 1Bag-of-Words
(Feature Vector)
SHOW_FLIGHTS
CHANGE_FLIGHTS
BOOK_FLIGHTS
Intent
Classification SHOW_FLIGHTS(location, location, date)
SHOW_FLIGHTS(Washington, Denver, Monday)Perform Action
26

27. Getting started with this Language Stuff
I recommend spaCy available
at spacy.io
27

28. How can bots help Security?
Some Ideas

29. Guidance and Training
29
Is the command line a chatbot?

30. Guidance and Training
30
disown

31. Guidance and Training
31

32. Search
32

33. Search (but not security twitter, let’s say your logs)
33
 “What can I help you with today?”
 “Show me all of the processes running that started in the last hour”
 “Ok which endpoints?”

34. Search (but not security twitter, let’s say your logs)
34
 “What can I help you with today?”
 “Show me all of the processes running that started in the last hour”
 “Ok which endpoints?”

35. Search (but not security twitter, let’s say your logs)
35
 “What can I help you with today?”
 “Show me all of the processes running that started in the last hour”
 “Ok which endpoints?”
 “Everyone!”
 “Ok”
 Search results ensue

36. Question and Answer Bots
36

37. Q&A Bots
37

38. Chat Ops and Checklists
38
 Hubot – GitHub *
 ScoreBot - Netflix
* https://www.youtube.com/watch?v=NST3u-GjjFw

39. Chat Ops and Checklists
39
Ignorance
Ineptitude

40. Chat Ops and Checklists
40

41. Chat Ops and Checklists
41
But checklists are
the worst!

42. Chat Ops and Checklists
42
Well maybe bake
the checklists into
the bots?

43. Outline
 Bots
 Safety (get shmoo balls out)
 How To!
43

44. Safety

45. Safety!
45

46. Just what is Safety?
How is Safety different from
Security?
46

47. Just what is Safety?
How is Safety different from
Security?
Safety generally means
preventing people getting hurt
47

48. Just what is Safety?
Safety ideas have historically
been borrowed by Security.
48

49. Domino Theory -- Heinrich 1931
49

50. Domino Theory -- Heinrich 1931
50
Bigotry
included

51. Others have followed
51
 Root Cause Analysis
 Fault Tree Analysis
 Swiss-cheese model

52. Engineering a Safer World
52
Engineering a Safer World
Systems Thinking Applied to Safety
Nancy G. Leveson
2012
Systems Theoretic Process Analysis
(STPA)

53. Engineering a Safer World
53
Engineering a Safer World
Systems Thinking Applied to Safety
Nancy G. Leveson
2012
Systems Theoretic Process Analysis
(STPA)

54. Systems Thinking Applied to Safety
54
Bertalanffy
Wiener

55. Systems Thinking Applied to Safety
55
Bertalanffy – Systems Theory
Wiener - Cybernetics

56. Safety Defined
56
 Hazard: a system state or set of
conditions that combined with worst case
environmental conditions will lead to an
accident
 Accident: an event that results in an
unacceptable level of loss
 Safety: freedom from accidents

57. The Driving Force of Systems Safety
57
 Accidents today happen while
the each component of the
system is working

58. The Driving Force of Systems Safety
58
 The accidents we see now are:
 Component Interaction
Accidents

59. The Driving Force (continued)
59
 “Preventing future accidents requires
shifting from a focus on preventing
failures to the broader goal of
designing and implementing controls
that will enforce the necessary
constraints.”

60. What it doesn’t do
60
 It doesn’t use Risk %s
 It doesn’t catalog possible failure
 It doesn’t blame operator error
 It doesn’t triage

61. Purpose
61
 What does your system do?

62. Losses
62
 Unacceptable.
 Not component failures.
 Real. Bad. Stuff.

63. Hazards
63
 What could make you suffer a
loss?
 Threats?

64. Hazards
64
 What could make you suffer a
loss?
 Threats?
 Check out William Young’s
STPA-Sec work

65. Constraints
65
 How can we stop hazards from
becoming losses.
 How can these in turn be
violated. (iterate)

66. The minimal system
66

67. The minimal system
67
Yup, just
boxes
and
arrows!

68. 68
Nancy Leveson’s
outline of possible
“control flaws leading
to hazards”
Engineering a Safer
World
p 93

69. Very High Level
69

70. Maybe this is a useful process?
70

71. Maybe this is a useful process?
71

72. STPA takes a high level view, early in the design
72
 What if we had a bot to help us
go through this process?

73. Outline
 Bots
 Safety
 How To! (SafetyBot!)
73

74. SafetyBot

75. STPA takes a high level view, early in the design
75
 What if we had a bot to help us
go through this process?
 What if you had ~ 2 weeks to
write such a bot!

76. GENERATIVE RETRIEVAL
76
OPEN DOMAIN CLOSED DOMAIN
LONG CONVERSATION SHORT CONVERSATION
Types of Bots
Active Research Surprisingly Easy

77. Choose a language you’ve never used!
77

78. SafetyBot
78
context
Context
manager
Node
entity
Node
entity
context
Node
entity
Current
context
Next
context
The context manager is the main
loop
Prints out bot talk and pulls in
user input.
That is then passed to the current
‘node’ for processing / storage

79. SafetyBot
Live Demo

80. Thanks thanks thanks thanks thanks!
80
Questions?