agile, bmnt, business model, corporate innovation, customer development,dod,nsa, diux, h4d, hacking for defense, lean, lean launchpad, lean startup, stanford, steve blank

1. Team Narrative Mind
Stanford University

2. Team Info
Sponsor: US Army Cyber Command (ARCYBER)
The Narrative Mind team contains experts in software engineering, social media design, and
web-based information operations (IO). We seek to develop tools that will optimize discovery
and investigation of communication trends on social media.
Weekly Total: 10 interviews
Users: 14
Experts: 38
Buyers: 4
Cumulative Total: 56 interviews

3. Problem
Statements,
NOT RFPs
Creates
“Innovation
Challenges”
Organizational Chart
CW3 “Fitzgerald”
MAJ “Kramer”
Reports analysis
COL “Tim”
LTC “Zeke”
Director & Deputy SIG
Assesses threats 1-
3yrs out
Proposes
CONOPs
BG Frost
Deputy
Commander Ops
ARCYBER
(Service Component)
NSA
ADM Rogers
Internal ARCYBER office
Program Managers
LTG Cardon
Commanding General
BG (O8-level)
Undergoing Leadership
Change
Contains own cyber
troops that carry out cyber
missions.
JOINT FORCES HQ-CYBER
USCYBERCOM
ADM Rogers
USSTRATCOM
COCOMs
CENTCOM,
AFRICOM,
NORTHCOM
Proposes
concepts
Secretary Carter
ARCYBER
Acquisition
Process
Mr. R. Pontius
Deputy to Commanding
General, Civilian
COL “John”
G3 Operations
Subordinate
Units:
- 1st IO Cmnd.
- 780th Military
Intelligence
Brigade
- NETCOM
Fully-staffed Joint
Force IO/SIGINT
troops.
Provides
specialized
analysis on ad
hoc basis.
Proposes
concepts
to be
implemented
C5 Consortium
President Charlie McBride
Board of Directors
Companies/Academics (by application)
Traditional DoD
Contractors
Non-traditional
Contractors
DIUX
Develops relationships
with commercial sector.
Submit White Papers
& Demos
Necessary
Partnership
Started 2015
PEO-EIS
ACT
Advanced
Concepts and
Technology
Directorate
Manages
CYBER
capabilities/
requirements.

4. Other Transactional Authorities
Key Objectives of OTAs:
● Reduce barriers to entry.
● Incentivize commercial
participation through expedited
awards and negotiable IP
terms.
Necessary Conditions of OTAs:
● Prototype projects only.
● Enhances mission effectiveness.
● Costs are fair/reasonable.
At least one criterion must be met:
● All significant participants are
small businesses or non-traditional
defense contractors.
● Non-traditional defense contractor
participates to a significant extent.
● One-third cost share for traditional
defense contractor
Example of Actual Cyber OTA Micro Cloud Computing System
OTA Timeline

5. All the Hypotheses
Global
Tweet-level
Awareness Response
Adversary IO Org Chart
No baseline for monitoring/aggregating
adversary use of tech; cyber targeting
Automatic Narrative Detection
Language/culture experts aren’t able
to work at scale
Virality Predictor
Understand which narratives are the most salient
Important Event Predictor
Preempt real world events
Counter-Narrative Generator
DoD/Gvt. social media
presence is weak
Persistent ID-Alias tracker
Account bans and multiple aliases across
different networks make IDs hard to track
Poison Link Generator
Monitor who views
adversary dark-web
content
Filesharing Site Scraper
Need more cached information
access
Expedited Content Categorization
Scale of social media makes manual efforts
painful
Bot Detector
Can’t determine actual scale of
adversary following and support.

6. Hypotheses ❏ Three previously proposed narrative units would be valuable to track.
- Co-occurring hashtags
- Ranked list of engaged content/tweets over time
- Geospatial sentiment maps
Experiments ❏ Ask users to define their own narrative units.
❏ Ask users to rank the salience of different narrative units.
❏ Individual trawling around twitter feeds of presidential candidates.
Results ❏ A ranked list of most popular content over time is the most valuable, along with information
on co-occurring hashtags.
❏ Sentiment data is not as helpful because it tends to be inaccurate.
❏ There are tools that exist to do any one of these things, but they are not stitched together for
easy use by operators.
Actions 1. Analyze political campaign dataset for hashtag co-occurrence frequency.
2. Discuss refined MVP and continue to develop new narrative units.
3. Research communication link between awareness and response groups.
Customer Discovery

7. Responses to Narrative Units Presented Previously
A ranked list of
most popular
content over time
A list of
hashtags that
appear with the
same message
with extreme
frequency.
Positive or negative opinion among
geographic or demographic
population.
Narrative Type #1
Narrative Type #2
Narrative Type #3
E.g
Popularitywithusers
Sentiment analysis is thought
to be too crude to provide
useful input. May pollute
signal-noise.
Unit #2 “popular content”
(commonly described as
media like photos or videos)
seemed to be the most
credible. Unit #1 co-occurring
hashtags might provide more
granularity and context than
just single hashtag trends.

8. MVP - 3 Types of “Narratives” Outputted by Our System
A list of hashtags that
appear with the same
message with extreme
frequency.
Positive or negative
opinion among geographic
or demographic
population.
Narrative Type #1 Narrative Type #2
Narrative Type #3
Displayed
to Analysts
Analysts associate
each “narrative unit”
with a real-world,
human-readable
description of a
narrative.
Gain creator: Software automatically
tracks activity of a real-world narratives by
tracking the activity of associated “units”.
Real World Narrative
Long-form, unstructured human description or
collection of info.
A ranked list of most
popular multi-media
content over time
Pain reliever: Social media firehoses
scanned for list of narrative units that match
predefined “schemas”, ranked by frequency.
Problems with this MVP:
● What are the right “units”?
● How is a narrative’s overall growth
calculated from the multiple units with
different sizes? (e.g. relative
“weights?)
Benefits of this MVP:
● Aside from schema, specific units
don’t need to be defined a priori.
● Model allows retroactive reindexing of
a single narrative’s growth based on
new associations.
● Builds a “training set” for more
sophisticated efforts.

9. Mission Model Canvas
- Understand viral potential of
social media posts in real-time.
- Track how groups use
technology over time.
- Gnip/Twitter/Facebook
- CrowdFlower,
Samasource, or
Mechanical Turk
- Pre-existing social
media service and micro-
labor aggregators
- Third-party access
platforms for social
media
-Data visualization (Zoic)
-Content analysis
platforms (Sens.ai,
Leidos)
Primary
ARCYBER
-Bg. General (decision
maker)
-MAJ/LTC/COL
(operational plan)
-Analysts/Operators
(actionable insights)
COCOMs
-General (decision
maker)
-MAJ/LTC/COL
-Analyst/Operator
Secondary
Political Campaigns
-Campaign managers
-Supporters
Consumer Brands
-CMO
-Public Relations Team
- Optimize workflow for
social media analysts.
-Deliver insights to
commanders about
online environment.
-Insights into responses
against specific
broadcasting narratives
-Detect opposition
narratives emerging in
real time
-Early warning on
emerging brand threats
- Customized UI
- Testing with analysts
-Enable faster problem awareness to problem response times for
decision-makers across organizations
- MechanicalTurk or crowdsourcing labor (microtasks)
- UI Development/Testing with ARCYBER analysts.
- Software Development
- Research aggregation? (e.g. Import and reformat existing
knowledge about extremist organizational structure)
- Access to Twitter
firehose or API
- Local language
speaking crowdsourcing
staff.
- Accurate testing for
intercoder reliability
- ARCYBER: Bg. General,
LTC, Strategic Initiatives
Group, OTA, Purchasing PMs,
End-User operator)
-COCOMs: OTA, Operators,
Purchasing PMs
-Political Campaign:
Opposition research team, ???
-Private Sector: CMO, ???
Beneficiaries
Mission AchievementMission Budget/Costs
Buy-In/Support
Deployment
Value
Proposition
Key Activities
Key Resources
Key Partners

10. Value Proposition Canvas - ARCYBER
Products
& Services
Web/Desktop
Application Update research
Customer
Jobs
- New mission priorities.
- Little familiarity with overall
environment.
- No repository for high-level
learnings
Gains
Pains
Gain
Creators
Pain
Relievers
- New opportunities for cyber-
ops
- Social network graph?
-Illustrates evolution of
networks
- Provides insight for
mapping “online terrain”
Visualize information about
how organizations are
using social media
ARCYBER - Operator/Analyst (Actionable Insights)

11. Value Proposition Canvas - ARCYBER
Products
& Services
Web/Desktop
Application
Aggregate insights
from analysts,
compile into plan
passed to decision
maker
- New mission priorities.
- New social media platforms
- Ability to prioritize focus
Customer
Jobs
Gains
Pains
Gain
Creators
Pain
Relievers
-Faster turnaround time with
better data gained from
information awareness software
and technology
- Clear hierarchy of
priorities
ARCYBER - MAJ/LTC/COL (Operational Plans)
-Better data in informing
operational plans provided
to decision makers
-Reducing uncertainty in
operational planning

12. Value Proposition Canvas - ARCYBER
Products
& Services
Web/Desktop
Application
Act on reports and
plans generated
about various cyber
issues across globe
- Little understanding of
ground-level nuances of
challenges
- Little understanding of
proliferating platforms
Customer
Jobs
Gains
Pains
Gain
Creators
Pain
Relievers
- Organizational chart of
technology broken down by
functional purpose
- Timeline-based viewer
- Social network graph?
-Reduce uncertainty of
decision making
-Add methodological rigor
to ARCYBER’s operations
- Accomplish mission
objectives
ARCYBER - Brigadier General (Decision Maker)

13. Value Proposition Canvas - COCOM
Products
& Services
Web/Desktop
Application
Target audience
analysis, finding
opportunities to
optimize influence
operations
Customer
Jobs
-Loss of visibility on key
communicators
- No repository for high-level
learnings
Gains
Pains
Gain
Creators
Pain
Relievers
-Auto-populated list & real-time
visualization
-Illustrates evolution of info
dissemination networks
- Provides insight for mapping
“online terrain”
Closure of visibility gap
COCOM - Operator/Analyst (Actionable Insights)

14. Value Proposition Canvas - COCOM
Products
& Services
Web/Desktop
Application
-Long term
operational planning
- Measuring
effectiveness
Customer
Jobs
- Unclear picture of how
networks evolve, adapt over
time
Gains
Pains
Gain
Creators
Pain
Relievers
- Organizational chart of
technology broken down by
functional purpose
- Visualization of dissemination
networks over time
-Illustrates evolution of info
dissemination networks
- Provides insight for
measuring operations,
mapping “online terrain”
Greater awareness of
network evolution improves
planning, indicators of
operational effectiveness
COCOM - Maj/LTC/Col. (O4-O6) (Operational Plans)

15. Value Proposition Canvas - COCOM
Products
& Services
Web/Desktop
Application
- Judge potential
effect and risk of
operations
- Approve/reject
plans
Customer
Jobs
- Less familiarity with
internet as an operational
environment
- Needs high level summary
of technology use
Gains
Pains
Gain
Creators
Pain
Relievers
- Organizational chart of
technology broken down by
functional purpose
- Timeline-based viewer
-Illustrates evolution of info
dissemination networks
- Provides insight for
mapping “online terrain”
Visualize high-level
information about use
of technology.
COCOM - Brigadier General (O7) (Decision Maker)

16. Value Proposition Canvas - Political Campaign
Products
& Services
Web/Desktop
Application
-Identify opposition
narratives on SM
channels
-Craft responses to
respond w/short
turnaround
Customer
Jobs
-Limited lead time in
anticipating threats from
opposition campaigns
-New stories/themes can
derail campaign momentum
without a targeted response
Gains
Pains
Gain
Creators
Pain
Relievers
-Faster response time to
opposition’s campaigns
-Maintain the initiative in media
environment by rapidly
responding to threats
-Early detection of
emerging opposition
narratives on SM
-Insight into evolving threat
environment
Counter opposition
themes quickly, identify
newly spreading
rumors that could
undermine campaign.
Political Campaigns - Opposition Analysis/Campaign Director

17. Value Proposition Canvas - Consumer Product Company
Products
& Services
Web/Desktop
Application
-Monitor brand
integrity and
customer themes
-Identify and address
emerging problems
Customer
Jobs
-Identify themes about
corporate brand, pinpoint
problem areas early
-Avoid negative narratives
from becoming costly PR or
marketing problems
Gains
Pains
Gain
Creators
Pain
Relievers
-List of customer narratives
about brand that are being
posted online
-Proactive response to
emerging brand threats, better
customer satisfaction
-Early warning about
negative themes
-Identify problem areas,
protect corporate brand
from costly problems
-Protect brand integrity,
create greater
shareholder value by
addressing customer
concerns
Consumer Product Company - CMO/Marketing Organization