The document outlines plans for the HSCB Pathways Program, which aims to develop a hybrid modeling engine for navigating socio-cultural topology. It discusses how the initial phase of HSCB involved diverse modeling projects, and how Phase 2 requires rationalizing these efforts and establishing common frameworks. The Pathways Program was established to meet this need by integrating data and models to enable analysts to explore "what if" and "what is" questions about the human and cultural aspects of operational environments. The document outlines the technical objectives and challenges of building a system that can assemble relevant models and data to support operational decision-making.
2. FOR OFFICIAL USE ONLY
FOR OFFICIAL USE ONLY
HSCB & The Need for Frameworks
• HSCB Phase 1: Shotgun of projects. All about models… “Seeds
in the field & Let’s see what grows” – Showcased at “Focus
2010”
• HSCB Phase 2: Need to rationalize & create foci for rapid
protyping into operational capabilities by FY12
– Define a common “framework” to ensure that models and data will
come together as needed
– Must be supportable & transitionable within DoD PORs
– Address urgent operational needs in a repeatable manner
– Form basis for “composable” modeling
– Serve as catalyst for next generation S-C modeling
– 6.3 / 6.4 Foci given anticipated HSCB Funding profile
• CTTSO releases HSCB BAA 09-Q-4590 in May 2009
2
3. CTTSO HSCB BAA 09-Q-4590
• Goal: Build and demonstrate end-to-end HSCB functional
capabilities:
– Enable an analyst to translate an operational requirement to into a
analytic strategy given available data & models
– Execute best available models against best available data (individually
& as hybrid / composites) to perform analyses
– Visualize and share results, source data & models in a consistent
manner amenable to supporting command / tactical decision-making
related to understand stability and threats in regions and develop
appropriate course of action
• Semi-automatically manage data, condition and load it into
appropriate models
• Provide a basis for an analyst to find and link together models
with data to develop a usable product that supports
operational decision making (i.e. enable Hybrid-Modeling).
FOR OFFICIAL USE ONLY
FOR OFFICIAL USE ONLY
3
4. FOR OFFICIAL USE ONLY
FOR OFFICIAL USE ONLY
Road to Pathways
BAA 09-Q-4590
Requirements
• 2531 – HSCB Modeling Decision Support Framework
• 2532 – HSCB Dataset Repository and Management System
• 2533 – HSCB Data Translation and Brokering System
• 2534 – HSCB Modeling Visualization Framework
114 White
Papers
• Papers evenly distributed across 4 requirements
• Source Selection Team recommends 13 papers for full proposals
• 7 Full Proposals selected for FY10 Funding
Pathways
Program
• Proposals for Req’ts 2531, 2532, 2533 combined into 2 single contracts, creating two teams:
• Team 1: Lockheed Martin ATL / Lockheed Martin ISGS - “Nexus”
• Team 2: BAE / BBN – “Prism”
• Oculus selected as sole performer for 2534 due to funding – “Aperture”
• Anticipatory Funding Authorized 26 April 2010 for the 3 Pathways contracts
• Contract awards to be complete by Aug 2010
• Formal Kick-Off planned for 28 July 2010
4
6. 6
MG Flynn, USA, J2 ISAF;
“Fixing Intel” - Jan 2010
“Our operators must find better ways to answer
fundamental questions about the environment in which we
operate and the people we are trying to protect and
persuade.”
7. HSCB Environment:
Socio-Cultural (S-C) Topology questions are diverse
Data Driven Methods
Model Driven Methods
“What is the village’s
sentiment toward
US? Has it changed
since the 2 new
schools were built?”
Individuals and
Small groups
Regional
Populations
General Population,
Government Institutions.
“What is the local
population’s attitude
toward the insurgents? Is
the population ready to
marginalize the
insurgency movement,
especially in the south.
“What are the key
factors that drive
popular support for
the insurgency
verses that for the
government?”
“What if local and
district groups were
empowered to
define the rule of
law and justice?”
“What if acceptable
stability for the
country is not
achieved in the
next 6 months?
“What if new
economic initiatives
are implemented in
the southern
provinces?
What if
Questions
What is
Questions
7
8. S-C Data Challenges
Diverse and Dynamic Data Variety of Models
Today’s Limits: Few extant methods and standards for joining information, analyses, and
forecasts of this breadth, volume, and variability. Real World Data
Structured tables
Unstructured text from reports
Blogs
Imagery
Chat rooms
Geo-spatial data
Dynamic,
Theory-based
Impact if Pathways is successful: Enhanced mission performance through easier
data organization and access, and by making analyses, and forecasts easier to
assemble and use
Descriptive,
Statistical
Pathways Modeling Engine
8
9. Pathways Objective Capability:
Socio-Cultural Navigation
Data Driven Methods
Hypothesis Driven
Methods
Models for
forward base
Models for
humanitarian
response
Models for Division
HQ to determining
tipping points
Socio-Cultural
Topology
Socio-Cultural Data
What is the Status?
What if this occurred; or action was taken?
Modeling Challenges:
1. Develop Models for the Mapping S-C topology
2. Develop Models for Navigating the topology
3. Develop Models for forecasting plausible futures
9
10. Develop a new hybrid modeling engine
for navigating the Socio-Cultural Topology.
Pathways Technical Objective
• Address the full spectrum of diverse, military, S-C needs as
they emerge
Enable the tailored and rapid assembly of models with best available
data
Enable the discovery and adaption of data to meet emergent
operational needs.
Enable better understanding the operational S-C environments.
Support exploring fundamental “what if” and “what is” questions.
Provide a common user interface to enable access to hybrid models &
data - with interactive visualizations that support S-C situation
awareness and commanders’ option assessment.
10
11. Each map may have its own topology,
coordinate/attribute scheme.
Socio-Culture Topology:
(n-Dimensional State Space)
- Geospatial & Temporal
- Entities, e.g.
- People (individuals)
- Groups
- Institutions
-Events
-Economics & Security
-Resources (& movement)
-Attitudes, Values & Influences,
and trends
x0
S0
Path1
Path2
State Data
y0
Data not
subject to
influence
Data that
can be
influenced
CollectibleDerived
State
Data
Socio-Culture Topology & Requirements
2. Model characterization to aid
discovery and integration into
different applications, i.e., use
State Data to generate S-C maps
(typically data-driven models in
this step)
3. Build Architecture for joining
heterogeneous collections of
models; with quick addition of new
models, to create COA models
4. Discovery of actionable
factors within the models
that influence any given
outputs
5. Visualization of threads that
link data, models, and analysis
to increase model/decision
transparency
1. Enable data
organization via
meta-tagging
Key Technology
Requirements:
11
12. Modeling Conceptual Challenge
“How do we know we are getting the right answers - not just
getting the wrong answers faster?”
• Better question: How do we make sure we are getting useful
results from hybrid models?
1. Develop Process Theories for how Hybrid Models will be used.
• Different models may be applicable to different analysts at different
echelons & with different problems.
2. Expose model & data at the appropriate levels to be meaningful to
S-C Theorists & Analysts
“Essentially, all models are wrong – but some are useful.”,
(George E. P. Box)
12
13. Meta-Theory for S-C Analysis
• Object:
1. Manage the S-C Modeler / Analyst Dialog as a repeatable Process.
2. Demonstrate that we can get better answers through Hybrid Modeling.
• Challenge: Start with needs based on what an analyst
does today: Define a process theory that will be
meaningful to S-C Hybrid Modeling
– Show what can & needs to be done
– Provide a System Diagram & narrative addressing Process
• Map out and expose an S-C Topology
• Develop Courses of Action given the S-C Topology
– Identify Strengths & Weaknesses
• Describe your plan to address weaknesses
• Define Metrics for assessment
– Propose program assessment protocols & metrics
13
14. A Meta-Model for the Analytic Process
TIME/EFFORT
Shoebox
Evidence
File
Search &
Filter
Read &
Extract
Schematize
Build
Case
Tell Story
Search for
Information
Search for
Relations
Search for
Evidence
Search for
Support
Reevaluate
STRUCTURE
Schemas
Hypotheses
Presentation
External
Data
Sources
Sense Making Loop
Foraging Loop
Courtesy of PARC (2007)
14
15. Notional Meta-Theory Objectives
Tell us where you think we will end up*
• Tells us what model decomposition is appropriate &
meaningful to end users.
• Tells us about data
– What sort of data is needed & help maps what data is available
– Tells the form(s) of data should be to answer need
• Suggests an “evidence calculus”;
– What we need to know from the models
– What we do / don’t know
– Necessary assumptions
– Hypotheses that need to be explored
– ID schemas to tell the story
– ID the Inputs/Outputs of the model to address the operational
questions
• *Subject to change without notice
15
16. Assemble:
Data, Causal &
Correlation
Relationships
Identify &
Assemble Hybrid
Models; Measure
Fit (for analysis
topics)
Generate Projections, i.e.,
Courses of Action
Assess evidence with respect to
the projections.
Deduce and generate:
-Additional data to improve fit, or
-Next question & supporting data
Situational:
Economics
Food
Medical
Media
Saturation
Stability
Public Works
Perceptual, I.e.,
How
Public supports:
OPFOR
Blue Force
Local Govt.
Other
Authorities
Evidence Calculus
“If we have these
observations , what
weight of evidence to
we accord to each
projected path?”
Policy/Treatment Question
S0
Path1
Path2
S1
Path1
Path2
S2
Path1
Path2
Analysis
Topic 1
Analysis
Topic 2
Analysis
Topic 3
17. Pathways Way-Ahead
Interactive Visualization tools
Brigade & Above
Strategic Influence for
Sudan, Congo &
Horn of Africa
Model- and Data-Driven
Applications
New Generation
Modeling System
Technology Base for Model and
Information System Development
1. Select
Exemplars that
Span Diverse
Applications
2. Identify Hybrid Models
that should Address the
Exemplars
3. Develop New
Modeling Integration
Framework joining Data-
and Model-driven uses
Battalion & Below
Stability Operations for
Provincial Reconstruction
Team in Afghanistan
Tools and Techniques
for Non-obvious
relationships
4. Integration
of point
solutions when
& where
appropriate
17
SA/OA
18. Pathways’ Key Milestones
Milestone 1 Milestone 3Milestone 2 Milestone 4
Establish
Static
System
Dynamic
Model,
Data
Selection
Aids to
Automate,
Guide
Analysis
Testing,
Transition
Technology Integration
Experiments (TIE)
TIE
#1
TIE
#2
TIE
#3
18
19. Pathways Program Overview
Phase 1 Phase 2 Phase 3 Phase 4 Transition
Establish Static
Framework &
Baseline Capability
Dynamic Model &
Data Selection
Demonstrate Data Discovery &
Model Selection for Operational
Decision Support
Mission Specific
Operational
& Transition Demos
Deploy &
Sustainment
Focus:
• Integrate models &
data into framework
• Define & Implement
S-C Meta-Model
• Define operational
decision support
requirements for
users
Focus:
• Integrate models on
demand
• Demonstrate use of
theoretically derived
S-C meta-models
• Code & manage data
in multiple security
enclaves
• Decision support for
model selection &
composition
Focus:
•End-to-end Capability
•Analyst & Planners able to take
command challenge problem & offer
valid, model-based solutions that
support command decision-making.
•Demonstrated utility & internal
validation of S-C models at meta-model
level
Focus:
•Prepare for Transition
•Documentation &
Training
•POR Integration
•In-Theater
Operational use
testing & Utility
assessment
Focus:
•Tools sustained via
Programs Of Record
Metrics:
• Demonstrate
repository mapping
of Open Source Data
• Define S-C Auto-
Tagging of structured
data
• Baseline models
show theoretically
derived
dependencies
interaction
• Demo Model
integration using
fixed data & models
Metrics:
• 200% reduction in
time for trend analysis
• 82% Meta-Data
Coding Accuracy
• Dynamically Map new
data source into
frame work <8
Person-Hours.
• Planner able to
compose models to
address requirement
<8 Person-Hours
Metrics:
• Automatic detection of significant
changes in S-C Topology.
• Models optimized and support
assessment of COA < 8 Person-
Hours.
• “Shrink Wrap” solutions for 3 mission
areas
• 92% Coding Accuracy
• Automatic anomaly detection from
open source data
• System guided drill down into model
pedigree & data for Analyst SA &
Command decision-maker OA.
Metrics:
• Models adapted for
use as tools to
support 3 operational
missions at 2
different echelons
• Able to exploit live
data in theater < 6
Hours
6 Mo. 8 Mo. 18 Months 12 Months 16Months
19
20. 20
Pathways: Evidence of Success
From: To:
Arbitrary technical approaches
guiding tool development
Combine Empirical and
Theory Driven Approaches
Interactive Visualization
Static views with no visibility
into inner workings
Hybrid Modeling Engine
Supporting “mash-up” with
data & tailoring on demand
One-off solutions with hard-
wired models & data
End-to-End SystemModel Components
User Composeable -
Scaleable Framework
Custom, brittle,
implementations
Hybrid Modeling for Navigating the Socio-Cultural Topology
21. 21
Combating Terrorism Technical Support Office, (CTTSO)
Human Social, Culture & Behavior Modeling Program (HSCB)
Questions?
HSCB Pathways Program
Defining a Navigation Framework
for Socio-Cultural Topology
Jeffrey G. Morrison, Ph.D
morrisonj@tswg.gov
703.604.0339
23. FOR OFFICIAL USE ONLY
FOR OFFICIAL USE ONLY
Initial Insights, Concerns & Caveats
• Insights & Concerns:
– Pathways needs to define Challenge Problems (by program
phase).
• Clear, Compelling, Operationally Relevant
• Framed in a suitable context
– S-C Theory derived meta-models
– Strategically chosen Mission Areas
– Assessment & Metrics – Evolving as we go … we need input
ASAP!
• Programmatic – What progress are you making & is it consistent
with the plan?
• Technical – Does it do anything? Does it do what it needs to do?
• Operational – Does it do anything useful & of interest to our
customers? Why should we care?
• Transition – Will it go / is it going … anywhere?
23
24. FOR OFFICIAL USE ONLY
FOR OFFICIAL USE ONLY
Initial Insights, Concerns & Caveats
• Insights & Concerns:
– Need to develop “stories” for inside (HSCB) & outside
consumption (Scientific Modeling /End-users)
(Know you audience!!)
• Vision: Complementary... To HSCB, to each other, to related
sponsored efforts
• Mission, Goals & Objectives (by program phase)
– Technology Integration Events (TIEs) & Formal Assessments
– HSCB Outreach Events… as directed.
• Need media & stories ASAP to support “emergent” HSCB outreach
events.
24
25. FOR OFFICIAL USE ONLY
FOR OFFICIAL USE ONLY25
Tell a “Story”
• BLUF: You need a story...
– You need to tell us what you are going to do...
– It needs to have an elevator speech ...
– Every story will be different (know your audience!)…
– We need supporting media (stand alone): Posters, 3-Minute Video, etc.
– We will do what we can to help…
• Each project needs to tell a story that makes it clear where you are going,
why it is important, how it is relevant to HSCB / Pathways, and why it will
make a difference
• Heilmeier Catechism: (George H. Heilmeier, DARPA TD 1975-77)
• What are we trying to do?
• How does this get done at present? Who does it? What are the limitations of the
present approaches?
• What is new about your approach? Why do we think you can be successful at this
time?
• If you succeed, what differences do you think it will make?
• How long do you think it will take? What are your mid-term and final exams?
• How much will it cost?
26. FOR OFFICIAL USE ONLY
FOR OFFICIAL USE ONLY
Notional Pathways Story Template
• Commander Smith needs to know X in order to do Y – turns to his staff…
• An analyst has n hours to help the commander. He breaks the information
need into these questions: A, B, C.
• He uses (Pathways Framework) as a decision support system to find best
available information that will answer the questions.
– Discovers models that might help answer the questions
– Looks for and discovers contextually relevant data that could feed the models.
– Assesses the applicability of data to the models & conditions the data as needed.
– Develops (or verifies) an understanding the S-C topology. (Note: May link together
models to create the needed S-C Topology.)
– Starts Hybrid Modeling; linking models & data together to create Courses Of
Action (COAs) within the topology (s). Documents Assumptions & Limitations.
– Evaluates COAs & Develops recommendations
– Packages Recommendations to give to Commander.
– Command staff looks at recommendations and have visibility into supporting
models / analyses.
• Commander makes decisions, wins the war, and everyone lives happily
ever after!
26
28. 28
Pathways Nexus Team Advanced Technology Laboratories
For Official Use Only
Preliminary
Recommendations
• SSTR – Somalia - Hospital opening in Mogadishu
– Leverage LM pathfinder project (availability of initial models, data)
– Scenario defined by LM SME (Alex Moore) – value / correctness verified w/ active duty
members of reserves
• IO - Mexico – Influence campaign / Military campaign analysis
– Leverage LM Opinion Propagation Models, structured equation models
– Leverage relevant Columbian analyses / models – agent based models related to drug interdiction
operations
• Stability Analysis - Congo – Potential partition / stability analyses
– Potential leverage of statistical models and open source models
– Leverage of LM – ORNL Shared vision program – Congo interest
– Considered high interest to AFRICOM
• IO - Disappearance of Leaders – Single Bullet
– Multiple theories could be implemented as models
– Significant literature on authoritarian governments
– Instances of models could be applied to multiple situations
29. 29
HSCB Analysis Users
Category User (Analyst) Organizations HSCB Roles
Strategic Human Factors
Analysis
Human Factors Analyst HF IPT;DIA; JIOC Global
Harvest, NASIC, NGIC,
4POG, others
Rigorous analysis of foreign
individuals and networks
Special Analysis; Manhunt SOCOM; SKOPE; others All-source analysis; focus problems
Cultural Analysis and Targeting SOCOM All-source analysis; focus problems
Targeting Analysis NSA Special analysis; focus problems
Operational Socio-Cultural
Dynamics Analysis
Human Terrain Geospatial
Intelligence Analyst (General)
NGA HT Pilot program developing broad
application analytic methodology
S2/J2 Intel Analyst ; PMESII
targeting Cells Brigade and Above
COCOMS (ex. PACOM
Socio-cultural Dynamics
Center)
DCGS-A users; JIPOE analysis;
socio-cultural
HTS Reach-back Cell TRADOC CONUS expert support to HTT’s
Targeting; Socio-Cultural Analyst JIEDDO COIC Threat targeting analysis; social
environment analysis
Operational/Tactical
Field Analysis
S2 Intel Analyst Brigade and
Below
COCOM S2 cell JIPOE analysis; socio-cultural
HTT Human Terrain System-
Human Terrain Team (HTT)
TRADOC Support BCT Brigade Combat
Team; Interact with S3 effects cell
Stability Operations Information
Center (SOIC) Analyst
COCOM Social analysis; populations,
organizations, leaders and influences
30. 30
HSCB Planning Users
Category User (Planner) Organizations HSCB Roles
Strategic and
Operational Influence
Planning
Strategic Influence Planner CIA, DIA; COCOMs Develop national level deterrence,
influence plans (Strategic
Communication, IO, Lethal Force)
Operational PSYOP Planner SOCOM; 4POG Plan operational PSYOP campaigns
and activities
National IO Planner STRATCOM (JIOWC) ISPAN; VISION (JFCOM) Perform
coordinated theater and national-level
IO planning
Operational Planning
Support
J3 IO Planners COCOM J3 Cells, JIEDDO,
Ist IO Command, other IO
cells
Plan coordinated Information
Operations campaigns
Wargaming Analysts (J8) Joint Collaborative Analysis
Conf (JCAC) IPT
Conduct socio-cultural analyses in
support of Operational-level planning
Classified Task Cells JWAC Special planning activities
Operational Field
Planners
S3 PMESII Targeting Cells XVII Airborne, others Plan and monitor special PMESII
targeting actions
S3 Brigade and below IO lanners COCOM Units Tactical level, rapid response planning;
HSCB provides support with planning
products (organized information,
progress tracking, human terrain
products, etc.)
S5 Brigade and Below Planners COCOM units
31. 31
Challenge 1 – COIN Operations in Afghanistan
• Three ISAF LOOs
– Governance
– Development
– Security
• Data: Availability of adequate
social-cultural and economic data.
• Decision Support: Availability of
decision support tools in
combination with data to:
– Enable analysts to understand
social-cultural dynamics
– Enable planners to develop lines
of effort and COA’s within the S-C
context
– Enable policy-makers to make
effective decisions and create
measurable social, security,
economic opportunities in
Afghanistan.
Inefficient or Corrupt
Governance
Practices; Urban-
Rural divide
Dynamic social
political environment;
violent disruptions
and coercive
influences
Widely distributed
rural societies;
embedded threats
Social-Cultural Analysis Challenges in Quasi-Stable
Environments
Regional geo-
political influences
32. 32
Challenge 2- Strategic Influence Planning in HOA
• Phase 0 (Shaping) Challenges
– Counter growing influence of China
– Preventing spill-over of instability across
borders
– Deny safe havens
• Data: Sparse social-cultural and
economic data to support sources of
influence and behavior to sources
• Decision Support: Availability of
decision support tools in combination
with data to:
– Enable analysts to understand diverse
interests, perspectives, perceptions, and
influences
– Enable planners to develop lines of effort
and COA’s within the S-C context
– Enable policy-makers to make effective
decisions and create measurable effects
of Strategic Communications, diplomatic ,
security, and economic COA’s.
Horn of Africa
• AFRICOM challenge problems; phase 0 shaping
• Tribal power struggles; environmental degradation;
resource competition
• Marginal infrastructure; limited media sources
• Competing foreign influences
34. For Official Use Only
NEXUS Pathways
Integrated Socio-Cultural Model and Data
Exploitation for Multiple Missions & Granularity
Schedule: 60 Months
PERFORMERS: Lockheed Martin, Lustick Consulting, SAE
Inc. , The Penn State University, The Rendon Group
TECHNICAL APPROACH: Unlock and link the power
of heterogeneous models, simulations, tools, and
data through a services oriented architecture
(SOA) focused on:
• Providing innovative modeling and data analytic
capabilities including composing hybrid models, semantic
and theoretically grounded model interoperability, mixed-
method forecast triangulation, etc.
• Implementing translation and brokering services to
support data-dependent modeling/simulation needs from
a virtual distributed heterogeneous pool of data sources.
• Demonstrating framework flexibility by handling high-
volume input of raw structured and unstructured data
sources to feed a range of mission-specific prototypes.
OBJECTIVE: Exploit a broad range of extant and evolving
heterogeneous Socio-Cultural Modeling and Simulation services
to foster improved 1) Situation Understanding and Exploitation;
(2) Cultural Drivers and Theories; 3) Course of Action
Assessment and (4) Decision Support Options
MILITARY RELEVANCE: Forecast and Assess the impact and
consequences of potential actions on beliefs of hostile, friendly,
and neutral actors for specific areas and contexts of interest.
Enable commanders and command staff to readily collect,
model, forecast, and monitor pertinent situation, trends, and
activities.
Support a broad range of critical military mission areas including
Stability, Security, Transition and Reconstruction (SSTR),
Influence Operations (IO), Stability Analysis, Humanitarian
Assistance and Disaster Relief (HA/DR), etc.
Task FY 10 FY 11 FY 12 FY 15
Capability Develop. &
utility assessment
System Design
FY 14FY 13
Concept Development
and Requirements
System Prototype
Development and
Evaluation
Capability Enhance,
Eval & Transition
35. 35
SCHEDULE:
TECHNICAL APPROACH: Develop and Implement a
Computational Model-Based Analysis Capability:
• Modeling: Build and compose custom computational
models of the environment
• Data Management: Construct a data management
system capability to support model building and sharing
• Interactive Viewing: Explore causal chains and
indirect consequences of actions.
MILITARY RELEVANCE / OPERATIONAL IMPACT: Improve
intelligence modeling and analysis using integrated analysis tools
supported by critical automated information management and coding
processes. Provide tools to extract and monitor metrics such that the
effects of given actions can be understood by the command staff.
Task FY 10 FY 11 FY 12 FY 15
Capability Develop. &
utility assessment
System Design
FY 14FY 13
Concept Development
and Requirements
System Prototype
Development and
Evaluation
Capability Enhance,
Eval & Transition
OBJECTIVES: Develop and implement an hybrid modeling system to
be both scalable & robust, and can be transitioned to operational use.
PRISM features: provide for unbiased, objective, valid science-based
tools to enable DOD Stability Operations, Analysis, Intelligence, and
Experimentation.
PRISM Team
Analysis
J2
Planning
J5 (plan) J3 (0ps)
J8 (Assessment)Data
Acquisition
Reporting
Analysis
Plan
Assessment
Plan
Execution
Situation
Assessment
Mission
Intent
Plan
Development
Data
Translation
•Language
•Extraction
•Organize
•Search
•Acquire
•Metatag
•Structure
•Model,Test
•Anticipate
Social-Cultural Target
•Summarize;explain
•Focus Issues
•Warn,Predict
•Conceive Courses of
actions within constraints,
restraints,and resources
•Assess COA’s,
effects and outcomes
2532,
2533
Data Mgmt,
Brokering
2531 Scalable
Modeling
System
There are different types of S-C models in this domain – all of which will be necessary to addressing operational S-C needs. It is useful to note that the different kinds of S-C models lend themselves to answering different kinds of questions. “What Is” questions are often best addressed through data-driven models and methods. “What if” questions which involve predictive analyses often are addressed using theory or model driven methods. We posit that every S-C analysis will need to incorporate both approaches.
Further, the context of these what if and what is analyses will change as the unit of interest changes from local people and groups to regional and national or even international analyses.
8
9
10
11
Might be useful to think of the Meta-Theory as a Business or Process model for S-C Analysis.
Different processes might be appropriate for different S-C domains, e.g.
How to do S-C modeling that marginalizes insurgents within a population
How to do S-C modeling that facilitates stabilizations at the local level
How to empower local governments during humanitarian crisis.
How to detect external influences to economic & political stability.
Performers:
Pick a context you are enthusiastic about (or two)
Need first cut by Aug QPR Meetings. Evaluate during Phase 2 (if not sooner).
Nice if you talk to each other….. The expectations is that one of our system metrics is interoperability…
Provide graphics as storyboards to support outreach. Video??
Tell us when they will be implemented for evaluation.
Provide alternate models considered & rationale.
Gov’t may seek to apply S-C meta-theories as point of convergence among architectures.
End of Phase 2 – Formal evaluations of Meta-Theories as implemented within architectures.
We believe that hybrid modeling for the S-C domain will need to be tied to a meta-model that is both meaningful and observable to the S-C theorists. It will instantiate a meta-theory for the creation of hybrid S-C models. It must be meaningful to the users of the hybrid-models, and as such, will have visualization components coupled to the meta-model within the Pathways visualization framework.
The meta-model shown here is representative of the kind of model we may need. It was developed under the IARPA A-SpaceX program and describes the common elements of the analytic process.
The analytic process is inherently structured, so this model seems to be very robust across a number of domains. This model is similar to those taught to analysts with the IC.
There are two conceptual loops embedded within the model: Foraging (Looking for and organizing raw information); and Sense Making (Organizing information to address a problem or topic).
a key feature is that the transition between stages are detectable as explicit behaviors.
Adding structure is tantamount to adding information. The analyst ultimately composes a story to be shared with other people, and should be able to provide supporting evidence and explain the interpretation that led there.
When elements at an upper stage do not make sense, or require new information to supplement, the analyst iterates back down through the stages.
The process is highly iterative … but generally linear (one must go through all the steps to some degree for a given analysis to deliver a complete analysis and be able to explain it).
How should we implement the meta-model? Need to address:
How would you show a meta-model is correct?
How will you address what is required for care and feeding of the models / meta-models? (How will you know if the meta-model you are using is appropriate and current?)
Social theory should suggest the meta-model. We should look for ontologies that suggest the S-C variables and decompose them to show inter-dependencies – in effect reveal a meta-theory for the S-C theories & models we expect to deal with. We may want to think about levels of meta-models….
What would you propose as a “mechanical” construct for addressing this issue?
Linearize the state space so as to constrain hybrid models & complexities. See also optimal filtering… Use to limit the state space.
We cannot be using the models in an open-loop modeling process….
Might want to implement this function in terms of a “Model Predictive Controller”.
Take in observations to verify that the topology model is appropriate, and it will be valid to use a meta model.
Limit the hybrid modeling to 6-8 terms – these should be derived from the meta-model.
Do we have constant measurements that can feed the controller? What are they?
Note: The goal of a Model Predictive Controller is to facilitate goal constrained optimization. It should:
Show possible actions
Let the modeler Identify the best actions
Suggest the Schedule for actions
Tell us when the model is wrong; suggest how the model could be fixed. Tell us when we are operating outside identified constraints.
These are not definitive nor exhaustive. They are an initial attempt to characterize the performance characteristics we believe would demonstrate a suitable S-C meta-theory that would facilitate S-C Hybrid modeling.
This schematic provides a functional view of how Pathways contributes to the analysis process. It starts with an overall policy or treatment question, e.g., a commander is wanting to improve the economy in a province via business initiatives. The four steps are as follows:
A large corpus of data is assembled and a variety of relationships are determined. These can be correlations, or in some cases may represent causal relationships. These relationships are the basis for the next step, finding models appropriate to the analysis questions.
In this step, the analyst develops several topics or lines of inquiry to answer the overarching question. These could be, for this discussion, different business sectors in which the commander might recommend investments. Pathways would assemble hybrid models that allow projections of courses of action.
The third step simulates several potential outcomes for different settings of decision variables.
The last step compares these projections with respect to real world data. The actual evidence may not yield strong support for any of the projected paths. The Evidence Calculus would measure the degree of support, and would suggest where additional data would be most beneficial in improving this measure. In more complex scenarios, this assessment of evidence coupled with a larger set of projections can also help suggest what next questions should be addressed.
Metrics for Pathways development include:
How many data categories are incorporated in Step 1 and how many relationships are discerned
How many analysis topics are generated (with some computer support) and how well-matched are the projections to real data (measured in scientifically valid experiments)
How useful are the projections when measured by the evidence calculus. I.e., can the degree of support of the modeled projections, as measured by the evidence calculus, be improved upon with guided and feasible additional data collection.