Dr. Fariba Fahroo presents an overview of her program, Dynamics and Control, at the AFOSR 2013 Spring Review. At this review, Program Officers from AFOSR Technical Divisions will present briefings that highlight basic research programs beneficial to the Air Force.

1. 1DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution14 February 2013
Integrity  Service  Excellence
Dr. Fariba Fahroo
Program Officer
AFOSR/RTA
Air Force Research Laboratory
Dynamics & Control
Date: 4 March 2013

2. 2DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution
2013 AFOSR SPRING REVIEW
NAME: Fariba Fahroo
BRIEF DESCRIPTION OF PORTFOLIO:
Developing mathematical theory and algorithms based on the
interplay of dynamical systems and control theories with the aim of
developing innovative synergistic strategies for the design, analysis,
and control of AF systems operating in uncertain, complex, and
adversarial environments.
LIST SUB-AREAS IN PORTFOLIO:
• General Control Theory: optimal control, adaptive control, stochastic
control, hybrid control
• Distributed Multi-Agent Control: path planning, decision making,
sensing, task allocation with adversarial and stochastic elements with
incomplete information and communication constraints
• Mixed Human-Machine Interface
• Control of Distributed Parameter Systems
• Emerging Applications: Quantum Control, Vision-based Control

3. 3DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution
Key Portfolio Research
Challenges
Key enabling research areas in the science of autonomy
•Autonomous Dynamic Mission Planning : Hybrid System Formulation
(mix of finite, discrete state (decision variables) and continuous Dynamics)
• Human-Machine Interactions and operations: Stochastic Dynamic
Programming, Stochastic Control
• Incorporation of Uncertainty in Mission Environment, Model
Parameters: Adaptive Control, Stochastic Control
• Adversarial Behavior: Game theoretic Approaches, Stochastic game
theory (mean-field theory)
• Incomplete Information: incorporation of learning into games, task
allocation, planning
• Computational Issues: Computational Nonlinear Control Theory
• Emergent Applications: Quantum Control, Vision Control

4. 4DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution
Distributed Control for Networked Systems
• Objectives: To address fundamental problems in distributed control of
networked systems with non-traditional constraints and adversarial action.
• Challenges: A comprehensive research program that deals, within a
multi-agents context,
– with decentralization,
– localization of objectives,
– lossy communication and information exchange,
– incompleteness of information,
– adversarial interference,
– multiplicity of objectives,
– coordination through multi-level interaction.
• Modeling Framework: deterministic, stochastic and hybrid structures, cast
in discrete- as well as continuous-time settings, with a strong mathematical
underpinning.
• Multi-Disciplineary Research: Control theory, Information theory,
Decision theory, Networks, Coding, Communication, Computing, and Game theory.

5. 5DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution
Multi-Agent Autonomy
How (MIT)
• Vision: Team of agents that share experiences and learn to act more intelligently in
dynamic, uncertain, & contested environments
– Learn/update models & planners/policies to improve mission performance
• Approach: Intelligent Cooperative Control Architecture (iCCA) that builds/learns
effectively from prior knowledge
– Synergistic integration of planning and
learning to improve performance
– Bounds risk in learning process
• News:
– iCCA with adaptive modeling shown to
out-perform individual methods
– Developed new model learning techniques
that greatly reduce sample complexity of iCCA
– Performance improvement demonstrated in
experimentally using quadrotors
CBBA
CBBA
+
Sarsa
iCCA
Sarsa
Sarsa
CBBA
conservative
CBBA
Aggressive
iCCA
iCCA
Adaptive
Modeling

6. 6DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution
Distributed Task Allocation
• Goal: Automate mission planning to improve performance of networked team of
heterogeneous agents operating in a contested environment
– Distributed/decentralized planning is required for some communication
environments (e.g. slow, high cost, unreliable, ...).
– Planning issues: convergence guarantees/time, mission performance
• Approach: Consensus-Based Bundle Algorithm (CBBA) developed as multi-
assignment task allocation algorithm
– Recent results highlighted the performance limitations associated with the score
functions that must be used to guarantee convergence
• News: Developed bid-warping approach for CBBA
– Expands the set of score
functions that can be used
– Can use cost functions that
make the most sense
– Provable bound on
performance

7. 7DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution
Time-Critical Cooperative Missions for
Multi-Agent Systems
Naira Hovakimyan (UIUC)
 Time-critical applications for multiple vehicles with spatial constraints:
• Sequential auto-landing (UAVs)
• Coordinated reconnaissance missions
• Simultaneous arrival at multiple locations
How to:
1) Efficiently generate trajectories that satisfy desired relative timing constraints?
2) Enforce these constraints with desired performance levels in communication-
limited or communication-deprived environments?
Courtesy: NPS Courtesy: LARSyS, IST

8. 8DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution
Coordination in Communication-Limited
or -Deprived Environments
 Connectivity of the communications network satisfies:
• Graph connected in an integral
sense, not piecewise in time
 Rate of convergence of the coordination dynamics proportional to the connectivity level:
Low connectivity implies slow convergence
 Potential directions for research:
• Use of onboard estimators:
 can improve learning of other agents’
dynamics…
 but leads to larger networks (characterized by
smaller convergence rates).
• Topology control is needed to decide which estimators
to listen to and which not:
 ‘Edge snapping’ based on link quality and
coordination error.
How to modify coordination control algorithms to improve convergence rate?

9. 9DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution
Autonomous Optimal Pursuit
David Casbeer, Krishna Kalyanam, Phil Chandler, AFRL/RQ
Meir Pachter, AFIT, Swaroop Darbha, TAMU
• Problem Formulation: Develop practical and
autonomous decision-making algorithms under
uncertainty
• UGS monitors a road network, while a UAV(s) continually
visits the UGS for status updates
• When an UGS alerts a UAV of an intruder, the UAV must
decide where to go to capture/image the intruder and
deliver imagery to operator
• Challenging Problem: Stochastic optimization with dual-control aspects
including partial and delayed observations
• Very few problems of this type have been addressed
• Solution does not require complete information history (implementable)
• Developed probabilistic guarantees for autonomous capture in stochastic setting
• Transition: Flight test and demonstration

10. 10DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution
Flight Test (Transition)
– Onboard computation and UAV-to-UGS communication
have been designed, tested, and characterized
• An estimation scheme that predicts the location of the
intruder given a set of UGS measurements has been
integrated with human interface software developed by
AFRL/RH
• Flight test algorithm (Spring ‘13) and demonstrate
capability at a joint US/AUS military exercise; Summer
2013
• Three flight tests have validated
hardware and software architecture to
implement intruder capture algorithms

11. 11DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution
NETWORKED GUIDANCE AND CONTROL FOR
MOBILE MULTI-AGENT SYSTEMS
Nuno C. Martins, Univ of Maryland
Objective: Tractable methods to design
Markov Decision Processes (MDPs) that
maximize
coverage subject to operational constraints

12. 12DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution
NETWORKED GUIDANCE AND CONTROL FOR MOBILE
MULTI-AGENT SYSTEMS
Nuno C. Martins
Potential AF impact:
- Characterization of the largest configuration set that can be
persistently surveyed by a team of autonomous vehicles
- Determine the minimum number of vehicles required to
survey a certain area, subject to safety and performance
constraints.
Possible Future directions:
- Extension to continuous state-space.
- Include output feedback.
Scientific impact:
- Defined a new optimization paradigm for MDPs,
based on maximum entropy methods.
locations and directions
that can be persistently visited
forbidden locations

13. 13DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution
2010 MURI (PI: Tamer Başar, UIUC)
Multi-Layer and Multi-Resolution Networks of
Interacting Agents in Adversarial Environments
Vision and Overarching Goal:
To address fundamental issues that arise in networked heterogeneous
agents, among which are:
complex interactions; uncertainty and adversarial actions; trust,
learning; humans-in-the-loop; information and communication; and design
of architectures to facilitate generation and transmission of actionable
information for performance improvement under different equilibrium
solutions.
MLMR Games: Games played at different
levels, interacting through their outcomes,
action spaces, and costs
“Games within Games”: Zooming in and out
providing game structures with different levels
of granularity

14. 14
Multi-Layer and Multi-Resolution Hybrid Stochastic Differential
Games in the Large Population Regime and Their Equilibria
• Large-scale interactions happen at different layers of a complex system. Players
have inner interactions with members in the group as well as long-range
interactions with other groups.
• Coupled decisions = Game theory
• – Players/Actors/Agents
• – Actions/Strategies/Choices
• – Preferences over joint choices (utility functions)
• – Solution concept (e.g., Nash equilibrium)
• Approach: A multi-layer, multi-resolution (MLMR) game framework
DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution

15. 15
Large Population Game Model
• The problem is computationally quite challenging. For the case of large populations, the
idea of mean-field representation is used.
• Key Intuition from Statistical Mechanics: Extremely complex large population particle
systems may have simple continuum limits with distinctive bulk properties.
• When the number of players is large, the cost of a player does not explicitly depend
on the actions of the other players but rather coupled through the distribution of the
agent states m.
• Characterization of the Nash equilibrium for this case is a significant result.
• The mean-field equilibrium is obtained by solving coupled HJB backward equations
together with the generalized FPK equation
DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution

16. 16DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution
MURI ‘07: Decision Dynamics in Mixed
Teams of Humans and Robots
PI: John Baillieul (BU)
Toward a Control theory of Planning, Perception, and Reaction
• Human decision making –
• the effects of peer and other social pressure
• Modeling spatial and topological knowledge and
defining knowledge acquisition metrics for mixed
teams of humans and automata
• How perceptual dynamics affect human decision
makers
• Elements of a formal theory perception-based
control

17. 17DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution
Challenges: An encompassing theoretical basis for joint
cognitive systems did not exist.
• Need to understand how human decisions differ from those of ideal
decisions makers due to: biases, fear of disapproval, and other forms of
group pressure. (Psychology addresses such questions.)
• Paradigm shift: humans operating on parity with automatons.
Approaches and Results:
•Design of reward-incentivized games for play by individual subjects and by groups of
subjects, with various forms of information sharing.
•Trade-offs between exploitation of strategies with known payoff and exploration of
alternative strategies in the hope of higher payoff.
• Shared information about reward sizes tended to encourage exploration of strategy
alternatives in games that were studied.
•When people play with awareness of others playing simultaneously, competitiveness
emerges.
• Knowledge of others’ reward affects play style more than knowledge of others’ play
strategies
A deeper understanding of decision making in mixed human/robot
teams can be pursued through research on information
acquisition, sharing and propagation in complex networks.
Decision Making in Mixed
Human/Robot Teams

18. 18DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution
Task Partitioning in Mixed
Human/Robot Teams
•Challenges:
•Understand task partitioning between machines and humans in search and
reconnaissance.
• Use stochastic control principles to develop automated task partitioning and
adaptive resource assignment algorithms for teams of agents with finite resources.
• Determine advantages/disadvantages of alternative interaction structures
between human/automata.
•Results:
•Automated task partitioning and scheduling algorithms that incorporate risk of
vehicle loss; formulated as a stochastic combinatorial problem on graphs.
• Dynamic Search/Classify Problems: Agents’ actions generate noisy observations,
processed through Bayesian techniques, that are then used by agents in selecting
future actions.
• Normative solution of such problems requires full stochastic dynamic
programming; constraint relaxation allows scalable solution
A fundamental distinction between humans and robots in mixed teams is
that robots react rapidly to real-time sensor data, whereas humans react to
more complex perceptions of the environment in which cognitive processes
and prior experience play a role.
:

19. 19DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution
Computational Control Theory
McEneaney (UCSD) ,Dower (Uni-Melbourne)
• Challenges: Develop new theory and computational tools to solve
previously intractable optimal control and dynamic game
problems.
• Approach: 1) Develop and exploit new and existing theory
connecting idempotent algebra and dynamic programming to
identify or construct fast solution propagators for specific classes
of optimal control and dynamic game problem.
• 2) Develop new computational tools to implement these fast
solution propagators to solve practical problems.
• The N-body problem stands as a benchmark in astrodynamics.
• The focus lies in the (harder) two-point boundary value N-body
problem:
– Given a combination of initial and some terminal data, determine the
N-body trajectory meeting these conditions.
• Applications: Asteroid danger estimation, Efficient space vehicle trajectory design,
Concept generalizable to other conservative systems and potentials.

20. 20DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution
Least Action and Game-Based
Gravitational Potential Representation
• Max-plus based fundamental solutions for Riccati ODE and PDE are
immediately transferable to least-action approach to two-point boundary
values problems (TPBVPs) with quadratic potentials (e.g., mass-spring, wave
equation dynamics).
• Least-Action Principle: Conservative system evolves so as to minimize
(more correctly find stationary point) of the action (kinetic minus potential
energy).
• Re-interpretation of the gravitational potential energy. Consider two bodies
separated by distance, r.
– Traditional potential energy representation:
– Competing controller representation:
– Note quadratic form for potential as function of position –
computationally very useful.

21. 21DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution
Fundamental Solutions =>
Extremely Rapid Evaluations
• Fundamental solution given as a set of solutions of Riccati equations, S(t).
(Least action is obtained as max of linear function over convex set S(t).)
• Having a fundamental solution allows for very rapid solution of large
numbers of TPBVPs (fixed set of masses and duration).
• Suppose we are given all N initial positions, x, and N terminal velocities, vf .
Want initial velocities (and trajectories).
• Repeat ad nauseum for variety of x, vf .
S(t) depicted as layers
indexed by γ, two-body case.
(=initial velocities)

22. 22DISTRIBUTION STATEMENT A – Unclassified, Unlimited Distribution
Summary
• Dynamics and Control is the key enabler in the science of autonomy. There
have been fruitful collaborations with other disciplines such ad cognitive
science, network science, communication, information science.
– Plan is to initiate new MURI and AFOSR Basic Research Initiative (BRI) topics in V&V,
Human-Machine interaction, Autonomy in contested environments
• Close collaboration and sharing of information with other agencies: ONR,
ARO and NSF
• New application areas have benefited from control theoretic approaches,
and control theory has grown and changed because of these applications=>
Examples: Synthetic biology (new BRI topic), Quantum Control (plan for a
future BRI)
• Support of fundamental research in Control and Dynamics while considering
challenging and relevant application areas is what distinguishes this
portfolio.