1. A Machine Learning Approach for Real-Time
Reachability Analysis
Ross E. Allen, Ashley A. Clark, Joseph A. Starek, and Marco
Pavone
Autonomous Systems Laboratory
Department of Aeronautics & Astronautics
Stanford University
IROS 2014
September 16th, 2014

2. IROS 2014
R. Allen, A. Clark,
J. Starek, M.
Pavone
Introduction
Motivation
Approach
Results
Literature
Problem Description
Figure: Simplified, 2D illustration of a cost-limited
reachable set for an initial state of a dynamical system.
Green endpoints are reachable and red endpoints are
non-reachable for the given threshold Jth.
R(x0, U, Jth) = xf ∈ X | ∃tf and ∃u(τ) ∈ U, τ ∈ [t0, tf ]
s.t. x(t0) = x0, ˙x(τ) = f (x(τ), u(τ)),
1 / 8

3. IROS 2014
R. Allen, A. Clark,
J. Starek, M.
Pavone
Introduction
Motivation
Approach
Results
Literature
Motivation Slide
• Reachability analysis central to motion planning,
collision avoidance, differential games, etc.
• Simple for geometric systems; difficult/costly for
kinodynamic systems
• Analytical results available for few problems;
numerical solutions necessary for most
2 / 8

4. IROS 2014
R. Allen, A. Clark,
J. Starek, M.
Pavone
Introduction
Motivation
Approach
Results
Literature
Approach
3 / 8

5. IROS 2014
R. Allen, A. Clark,
J. Starek, M.
Pavone
Introduction
Motivation
Approach
Results
Literature
Approach
Cost Function Regression
4 / 8

6. IROS 2014
R. Allen, A. Clark,
J. Starek, M.
Pavone
Introduction
Motivation
Approach
Results
Literature
Approach
Cost Function Regression
4 / 8

7. IROS 2014
R. Allen, A. Clark,
J. Starek, M.
Pavone
Introduction
Motivation
Approach
Results
Literature
Approach
Cost Function Regression
4 / 8

8. IROS 2014
R. Allen, A. Clark,
J. Starek, M.
Pavone
Introduction
Motivation
Approach
Results
Literature
Approach
Cost Function Regression
4 / 8

9. IROS 2014
R. Allen, A. Clark,
J. Starek, M.
Pavone
Introduction
Motivation
Approach
Results
Literature
Approach
Cost Function Regression
4 / 8

10. IROS 2014
R. Allen, A. Clark,
J. Starek, M.
Pavone
Introduction
Motivation
Approach
Results
Literature
Approach
Classification into Reachable and Non-Reachable Sets
5 / 8

11. IROS 2014
R. Allen, A. Clark,
J. Starek, M.
Pavone
Introduction
Motivation
Approach
Results
Literature
Approach
Classification into Reachable and Non-Reachable Sets
5 / 8

12. IROS 2014
R. Allen, A. Clark,
J. Starek, M.
Pavone
Introduction
Motivation
Approach
Results
Literature
Approach
Classification into Reachable and Non-Reachable Sets
5 / 8

13. IROS 2014
R. Allen, A. Clark,
J. Starek, M.
Pavone
Introduction
Motivation
Approach
Results
Literature
Approach
Classification into Reachable and Non-Reachable Sets
5 / 8

14. IROS 2014
R. Allen, A. Clark,
J. Starek, M.
Pavone
Introduction
Motivation
Approach
Results
Literature
Simulations
Dubins Car
Figure: Predicted reachability set plotted with true cost for
the Dubins Car with 3 different cost thresholds (black
lines). Blue circles = SVM predicted reachable, red
diamonds = SVM predicted non-reachable, blue dot =
linear regression predicted reachable, red cross = linear
regression predicted non-reachable.
6 / 8

15. IROS 2014
R. Allen, A. Clark,
J. Starek, M.
Pavone
Introduction
Motivation
Approach
Results
Literature
Simulations
Deep-Space Spacecraft
Figure: Predicted reachability set plotted with true cost for
the deep-space spacecraft with 3 different cost thresholds
(black lines). Blue circles = SVM predicted reachable, red
diamonds = SVM predicted non-reachable, blue dot =
linear regression predicted reachable, red cross = linear
regression predicted non-reachable
6 / 8

16. IROS 2014
R. Allen, A. Clark,
J. Starek, M.
Pavone
Introduction
Motivation
Approach
Results
Literature
Simulations
Execution Time vs. Accuracy
Table: Average computation time and percent of
misclassification for the two-point boundary value problem
solver, the linear regression cost estimation (best fit model
using all features), and SVM classification (average over all
3 cost thresholds).
System 2PBVP Solve Lin. Reg. SVM
Time % Err Time % Err Time % Err
Dubins 1.23 s 0.0 9.4 ms 3.8 0.44 ms 4.9
Spacecraft 10.3 s 0.0 9.0 ms 5.8 0.40 ms 8.8
• Orders of magnitude decrease in computation time
if you are willing to accept some level of error
• Linear Regression slower than SVM but provides
more information, i.e. approximate cost of query
6 / 8

17. IROS 2014
R. Allen, A. Clark,
J. Starek, M.
Pavone
Introduction
Motivation
Approach
Results
Literature
Thank you!
Ross E. Allen, Ashley A. Clark, Joseph A.
Starek, and Marco Pavone
Stanford University
Aeronautics & Astronautics
6 / 8

18. IROS 2014
R. Allen, A. Clark,
J. Starek, M.
Pavone
Introduction
Motivation
Approach
Results
Literature
Literature Review
A direct approach requires the numerical solution to a
2PBVP. An alternative, indirect method relies on the
numerical computation of the zero-level set of the
viscosity solution of the Hamilton-Jacobi-Bellman
equations, though with exponential time complexity
[1]. Modern research on reachability analysis has
therefore focused on producing efficient over- or
under-approximations of reachable sets for various
systems:
• discrete linear case [2]
• continuous-time linear case [3]
• general nonlinear dynamics [4, 5]
• nonlinear differential-algebraic dynamics [6]
• hybrid systems [7]
Drawbacks of traditional techniques:
• Computationally intensive
• Do not appear amenable to real-time implementation for
all but the simplest cases.
7 / 8

19. IROS 2014
R. Allen, A. Clark,
J. Starek, M.
Pavone
Introduction
Motivation
Approach
Results
Literature
References I
Duˇsan M. Stipanovi´c, Inseok Hwang, and Claire J. Tomlin.
Computation of an Over-Approximation of the Backward Reachable Set using
Subsystem Level Set Functions.
In Dyn. of Cont., Discrete and Impulsive Systems, volume 11 of A: Mathematical
Analysis, pages 397–411. Watam Press, 2004.
Francesco Borrelli, A. Bemporad, and M. Morari.
Predictive Control, 2014.
In Preparation.
Antoine Girard and Colas Le Guernic.
Efficient Reachability Analysis for Linear Systems Using Support Functions.
In Myung Jin Chung and Pradeep Misra, editors, IFAC World Congress, volume 17,
pages 8966–8971, Gangnam-gu Seoul, South Korea, July 2008. Int. Fed. of Automatic
Control, IFAC PapersOnLine.
Eugene Asarin, Thao Dang, and Antoine Girard.
Reachability Analysis of Nonlinear Systems Using Conservative Approximation.
In Oded Maler and Amir Pnueli, editors, Hybrid Systems: Comp. and Control, volume
2623 of Lecture Notes in Comp. Science, pages 20–35. Springer, Prague, Czech
Republic, April 2003.
Romain Testylier and Thao Dang.
NLTOOLBOX: A Library for Reachability Computation of Nonlinear Dynamical
Systems.
In Dang Van Hung and Mizuhito Ogawa, editors, Autom. Tech. for Verif. and Analysis,
volume 8172 of Lecture Notes in Comp. Science, pages 469–473. Springer, Hanoi,
Vietnam, Oct 2013.
7 / 8

20. IROS 2014
R. Allen, A. Clark,
J. Starek, M.
Pavone
Introduction
Motivation
Approach
Results
Literature
References II
Matthias Althoff and Bruce H. Krogh.
Reachability Analysis of Nonlinear Differential-Algebraic Systems.
IEEE Trans. on Automatic Control, 59(2):371–383, Feb 2014.
Herv´e Gu´eguen, Marie-Anne Lefebvre, Janan Zaytoon, and Othman Nasri.
Safety Verification and Reachability Analysis for Hybrid Systems.
Annual Reviews in Control, 33(1):25–36, April 2009.
8 / 8

21. IROS 2014
R. Allen, A. Clark,
J. Starek, M.
Pavone
Backup Slide #1
1 / 1