This document discusses using probabilistic models to optimize multi-robot systems. It describes modeling system behavior, extracting parameters, and using models to find optimal control parameters and system designs. Optimization aims to maximize metrics like speed, reliability and user benefits while minimizing cost and size. The document presents examples of optimizing a coverage task by finding optimal waiting times between robot inspections using analytical models. It also discusses estimating model and control parameters simultaneously from experimental data.

1. Multi-Robot Systems CSCI 7000-006 Monday, October26, 2009 NikolausCorrell

2. So far Probabilistic models for multi-robot systems Extract probabilistic behavior of sub-systems Small state space: rate equations Large state space: DES simulation

3. Today System optimization using probabilistic models Find optimal control parameters Explore new capabilities using models Find optimal control and system parameters

4. Comparison of Coordination Schemes Too large Metrics for comparison Speed System cost System Size System Reliability Benefits to the User Size Speed Cost Too slow No Mapping Reliability Too expensive Mapping Benefits System-design is a constraint optimization problemSolution: Appropriate Models

5. Model-based design Size, Cost, … Speed, Reliability, … Control parameters Real System Model Controller Design

6. Model-based optimization Physical simulator Simulate controllers and robot designs DES simulator Simulate controllers and available information Optimize using Systematic experiments Learning/optimization Communication Navigation accuracy

7. Optimization using analytical models Probabilistic state machine is derived from the robot controller One difference equation per state pc Search Collision Rest pr Ns Tr Nc 1/Tc

8. Optimization using analytical models Probabilistic state machine is derived from the robot controller One difference equation per state System parameters pc Search Collision Rest pr Ns Tr Nc 1/Tc

9. Optimization using analytical models Probabilistic state machine is derived from the robot controller One difference equation per state Control parameters pc Search Collision Rest pr Ns Tr Nc 1/Tc

10. Optimal Control: Brief Intro Find optimal control inputs for a dynamical system to optimize a metric of interest Example: Tank reactor, maximize quantity B by tuning inflow and outflow Known: system dynamics inflow A outflow A->B->C A->C

11. Static Optimization A inflow outflow Find optimal control inputs (constant) Example: inflow 50l/min, outflow 10l/min Constraint: Volume of the tank at final time flow volume time

12. Dynamic Optimization A inflow outflow Find optimal control input profiles (time-varying) Example: max inflow for 10s, outflow off, after 50s and outflow max Constraint: Volume of the tank during the whole process flow volume time

13. Optimal Control Capture terminal and stage cost as well as constraints using a single cost function The optimization problem is then solved by minimizing this cost function

14. Example: Coverage Collaboration policy: Robots wait at tip for Ts Waiting robots inform other robots to abandon coverage Trade-Off between additional exploration versus decreased redundancy Communication introduces coupling among the robots (non-linear dynamics) N. Correll and A. Martinoli. Modeling and Analysis of Beaconless and Beacon-Based Policies for a Swarm-Intelligent Inspection System. In IEEE International Conference on Robotics and Automation (ICRA), pages 2488-2493, Barcelona, Spain, 2005.

15. Optimal Control Problem A static beacon policy does not reduce completion time but only energy consumption Is there a dynamic policy which improves coverage performance? Find the optimal profile for the parameter Tsminimizing time to completion

16. Model N. Correll and A. Martinoli. Towards Optimal Control of Self-Organized Robotic Inspection Systems. In 8th International IFAC Symposium on Robot Control (SYROCO), Bologna, Italy, 2006.

17. Optimization Problem u= Terminal cost: time to completion Stage cost: energy consumption Constraints: number of virgin blades zero

18. Possible optimization method: ExtremumSeeking Control Necessary condition of optimality: Optimization as a feedback control problem: Gradient Estimate by sinusoidal perturbation:

19. Optimal Marker Policy Stationary Marker Optimal policy “Turn marker onafter around 180s,mark for 5s and go On.” Method fminconusing the macroscopic model and optimal parameters based on base-line experiment.

20. Results/Discussion Optimal results when beacon behavior is turned on toward the end of the experiment Intuition: Exploration more important in the beginning An optimal beacon policy only exists if there are more robots than blades

21. Randomized Coverage with Mobile Marker-based Collaboration Translate Inspect Inspect Avoid Obstacle Wall | Robot Obstacle clear Search Inspect Mobile Marker pt Blade 1-pt | Marker Tt expired

22. g=0 no collaboration g=1 full collaboration

23. No Collaboration vs. Mobile Markers No Markers Mobile Markers 20 Real Robots Agent-based simulation

24. Model-based design: Pitfalls m e l Model-based controller design depends on accurate parameters Ideal model Optimization problem(s) Find optimal control parameters Find optimal model parameters Model M m Estimate both model and control parameters simultaneously e Model l “Optimal control under uncertainty of measurements” M

25. Simultaneous optimization of model and control parameters How to select pi when Ti are unknown? Optimization algorithm Initial guess for model and control parameters Run the system and collect data Find optimal fit for model parameters Find optimal control parameters Repeat until error between experiment and model vanishes N. Correll. Parameter Estimation and Optimal Control of Swarm-Robotic Systems: A Case Study in Distributed Task Allocation. In IEEE International Conference on Robotics and Automation, pages 3302-3307, Pasadena, USA, 2008.

26. Optimal Control ofSystem and Control Parameters Control Parameters System Parameters All experiments Next experiment

27. Case Study: Task Allocation Finite number of tasks Robots select task iwith probability pi =const. (Independent robots) pi(k) function of Nj(k) Task itakes time Ti in average K. Lerman, C. Jones, A. Galstyan, and M. Matarić, “Analysis of dynamic task allocation in multi-robot systems,” Int. J. of Robotics Research, 2006.

28. 1. Independent Robots Model(Number of robots in state i) Parameters probability to do a task System parameters Analytical optimization

29. 2. Threshold-based Task Allocation Probability to do a task Stimulus Threshold Stimulus: Number of robots doing the task already Model (non-linear) Optimization: numerical

30. Experiment Step 1: Estimate model parameters Ti are unknown Take random control parameters Measure steady state Find Ti given known control parameters Step 2: Find optimal control parameters

31. System dynamics DESSimulation Difference Equations Independent Robots (Linear Model) Threshold-based (Non-Linear Model) 100 robots 1000 robots

32. Results Linear Model Non-Linear Model 25 robots 25 robots 100 robots

33. Summary System models can be used for finding optimal control policies and parameters Models can be physical simulation, DES, or analytical More abstract model allow for more efficient search, even analytical System parameters can be optimized simultaneously with system in the loop

34. Upcoming Multi-Robot Navigation (M. Otte) Learning and adaptation in swarm systems 3 weeks lectures, 1 week fall break November 29: reports due 2 weeks project presentations, random order