"Decision-theoretic and game-theoretic approaches to decision making in collectives", Matthijs Spaan, 11 de Março de 2009, Ciclo de Conferências "Das Sociedades Humanas às Sociedades Artificiais" (edição 200), Instituto de Sistemas e Robótica, Instituto Superior Técnico, Lisboa

1. Decision-theoretic and game-theoretic approaches
to decision making in collectives
Matthijs Spaan
Institute for Systems and Robotics
´
Instituto Superior Tecnico
Lisbon, Portugal
`
Das Sociedades Humanas as Sociedades Artificiais,
March 11, 2009
PÓLO DO I.S.T 1/26

2. Introduction
• Our research goal: “intelligent” decision making for robots
in collectives.
• Here: intelligent = rational given a task.
• Examples:
◮ Robots to assist people;
◮ Multi-robot search and rescue;
◮ Robots playing football;
◮ Software agents on the internet.
PÓLO DO I.S.T 2/26

3. Artificial Societies vs. Human Societies
• Our perspective: design artificial systems with a purpose.
• Robot societies need not resemble human societies.
• Humans: not fully rational.
• Robots: hopefully more rational.
PÓLO DO I.S.T 3/26

4. Rational robots
• Rational robots do the right thing.
◮ Knowledge of the environment.
◮ Actions the robot can take.
◮ Perceptions seen so far.
◮ Performance measure.
PÓLO DO I.S.T 4/26

5. Rational paths
PÓLO DO I.S.T 5/26

6. Limited rationality
PÓLO DO I.S.T 6/26

7. Rational behavior
• Don’t program how the robot should behave.
• Program its environment, actions, perceptions, performance
measure.
⇒ Robot will behave rationally.
• Equip robot with tools to behave rationally.
PÓLO DO I.S.T 7/26

8. Utility
• Utility: degree of happiness.
• If a robot prefers one situation over another one, it has
higher utility.
• Utility function: utility of each situation.
• We define our robot’s utility function.
PÓLO DO I.S.T 8/26

9. Utility
Gold More Gold
Recharge
Point Lion
PÓLO DO I.S.T 9/26

10. Utility
Utility = +0
PÓLO DO I.S.T 10/26

11. Utility
Utility = +10
PÓLO DO I.S.T 11/26

12. Utility
Utility = +30
PÓLO DO I.S.T 12/26

13. Utility
Utility = −100
PÓLO DO I.S.T 13/26

14. Utility
Battery high ⇒ Utility = +0
Battery low ⇒ Utility = +10
Battery very low ⇒ Utility = +100
PÓLO DO I.S.T 14/26

15. Maximizing Utility
• Rational robots maximize utility over their lifetime.
• Robots needs to consider sequential decisions.
• Maximize benefits while minimizing costs.
• Each action uses energy.
PÓLO DO I.S.T 15/26

16. Maximizing Utility
PÓLO DO I.S.T 16/26

17. Maximizing Utility
Utility: 5 ∗ −1 + 10 = 5
Utility: 3 ∗ −1 + 10 = 7
−1 −1 −1
−1
+10
+10 −1
−1 −1 −1
PÓLO DO I.S.T 17/26

18. Maximizing Utility
Utility: 4 ∗ −1 + 20 = 16
Utility: 2 ∗ −1 + 10 = 8
+20
+10
PÓLO DO I.S.T 18/26

19. Maximizing Utility
Utility: 2 ∗ −1 − 100 + 20 = −82
Utility: 3 ∗ −1 + 10 = 7
Utility: 5 ∗ −1 + 20 = 15
−100
+20
+10
PÓLO DO I.S.T 19/26

20. Uncertainty
• Things don’t always work out as you want. . .
• Robots need to deal with noise.
• Uncertainty complicates maximizing utility.
PÓLO DO I.S.T 20/26

21. Maximizing Utility under Uncertainty
safe path
dangerous path
+10
+20
−20
−20
−20
PÓLO DO I.S.T 21/26

22. Robot collectives
• A collective of robots brings interesting challenges:
◮ How do they interact?
◮ Can they communicate?
◮ Are they cooperating or competing?
◮ Do all robots have identical capabilities?
• Cooperating robots ⇒ decentralized decision theory.
• Non-cooperating robots ⇒ game theory.
PÓLO DO I.S.T 22/26

23. Competing Robot Collectives
Utility: −1 + 10 = 9
Utility: = 0
Utility: 2 ∗ −1 + 20 = 18
+10
+20
PÓLO DO I.S.T 23/26

24. Cooperating Robot Collectives
2∗−1+10+20
Utility: 3
= 9
2∗−1+10+20
Utility: 3
= 9
2∗−1+10+20
Utility: 3
= 9
+10
+20
PÓLO DO I.S.T 24/26

25. Decentralized decision theory at ISR
• Robots have noisy, limited sensors.
• No or limited communication.
?
?
PÓLO DO I.S.T 25/26

26. Conclusions
• In contrast with humans, but we design our robots to be
rational.
• Maximizing expected utility ⇒ rational behavior.
• In reality, we can only achieve limited rationality.
• Robot societies can be designed to maximize social welfare.
• Human societies, on the other hand. . .
PÓLO DO I.S.T 26/26