Dynamics

On the Convergence of Regret Minimizing Dynamics in Concave Games Joint work with Eyal Even Dar , Yishay Mansour Microsoft Research, Cambridge UK, March 26, 2009 Uri Nadav Tel Aviv University, Tel Aviv Israel

Nash Equilibrium ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Player I Player II 0 ,0 4 ,1 1 ,4 3 ,3

Dynamics Player 1: Player 2: Day 1 Day 2 Day 3 Day 4 Day 5 0,0 4,1 1,4 3,3

Example Dynamics ,[object Object],[object Object],[object Object],Player 1: Player 2: Day 1 Day 2 Day 3 Day 4 Day 5 Unfortunately, does not always converge to equilibrium 0,0 4,1 1,4 3,3

No External Regret ,[object Object],[object Object],( total cost of best fixed row in hindsight ) ( total cost of alg ) - Regret Alg ( T ) = A procedure is “without external regret” if for every sequence the external regret is sublinear in T ,[object Object],4 Alg Cost Weather 3 Umbrella 4 No umbrella 0 1 1 0

Our Main Result ,[object Object],Resource Allocation Cournot Oligopoly Socially concave games Selfish Routing TCP Congestion Control

Cournot Oligopoly [Cournot 1838] ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],X Y Cost 1 (X) Cost 2 (Y) We will show no-regret dynamics converges to NE for any number of players Market price Overall quantity X y P

Resource Allocation Games ,[object Object],$5M $10M $17M $25M ,[object Object],‘ s allocated rate = 5+10+17+25 25 ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],We can show that the best response dynamics generally diverges for linear resource allocation games f ( U = ) - $25M

Routing Games ,[object Object],s 1 t 1 t 2 s 2 f 1, L f 1, R f 1,L f 2,T Latency on edge e = L e (f 1,L + f 2,T ) e f 2, T f 2, B ,[object Object],f 2 f 1 ,[object Object]

Socially Concave Games ,[object Object],[object Object],[object Object],R There exists  1 ,…,  n > 0 Such that  1 u 1 (x) +  2 u 2 (x)+…+  n u n (x) Zero Sum Games ½ Socially concave games ,[object Object],[object Object],[object Object],[object Object],[object Object]

Our Main Result ,[object Object],Player 1: Player 2: Player n : Day 1 Day 2 Day 3 Day T Average of days 1… T  - Nash equilibrium ,[object Object],If each players uses a procedure without regret in socially concave games then their joint play converges to Nash equilibrium: - Nash Eq

Convergence to NE – Proof Outline ,[object Object],Utility of player i at average  Utility of i playing Best Response to the average   Definition of  - Nash equilibrium Player 1: Player 2: Player n : Day 1 Day 2 Day 3 Day T Average

Convergence to NE – Proof Outline ,[object Object],For each player i : By definition of Best Response Sum of utilities Utility of average action profile Convexity: The minimum utility is attained when the others play fixed action Holds for every action z

Convergence to NE – Proof Outline ,[object Object],By assumption, there exists  1 ,…,  n such that : is concave Utility of average action profile (Average) Sum of utilities Concavity: The sum of utilities at average is greater than the average utility

Convergence to NE – Proof Outline ,[object Object],Upper Bound = Lower Bound + Average Regret Upper Bound ¸ ¸ Lower Bound Q.E.D

Convergence in Almost Socially Concave Games ,[object Object],[object Object],[object Object],[object Object],Therefore, the convergence theorem cannot be directly applied Playing gradient based dynamics, guarantees “playing” in a “socially concave zone” Playing gradient based dynamics, guarantees “no regret” in concave decision making [Zinkevich]

Regret Minimization & Equilibrium ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Ongoing Research I ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Ongoing Research II ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Other stuff I work on FIFO ,[object Object],[object Object],Slot #1 Slot #2 Slot #3 Slot #4 Slot #5 Slot #6 time ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

TCP Congestion Control [kkps 01 ] User Utility: u i = g i –  i l i f i g i l i User action: push flow f i good-put: fraction of f i forwarded loss: l i = fraction of f i discard Channel Fraction of good-put determined by router policy  i : associated cost with lost flow retransmission, lost bandwidth, utilization

Router Policy ,[object Object],[object Object],[object Object],[object Object],Amount of flow to discard depends on the total amount of flow ,[object Object]

Nash Equilibrium ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Player I: Player II ½ ½ ½ ½ 0 ,0 4 ,1 1 ,4 3 ,3

Dynamics

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Dynamics

Semelhante a Dynamics (20)

Dynamics

Notas do Editor