# Selfish Routing and the Price of Anarchy - Princeton University

Document Sample

```					    Approximation in
Algorithmic Game Theory
Robust Approximation Bounds for
Equilibria and Auctions

Tim Roughgarden
Stanford University
1
Motivation
Clearly: many modern applications in CS involve
autonomous, self-interested agents
– motivates noncooperative games as modeling tool

Unsurprising fact: this often makes full
optimality hard/impossible.
– equilibria (e.g., Nash) of noncooperative games are
typically suboptimal
– auctions lose revenue from strategic behavior
– incentive constraints can make poly-time
approximation of NP-hard problems even harder
2
Approximation in AGT
• The Price of Anarchy (etc.)
– worst-case approximation
guarantees for equilibria                 this talk

• Revenue Maximization
– guarantees for auctions in non-Bayesian
settings (information-theoretic)

• Algorithm Mechanism Design                    FOCS 2010
– approximation algorithms robust to        tutorial
selfish behavior (computational)

•   Computing Approximate Equilibria
–   e.g., is there a PTAS for computing
an approximate Nash equilibrium?
3
4
Price of Anarchy
quantify inefficiency w.r.t some objective
function.
– e.g., Nash equilibrium: an outcome such that no
player better off by switching strategies
Definition: price of anarchy (POA) of a game
(w.r.t. some objective function):
the closer to 1
equilibrium objective fn value       the better
optimal obj fn value

5
The Price of Anarchy
Network w/2 players:

2x        12
s          0         t
5         5x

6
The Price of Anarchy
Nash Equilibrium:

2x         12
s           0           t
5         5x

cost = 14+14 = 28

7
The Price of Anarchy
Nash Equilibrium:          To Minimize Cost:

2x                            2x          12
12
s           0           t      s          0             t
5          5x                 5           5x

cost = 14+14 = 28              cost = 14+10 = 24

Price of anarchy = 28/24 = 7/6.
• if multiple equilibria exist, look at the worst one
8
The Need for Robustness
Meaning of a POA bound: if the game is at an
equilibrium, then outcome is near-optimal.

9
The Need for Robustness
Meaning of a POA bound: if the game is at an
equilibrium, then outcome is near-optimal.
Problem: what if can’t reach equilibrium?
• (pure) equilibrium might not exist
• might be hard to compute, even centrally
• might be hard to learn in a distributed way
Worry: are our POA bounds “meaningless”?
10
Robust POA Bounds
High-Level Goal: worst-case bounds that
apply even to non-equilibrium outcomes!
• best-response dynamics, pre-convergence
– [Mirrokni/Vetta 04], [Goemans/Mirrokni/Vetta 05],
[Awerbuch/Azar/Epstein/Mirrokni/Skopalik 08]
• correlated equilibria
– [Christodoulou/Koutsoupias 05]
• coarse correlated equilibria aka “price of
total anarchy” aka “no-regret players”
– [Blum/Even-Dar/Ligett 06],
[Blum/Hajiaghayi/Ligett/Roth 08]
11
Abstract Setup
• n players, each picks a strategy si
• player i incurs a cost Ci(s)

Important Assumption: objective function is
cost(s) := i Ci(s)

Key Definition: A game is (λ,μ)-smooth if, for
every pair s,s* outcomes (λ > 0; μ < 1):
i Ci(s*i,s-i) ≤ λ●cost(s*) + μ●cost(s)   [(*)]
12
Smooth => POA Bound
Next: “canonical” way to upper bound POA
(via a smoothness argument).
• notation: s = a Nash eq; s* = optimal
Assuming (λ,μ)-smooth:
cost(s) = i Ci(s)             [defn of cost]
≤ i Ci(s*i,s-i)        [s a Nash eq]
≤ λ●cost(s*) + μ●cost(s)     [(*)]
Then: POA (of pure Nash eq) ≤ λ/(1-μ).
13
Why Is Smoothness Stronger?
Key point: to derive POA bound, only needed
i Ci(s*i,s-i) ≤ λ●cost(s*) + μ●cost(s)       [(*)]
to hold in special case where s = a Nash eq
and s* = optimal.

Smoothness: requires (*) for every pair s,s*
outcomes.
– even if s is not a pure Nash equilibrium

14
Some Smoothness Bounds
• atomic (unweighted) selfish routing
[Awerbuch/Azar/Epstein 05], [Christodoulou/Koutsoupias 05],
[Aland/Dumrauf/Gairing/Monien/Schoppmann 06], [Roughgarden 09]
• nonatomic selfish routing
[Roughgarden/Tardos 00],[Perakis 04] [Correa/Schulz/Stier Moses 05]
• weighted congestion games
[Aland/Dumrauf/Gairing/Monien/Schoppmann 06],
[Bhawalkar/Gairing/Roughgarden 10]
• submodular maximization games
[Vetta 02], [Marden/Roughgarden 10]
• coordination mechanisms
[Cole/Gkatzelis/Mirrokni 10]

15
Beyond Nash Equilibria
Definition: a sequence s1,s2,...,sT of
outcomes is no-regret if:                 no-regret

• for each player i, each                  correlated eq
fixed action qi:                         mixed Nash
– average cost player i incurs
pure
over sequence no worse than
Nash
playing action qi every time
– if every player uses e.g. “multiplicative weights”
then get o(1) regret in poly-time
– empirical distribution = "coarse correlated eq"
16
An Out-of-Equilibrium Bound

Theorem: [Roughgarden STOC 09]
in a (λ,μ)-smooth game, average cost of
every no-regret sequence at most
[λ/(1-μ)] x cost of optimal outcome.

(the same bound we proved for pure Nash equilibria)

17
Smooth => No-Regret Bound
• notation: s1,s2,...,sT = no regret; s* = optimal
Assuming (λ,μ)-smooth:
t cost(st) = t i Ci(st)         [defn of cost]

18
Smooth => No-Regret Bound
• notation: s1,s2,...,sT = no regret; s* = optimal
Assuming (λ,μ)-smooth:
t cost(st) = t i Ci(st)               [defn of cost]
= t i [Ci(s*i,st-i) + ∆i,t]   [∆i,t:= Ci(st)- Ci(s*i,st-i)]

19
Smooth => No-Regret Bound
• notation: s1,s2,...,sT = no regret; s* = optimal
Assuming (λ,μ)-smooth:
t cost(st) = t i Ci(st)               [defn of cost]
= t i [Ci(s*i,st-i) + ∆i,t]   [∆i,t:= Ci(st)- Ci(s*i,st-i)]
≤ t [λ●cost(s*) + μ●cost(st)] + i t ∆i,t [(*)]

20
Smooth => No-Regret Bound
• notation: s1,s2,...,sT = no regret; s* = optimal
Assuming (λ,μ)-smooth:
t cost(st) = t i Ci(st)               [defn of cost]
= t i [Ci(s*i,st-i) + ∆i,t]   [∆i,t:= Ci(st)- Ci(s*i,st-i)]
≤ t [λ●cost(s*) + μ●cost(st)] + i t ∆i,t [(*)]
No regret: t ∆i,t ≤ 0 for each i.
To finish proof: divide through by T.
21
Intrinsic Robustness
Theorem: [Roughgarden STOC 09] for every set C,
unweighted congestion games with cost
functions restricted to C are tight:

maximum [pure POA] = minimum [λ/(1-μ)]
congestion games        (λ ,μ): all such games
w/cost functions in C   are (λ ,μ)-smooth

22
Intrinsic Robustness
Theorem: [Roughgarden STOC 09] for every set C,
unweighted congestion games with cost
functions restricted to C are tight:

maximum [pure POA] = minimum [λ/(1-μ)]
congestion games         (λ ,μ): all such games
w/cost functions in C    are (λ ,μ)-smooth

• weighted congestion games [Bhawalkar/
Gairing/Roughgarden ESA 10] and submodular
maximization games [Marden/Roughgarden CDC
10] are also tight in this sense           23
What's Next?
• beating worst-case POA bounds: want to reach
a non-worst-case equilibrium
– because of learning dynamics [Charikar/Karloff/
Mathieu/Naor/Saks 08], [Kleinberg/Pilouras/Tardos 09], etc.
– from modest intervention [Balcan/Blum/Mansour], etc.

• POA bounds for auctions
– practical auctions often lack "dominant strategies"
– want bounds on their (Bayes-Nash) equilibria
[Christodoulou et al 08], [Paes Leme/Tardos 10],
[Bhawalkar/Roughgarden 11], [Hassadim et al 11]
24
Key Points
• smoothness: a “canonical way” to bound the
price of anarchy (for pure equilibria)
• robust POA bounds: smoothness bounds
extend automatically beyond Nash equilibria
• tightness: smoothness bounds provably give
optimal POA bounds in fundamental cases
• extensions: approximate equilibria; best-
response dynamics; local smoothness for
correlated equilibria; also Bayes-Nash eq   25

26
Competitive Analysis Fails
Observation: which auction (e.g., opening bid) is
best depends on the (unknown) input.
•   e.g., opening bid of \$0.01 or \$10 better?

Competitive analysis: compare your revenue to
that obtained by an omniscient opponent.
Problem: fails miserably in this context.
• predicts that all auctions are equally terrible

• novel analysis framework needed

27
A New Analysis Framework
Prior-independent analysis framework:
[Hartline/Roughgarden STOC 08, EC 09]
compare revenue to that of opponent with

Goal: design a distribution-independent
auction that is always near-optimal for the
underlying distribution (no matter what the
distribution is).
•   distribution over inputs not used in the design of
the auction, only in its analysis
28
Bulow-Klemperer ('96)

Setup: single-item auction. Let F be a known
valuation distribution. [Needs to be "regular".]
Theorem: [Bulow-Klemperer 96]: for every n:
Vickrey's revenue           OPT's revenue

29
Bulow-Klemperer ('96)

Setup: single-item auction. Let F be a known
valuation distribution. [Needs to be "regular".]
Theorem: [Bulow-Klemperer 96]: for every n:
Vickrey's revenue              ≥   OPT's revenue
[with (n+1) i.i.d. bidders]       [with n i.i.d. bidders]

30
Bulow-Klemperer ('96)

Setup: single-item auction. Let F be a known
valuation distribution. [Needs to be "regular".]
Theorem: [Bulow-Klemperer 96]: for every n:
Vickrey's revenue              ≥   OPT's revenue
[with (n+1) i.i.d. bidders]       [with n i.i.d. bidders]

Interpretation: small increase in competition more
important than running optimal auction.
   a "bicriteria bound"!
31
Bayesian Profit Maximization

Example: 1 bidder, 1 item, v ~ known distribution F
  want to choose optimal reserve price p
  expected revenue of p: p(1-F(p))
   given F, can solve for optimal p*
   e.g., p* = ½ for v ~ uniform[0,1]
   but: what about k,n >1 (with i.i.d. vi's)?

32
Bayesian Profit Maximization

Example: 1 bidder, 1 item, v ~ known distribution F
  want to choose optimal reserve price p
need minor
  expected revenue of p: p(1-F(p))         technical
   given F, can solve for optimal p*      conditions
on F
   e.g., p* = ½ for v ~ uniform[0,1]
   but: what about n >1 (with i.i.d. vi's)?

Theorem: [Myerson 81] auction with max expected
revenue is second-price with above reserve p*.
   note p* is independent of n
33
Reformulation of BK Theorem

Theorem: [Bulow-Klemperer 96]: for every n:
Vickrey's revenue    ≥      OPT's revenue
[with (n+1) i.i.d. bidders]       [with n i.i.d. bidders]

Lemma: if true for n=1, then true for all n.
     relevance of OPT reserve price decreases with n

Reformulation for n=1 case:
2 x Vickrey's revenue        Vickrey's revenue
with n=1 and random      ≥   with n=1 and opt
reserve [drawn from F]          reserve r*
34
Proof of BK Theorem

expected
revenue
R(q)

0   selling probability q   1

35
Proof of BK Theorem
concave
if and only if
expected
F is regular
revenue
R(q)

0   selling probability q   1

36
Proof of BK Theorem

expected
revenue
R(q)
q*
0   selling probability q   1

     opt revenue = R(q*)

37
Proof of BK Theorem

expected
revenue
R(q)
q*
0   selling probability q   1

     opt revenue = R(q*)

38
Proof of BK Theorem

expected
revenue
R(q)

0   selling probability q   1

     opt revenue = R(q*)
     revenue of random reserve r (from F) =
expected value of R(q) for q uniform in [0,1] =
area under revenue curve

39
Proof of BK Theorem

expected
revenue
R(q)

0   selling probability q   1

     opt revenue = R(q*)
     revenue of random reserve r (from F) =
expected value of R(q) for q uniform in [0,1] =
area under revenue curve

40
Proof of BK Theorem
concave
if and only if
expected
F is regular
revenue
R(q)
q*
0   selling probability q   1

     opt revenue = R(q*)
     revenue of random reserve r (from F) =
expected value of R(q) for q uniform in [0,1] =
area under revenue curve

41
Proof of BK Theorem
concave
if and only if
expected
F is regular
revenue
R(q)
q*
0   selling probability q   1

     opt revenue = R(q*)
     revenue of random reserve r (from F) =
expected value of R(q) for q uniform in [0,1] =
area under revenue curve ≥ ½ ◦ R(q*)

42
Recent Progress

BK theorem: the "prior-free" Vickrey auction with
extra bidder as good as optimal (w.r.t. F)
mechanism, no matter what F is.
More general "bicriteria bounds":
[Hartline/Roughgarden EC 09],
[Dughmi/Roughgarden/Sundararajan EC 09]

Prior-independent approximations:
[Devanur/Hartline EC 09],
[Dhangwotnotai/Roughgarden/Yan EC 10],
[Hartline/Yan EC 11]
43
What's Next?

Take-home points:
       standard competitive analysis useless for worst-case
revenue maximization
       but can get simultaneous competitive guarantee with all
Bayesian-optimal auctions

Future Directions:
     thoroughly understand “single-parameter”
problems, include non "downward-closed" ones
     non-i.i.d. settings
     multi-parameter? (e.g., combinatorial auctions)
44
Approximation in AGT
• The Price of Anarchy (etc.)
– worst-case approximation
guarantees for equilibria                 this talk

• Revenue Maximization
– guarantees for auctions in non-Bayesian
settings (information-theoretic)

• Algorithm Mechanism Design                    FOCS 2010
– approximation algorithms robust to        tutorial
selfish behavior (computational)

•   Computing Approximate Equilibria
–   e.g., is there a PTAS for computing
an approximate Nash equilibrium?
45
Epilogue
Higher-Level Moral: worst-case approximation
guarantees as powerful "intellectual export" to
other fields (e.g., game theory).
•many reasons for approximation (not just
computational complexity)

46
Epilogue
Higher-Level Moral: worst-case approximation
guarantees as powerful "intellectual export" to
other fields (e.g., game theory).
•many reasons for approximation (not just
computational complexity)

THANKS!

47

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 4 posted: 11/21/2012 language: Unknown pages: 47