# 集合行為の効率性と公平性

Document Sample

```					The Design of Desired Collectives with
Multi-Agent Simulation

Akira Namatame
Dept. of Computer Science
nama@nda.ac.jp
Collectives of Interacting Agents

Collective of interacting agents is complex Collective
with the following properties:            behavior
(1) Non-linearity and path-
Micro-macro Loop
dependency
(2) Self-organization                         Interaction

(3) Emergence
Agent
(4) Unintended consequence

We propose the approach of designing desired collectives
with the agent-based simulation
2
Agent’s Behavior Based
on the Logic of Minority
goal
Collective
preferenc
Agent
e

interest
(1) Purposive decision
▲Decision based on preference or interest
(2) Contingent decision
▲Decision based on what others are doing
<Logic of minority>
Agents gain if they take the same action as minority does.
3
Highlights of The Talk
<The inverse and forward problems>
Characterize the inverse and forward problems to
self-organize desired collectives
<The interactive design with an agent-based model >
Propose the interactive design             Desired collectives
with multi-agent simulation.

Inverse problem Emergent
behavior
Forward problem

Interacting agents with micro-motives   4
Logic of Minority: Symmetric Problem (1)
•    At each time step,agents make a
•Minority games                         binary choice : Agents on the

•El Farol bar problem                   minority side get more payoffs than
those who are the majority side.
Resource              Payoff

U(S1)=a(1-n/N)
s                                                              U(S2)=b(n/N)
a
U (S1)         U (S2 )
Payoff

Resource 0     Resource 1
a       U (S1)
U (S2 )
b
0                  0.5             1   n/ N

a / (a  b)
Agents                                                          0                           1   n/ N

5
Logic of Minority: Asymmetric Problems
•Congestion problem          Time(minu                    S1 : use a car S2 : use a train
te)
Goal       t A  20  ( n        )  10
2000
By car (bridge)
Start                                        Route A
Route B
By train       40
tB    40

20                                       payoff=benefit - time
•Market entry     games        0       200     400      Number of
Payoff

0       0         passengers
(n)
Market                         System optimal User equilibrium
a

b

Agents                                                     0       a / 2(a  b) a /(a  b)   1   n/ N   6
Reasons for Undesirable Outcomes
<Most common observation >
(1) Bounded rationality of agents
(2) Inconsistency between individual rationality and group
rationality

<Points of this talk >
(1) Agents behave with false rules
How do agents learn desirable rules?
(2) Agents behave with inappropriate utility functions.
How do agent should modify their own utility
functions?
7
Symmetric Problem vs. Asymmetric Problem
(1) Nash equilibrium: U(S1)=
U(S2)
(2) Pareto optimal: Average utility is maximized
Payoff                                         Payoff

<Symmetric problem>                            <Asymmetric problem>

a      U (S1)                               a
U (S2 )
b

b

0        a / 2(a  b) a /(a  b)   1   n/ N
0              0.5 a / (a  b) 1 n/ N
Average utility E=pU(S1)+(1-p)U(S2)          Average utility E=pU(S1)+(1-p)U(S2)
=(a+b)(p-p2)                                 =a(p-p2)+b
Average utility is maximized at p=0.5        Average utility is maximized at p=a/2(a+b)
8
Decomposition to Pair-wise Problems

Collective Decision                      Pair-wise Decisions

qi                                              The payoff matrix of an agent
qN-2
q2
qN-1                  Collectives    S          S
q3                                               1          2
Agent Ai         (n/N)        (1-n/N)
(1) Symmetric problem
U(S1)= a(1-n/N)
S1               0         q
U(S2)= b(n/N)                                         S2           1-q           0
q=a/(a+b)      The payoff matrix of an agent
(2)Asymmetric problem                                Collectives    S      1   S     2
U(S1)= a(1-n/N)                                 Agent Ai         (n/N)       (1-n/N)
U(S2)= b
S1              0          1
S2              1-q       1-q     9
Desirable Collective: Stability, Efficiency, Fairness

Stability:
Desirable collective need to be equilibrium of underlying games
Efficiency:
Desirable collective need to be efficient of underlying games
Fairness
Since there are many equilibria, the criteria of stability and
efficiency are not enough, and fairness is evolution’s solution to
the equilibrium selection problem
UB
Underlying asymmetric games                         Pure Nash Equilibrium
(Payoff is not equal)
Other agent                                   1
S1           S2
Agent

S1                0       （1-θ）       1-q
0            1
S2                1           （1-θ）                          (
S1 : q , S2 : 1)- q
（1-θ）         （1-θ）
0   1-q    1       UA
10
Characterization of Learning Models

(1) Learning models without coupling with others
• Reinforcement learning
▲   Agents reinforce the strategy which gains the payoff
•   Evolutionary learning
▲   Agents evolve strategy of interaction

(2) Learning models with coupling
• Best-response learning
▲   Agents adapt based on the best-response strategy

11
Agents Make Choices without Coupling

Most common learning model in minority games
history           1   2 3 4 5 6 7 8 9
There is no
coupling       Status of collective    0   1 1 0 1 0 1 0 0

Memory of last     m=3
history

agent has several randomly                 Strategy
generated strategies of memory m.
•A each step, the player uses the
strategy that would have maximized               Next step,
its gains over the entire history.            choose 0

12
Coupling of Agents

(1) Coupling with collectives
Decision
(2) Coupling with neighbors
Information
Action
gathering

collectives

q1        q6   q3   qn-2   qn-1
qj
qi   q2
q4   q5        qn

13
Coupling Rule between Two
Agents make choice based on the past two history actions
past
t-2               t-1            next
Coupling rule                                        pattern No.   own         opp   own         opp     action

between agents
#1         0          0      0          1         #
#2         0          0      1          0         #
#3         0          0      1          1         #
Decision                   Decision
#4         0          1      0          0         #
#5         0          1      0          1         #
Information              Information                              #6         0          1      1          0         #
Action                     Action
gathering                gathering
#7         0          1      1          1         #
#8         1          0      0          0         #

Symmetric games                                 #9         1          0      0          1         #
#10         1          0      1          0         #
Other agent   S                         S2
1                       #11         1          0      1          1         #

Agent Ai                      (n/N)        (1-n/N)      #12         1          1      0          0         #
#13         1          1      0          1         #
S1                      0              q         #14         1          1      1          0         #

S2                     1-q             0         #15
#16
1
0
1
0
1
0
1
0
#
#
14
(# represents 0 or 1)
The Performances of Evolutional Learning

Noise=0%
Max
Ave
Min

Noise=5%

Max
Ave
Min

15
What Agents Acquired with Evolutinary Learning ?

400 agents with different rules at the beginning evolved to share one of
15 coupling rules.                          The number of agents

16
Commonality of Acquired Rules
The 15 meta-rules shared by all agents have the commonality

17
Coupling with Local Neighbors

N

N

…相互作用（ 対戦）
) t( ip
: The proportion of neighbors to choose1
S

The behavioral rule as give-and-tale
S  )1  t( ia  )niaG( 1S  ) t( ia ,5.0  ) t( ip   2

S  )1  t( ia  )niaG( 2S  ) t( ia ,5.0  ) t( ip 1

18
Simulation Results

• Efficient and equitable
dynamic orders are
emerged with give-and-take
S1
S2

Initial configuration
(random)

Neighborhood size = 4                       Neighborhood size = 8
19
Coupling Agents with Collectives
(1)The action variable of agent Ai ,
a1(t) = 1 : S1 (Go)
a1(t) = 0 : S2 (Stay)
(2)The Status of the Bar
Coupling between agents and collective(ｆield)
 (t )  1 The bar is crowded at time t
(t )  0 The bar is not crowded at time t

(3) Rules of give-and-take
<Gain>
0  )1  t( ia  )1  ) t( ia( )0  ) t( (   ★If gain, then yields,
((t )  1) (ai (t )  0)  ai (t  1)  1 if no gain, chooses randomly
<No gain>
((t )  1) (ai (t )  1)  ai (t  1)  random
( (t )  0) (ai (t )  0)  ai (t  1)  random                                                   20
Simulation Results （θ=0.5）
All agents choose Nash strategies
Payoff distribution
Blue line;S1, Red line;S2

Give & Take Learning

21
Efficient Utilization of Limited Resource
with Too Many Contestants
•Market entry games      The capacity of resource: q
•El Farol bar problem    The capacity of resource: q/2
Payoff

0        q /2 q   1   n/ N

• How limited resource is
maximally utilized under an
Agents            efficient and equitable situation?
22
How to Solve Inverse Problem?

Desired
(1)Design right behavioral rules                  Collective
Interacting agents need to develop right
Forward and
behavioral rules for desirable collectives     inverse problems
Interaction
(2) Design right utility functions
Agents need to modify their endogenous             Agent
utility functions for desirable collectives.

23
Exogenous Design with Subsidy or Tax
How should utility functions be redesigned with subsidy or tax?
<Asymmetric problem>
Nash equilibrium: n/N=q
U(S1)=1-n/N
U(S2)=1-q                       Pareto-optimal: n/N= q/2
U(S1)=1-n/N – (n/N)q/(2-q)
Payoff
Payoff
U(S2)=1-q  q/2                              (n/N)q/(2-q): Tax
Payoff

q/2: Subsidy
1                                               1
-
2(1 - q ) /(2 - q )
1-q / 2

1 -q                                           1 -q

0        q /2   q   1   n/ N                     0           q /2    1   n/ N

24
Endogenous Design with Give&Take
N 2(t ) The number of agent who stayed at time t
(Case 1) N 2(t )  Nq
•   Agents who chose S2(Stay), choose
S1(Enter)
•   A part of agents who chose S1(Enter),
choose S1(Enter) again.
x  ( Nq - N 2(t )) / N 1(t )
Agents         (Case 2) N 2(t )  Nq
The capacity of resource • Agents who chose S1(enter), choose
(bar) : Nq                  S2(stay)
• A part of agents who chose S2(stay)
choose S2(stay) again
y  Nq / N 2(t )
25
Evolutionary Design with
Agent-Based Simulation
Solving Inverse Problems with
Agent-based Simulation (ABS)

ABS with coupling

Co-evolution of
coupling rules
Evaluation of
emergent collective

Implementation of agents
with designed coupling rules

26
Conclusion: Achieving Desired Collectives

We showed that collective behavior with the logic
of minority is much complex than that with the logic
of majority!

 It is detrimental for large trait agents
to behave with proper coupling rule in order
to achieve efficient and equitable collective:
Give-and-take
 Agents need to evolve their utility functions
(internal models) for achieving desired collective

27

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 3 posted: 2/3/2013 language: Unknown pages: 27