集合行為の効率性と公平性

Document Sample
集合行為の効率性と公平性 Powered By Docstoc
					The Design of Desired Collectives with
      Multi-Agent Simulation


             Akira Namatame
        Dept. of Computer Science
     National Defense Academy, Japan
              nama@nda.ac.jp
           Collectives of Interacting Agents

Collective of interacting agents is complex Collective
  with the following properties:            behavior
(1) Non-linearity and path-
                                           Micro-macro Loop
   dependency
(2) Self-organization                         Interaction

(3) Emergence
                                                 Agent
(4) Unintended consequence

  We propose the approach of designing desired collectives
  with the agent-based simulation
                                                              2
                 Agent’s Behavior Based
                 on the Logic of Minority
     goal
                                                    Collective
   preferenc
               Agent
       e

    interest
                  (1) Purposive decision
                       ▲Decision based on preference or interest
                  (2) Contingent decision
                       ▲Decision based on what others are doing
<Logic of minority>
  Agents gain if they take the same action as minority does.
                                                                   3
                     Highlights of The Talk
<The inverse and forward problems>
 Characterize the inverse and forward problems to
self-organize desired collectives
<The interactive design with an agent-based model >
Propose the interactive design             Desired collectives
with multi-agent simulation.

                                 Inverse problem Emergent
                                                 behavior
                                                               Forward problem




                                     Interacting agents with micro-motives   4
        Logic of Minority: Symmetric Problem (1)
                                     •    At each time step,agents make a
  •Minority games                         binary choice : Agents on the

  •El Farol bar problem                   minority side get more payoffs than
                                          those who are the majority side.
      Resource              Payoff

                                                                        U(S1)=a(1-n/N)
         s                                                              U(S2)=b(n/N)
                              a
                                         U (S1)         U (S2 )
                                                                        Payoff

Resource 0     Resource 1
                                                                         a       U (S1)
                                                                                                        U (S2 )
                                                                         b
                               0                  0.5             1   n/ N




                                                                                          a / (a  b)
             Agents                                                          0                           1   n/ N

                                                                                                                  5
          Logic of Minority: Asymmetric Problems
•Congestion problem          Time(minu                    S1 : use a car S2 : use a train
                             te)
                            Goal       t A  20  ( n        )  10
                                                         2000
          By car (bridge)
  Start                                        Route A
                                     Route B
              By train       40
                                                      tB    40

                             20                                       payoff=benefit - time
•Market entry     games        0       200     400      Number of
                                                                      Payoff


                                       0       0         passengers
                                                        (n)
    Market                         System optimal User equilibrium
                                                                      a



                                                                      b



            Agents                                                     0       a / 2(a  b) a /(a  b)   1   n/ N   6
       Reasons for Undesirable Outcomes
<Most common observation >
(1) Bounded rationality of agents
(2) Inconsistency between individual rationality and group
   rationality

<Points of this talk >
 (1) Agents behave with false rules
   How do agents learn desirable rules?
 (2) Agents behave with inappropriate utility functions.
   How do agent should modify their own utility
    functions?
                                                             7
     Symmetric Problem vs. Asymmetric Problem
(1) Nash equilibrium: U(S1)=
                      U(S2)
(2) Pareto optimal: Average utility is maximized
   Payoff                                         Payoff

              <Symmetric problem>                            <Asymmetric problem>

     a      U (S1)                               a
                                 U (S2 )
     b

                                                 b



                                                  0        a / 2(a  b) a /(a  b)   1   n/ N
     0              0.5 a / (a  b) 1 n/ N
Average utility E=pU(S1)+(1-p)U(S2)          Average utility E=pU(S1)+(1-p)U(S2)
                  =(a+b)(p-p2)                                 =a(p-p2)+b
Average utility is maximized at p=0.5        Average utility is maximized at p=a/2(a+b)
                                                                                                8
               Decomposition to Pair-wise Problems

        Collective Decision                      Pair-wise Decisions



   qi                                              The payoff matrix of an agent
                       qN-2
          q2
                              qN-1                  Collectives    S          S
                      q3                                               1          2
                                                 Agent Ai         (n/N)        (1-n/N)
(1) Symmetric problem
U(S1)= a(1-n/N)
                                                      S1               0         q
U(S2)= b(n/N)                                         S2           1-q           0
                                     q=a/(a+b)      The payoff matrix of an agent
(2)Asymmetric problem                                Collectives    S      1   S     2
  U(S1)= a(1-n/N)                                 Agent Ai         (n/N)       (1-n/N)
  U(S2)= b
                                                       S1              0          1
                                                       S2              1-q       1-q     9
        Desirable Collective: Stability, Efficiency, Fairness

Stability:
  Desirable collective need to be equilibrium of underlying games
Efficiency:
  Desirable collective need to be efficient of underlying games
Fairness
  Since there are many equilibria, the criteria of stability and
 efficiency are not enough, and fairness is evolution’s solution to
 the equilibrium selection problem
                                                 UB
        Underlying asymmetric games                         Pure Nash Equilibrium
                                                              (Payoff is not equal)
    Other agent                                   1
                       S1           S2
Agent


          S1                0       (1-θ)       1-q
                   0            1
          S2                1           (1-θ)                          (
                                                                           S1 : q , S2 : 1)- q
                  (1-θ)         (1-θ)
                                                  0   1-q    1       UA
                                                                                                 10
     Characterization of Learning Models

(1) Learning models without coupling with others
• Reinforcement learning
    ▲   Agents reinforce the strategy which gains the payoff
•   Evolutionary learning
    ▲   Agents evolve strategy of interaction

(2) Learning models with coupling
• Best-response learning
    ▲   Agents adapt based on the best-response strategy




                                                               11
        Agents Make Choices without Coupling

    Most common learning model in minority games
                      history           1   2 3 4 5 6 7 8 9
There is no
 coupling       Status of collective    0   1 1 0 1 0 1 0 0


                                   Memory of last     m=3
                                      history

         agent has several randomly                 Strategy
   generated strategies of memory m.
     •A each step, the player uses the
   strategy that would have maximized               Next step,
      its gains over the entire history.            choose 0

                                                                 12
                           Coupling of Agents

(1) Coupling with collectives
                                                                 Decision
(2) Coupling with neighbors
                                                   Information
                                                                            Action
                                                   gathering




                 collectives



                 q1        q6   q3   qn-2   qn-1
                                qj
       qi   q2
                 q4   q5        qn

                                                                                     13
                                Coupling Rule between Two
 Agents make choice based on the past two history actions
                                                past
                                                                                  t-2               t-1            next
         Coupling rule                                        pattern No.   own         opp   own         opp     action

         between agents
                                                                  #1         0          0      0          1         #
                                                                  #2         0          0      1          0         #
                                                                  #3         0          0      1          1         #
              Decision                   Decision
                                                                  #4         0          1      0          0         #
                                                                  #5         0          1      0          1         #
Information              Information                              #6         0          1      1          0         #
                          Action                     Action
gathering                gathering
                                                                  #7         0          1      1          1         #
                                                                  #8         1          0      0          0         #

                  Symmetric games                                 #9         1          0      0          1         #
                                                                 #10         1          0      1          0         #
              Other agent   S                         S2
                                         1                       #11         1          0      1          1         #

         Agent Ai                      (n/N)        (1-n/N)      #12         1          1      0          0         #
                                                                 #13         1          1      0          1         #
                S1                      0              q         #14         1          1      1          0         #

                S2                     1-q             0         #15
                                                                 #16
                                                                             1
                                                                             0
                                                                                        1
                                                                                        0
                                                                                               1
                                                                                               0
                                                                                                          1
                                                                                                          0
                                                                                                                    #
                                                                                                                    #
                                                                                                                        14
                                                                                                     (# represents 0 or 1)
The Performances of Evolutional Learning

                              Noise=0%
                 Max
    Ave
                  Min




                              Noise=5%

     Max
                        Ave
           Min



                                           15
  What Agents Acquired with Evolutinary Learning ?

400 agents with different rules at the beginning evolved to share one of
  15 coupling rules.                          The number of agents




                                                                       16
          Commonality of Acquired Rules
The 15 meta-rules shared by all agents have the commonality




                                                              17
                Coupling with Local Neighbors

                N



  N




                                       …相互作用( 対戦)
      ) t( ip
                : The proportion of neighbors to choose1
                                                      S

The behavioral rule as give-and-tale
       S  )1  t( ia  )niaG( 1S  ) t( ia ,5.0  ) t( ip   2


       S  )1  t( ia  )niaG( 2S  ) t( ia ,5.0  ) t( ip 1


                                                                 18
                   Simulation Results

• Efficient and equitable
                        dynamic orders are
  emerged with give-and-take
                                                                  S1
                                                                  S2


                           Initial configuration
                                 (random)




   Neighborhood size = 4                       Neighborhood size = 8
                                                                       19
                   Coupling Agents with Collectives
(1)The action variable of agent Ai ,
      a1(t) = 1 : S1 (Go)
      a1(t) = 0 : S2 (Stay)
(2)The Status of the Bar
                                                    Coupling between agents and collective(field)
    (t )  1 The bar is crowded at time t
   (t )  0 The bar is not crowded at time t

(3) Rules of give-and-take
    <Gain>
0  )1  t( ia  )1  ) t( ia( )0  ) t( (   ★If gain, then yields,
((t )  1) (ai (t )  0)  ai (t  1)  1 if no gain, chooses randomly
   <No gain>
((t )  1) (ai (t )  1)  ai (t  1)  random
( (t )  0) (ai (t )  0)  ai (t  1)  random                                                   20
                Simulation Results (θ=0.5)
All agents choose Nash strategies
                                          Payoff distribution
              Blue line;S1, Red line;S2




Give & Take Learning




                                                                21
      Efficient Utilization of Limited Resource
             with Too Many Contestants
•Market entry games      The capacity of resource: q
•El Farol bar problem    The capacity of resource: q/2
                            Payoff




                            0        q /2 q   1   n/ N


                        • How limited resource is
                          maximally utilized under an
        Agents            efficient and equitable situation?
                                                               22
                How to Solve Inverse Problem?

                                                   Desired
(1)Design right behavioral rules                  Collective
  Interacting agents need to develop right
                                               Forward and
behavioral rules for desirable collectives     inverse problems
                                                  Interaction
(2) Design right utility functions
  Agents need to modify their endogenous             Agent
utility functions for desirable collectives.




                                                                  23
              Exogenous Design with Subsidy or Tax
How should utility functions be redesigned with subsidy or tax?
    <Asymmetric problem>
                                    Nash equilibrium: n/N=q
    U(S1)=1-n/N
    U(S2)=1-q                       Pareto-optimal: n/N= q/2
                                    U(S1)=1-n/N – (n/N)q/(2-q)
                                           Payoff
     Payoff
                U(S2)=1-q  q/2                              (n/N)q/(2-q): Tax
                                                     Payoff

                q/2: Subsidy
     1                                               1
                                                              -
                                       2(1 - q ) /(2 - q )
1-q / 2

 1 -q                                           1 -q

     0        q /2   q   1   n/ N                     0           q /2    1   n/ N

                                                                                     24
           Endogenous Design with Give&Take
                      N 2(t ) The number of agent who stayed at time t
                             (Case 1) N 2(t )  Nq
                             •   Agents who chose S2(Stay), choose
                                 S1(Enter)
                             •   A part of agents who chose S1(Enter),
                                 choose S1(Enter) again.
                                                   x  ( Nq - N 2(t )) / N 1(t )
          Agents         (Case 2) N 2(t )  Nq
The capacity of resource • Agents who chose S1(enter), choose
(bar) : Nq                  S2(stay)
                         • A part of agents who chose S2(stay)
                            choose S2(stay) again
                                                  y  Nq / N 2(t )
                                                                             25
   Evolutionary Design with
   Agent-Based Simulation
          Solving Inverse Problems with
          Agent-based Simulation (ABS)

  ABS with coupling

                               Co-evolution of
                               coupling rules
 Evaluation of
 emergent collective


Implementation of agents
with designed coupling rules


                                                 26
       Conclusion: Achieving Desired Collectives

We showed that collective behavior with the logic
of minority is much complex than that with the logic
of majority!

 It is detrimental for large trait agents
to behave with proper coupling rule in order
to achieve efficient and equitable collective:
Give-and-take
 Agents need to evolve their utility functions
(internal models) for achieving desired collective


                                                       27

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:3
posted:2/3/2013
language:Unknown
pages:27