Document Sample

The Design of Desired Collectives with Multi-Agent Simulation Akira Namatame Dept. of Computer Science National Defense Academy, Japan nama@nda.ac.jp Collectives of Interacting Agents Collective of interacting agents is complex Collective with the following properties: behavior (1) Non-linearity and path- Micro-macro Loop dependency (2) Self-organization Interaction (3) Emergence Agent (4) Unintended consequence We propose the approach of designing desired collectives with the agent-based simulation 2 Agent’s Behavior Based on the Logic of Minority goal Collective preferenc Agent e interest (1) Purposive decision ▲Decision based on preference or interest (2) Contingent decision ▲Decision based on what others are doing <Logic of minority> Agents gain if they take the same action as minority does. 3 Highlights of The Talk <The inverse and forward problems> Characterize the inverse and forward problems to self-organize desired collectives <The interactive design with an agent-based model > Propose the interactive design Desired collectives with multi-agent simulation. Inverse problem Emergent behavior Forward problem Interacting agents with micro-motives 4 Logic of Minority: Symmetric Problem (1) • At each time step,agents make a •Minority games binary choice : Agents on the •El Farol bar problem minority side get more payoffs than those who are the majority side. Resource Payoff U(S1)=a(1-n/N) s U(S2)=b(n/N) a U (S1) U (S2 ) Payoff Resource 0 Resource 1 a U (S1) U (S2 ) b 0 0.5 1 n/ N a / (a b) Agents 0 1 n/ N 5 Logic of Minority: Asymmetric Problems •Congestion problem Time(minu S1 : use a car S2 : use a train te) Goal t A 20 ( n ) 10 2000 By car (bridge) Start Route A Route B By train 40 tB 40 20 payoff=benefit - time •Market entry games 0 200 400 Number of Payoff 0 0 passengers (n) Market System optimal User equilibrium a b Agents 0 a / 2(a b) a /(a b) 1 n/ N 6 Reasons for Undesirable Outcomes <Most common observation > (1) Bounded rationality of agents (2) Inconsistency between individual rationality and group rationality <Points of this talk > (1) Agents behave with false rules How do agents learn desirable rules? (2) Agents behave with inappropriate utility functions. How do agent should modify their own utility functions? 7 Symmetric Problem vs. Asymmetric Problem (1) Nash equilibrium: U(S1)= U(S2) (2) Pareto optimal: Average utility is maximized Payoff Payoff <Symmetric problem> <Asymmetric problem> a U (S1) a U (S2 ) b b 0 a / 2(a b) a /(a b) 1 n/ N 0 0.5 a / (a b) 1 n/ N Average utility E=pU(S1)+(1-p)U(S2) Average utility E=pU(S1)+(1-p)U(S2) =(a+b)(p-p2) =a(p-p2)+b Average utility is maximized at p=0.5 Average utility is maximized at p=a/2(a+b) 8 Decomposition to Pair-wise Problems Collective Decision Pair-wise Decisions qi The payoff matrix of an agent qN-2 q2 qN-1 Collectives S S q3 1 2 Agent Ai (n/N) (1-n/N) (1) Symmetric problem U(S1)= a(1-n/N) S1 0 q U(S2)= b(n/N) S2 1-q 0 q=a/(a+b) The payoff matrix of an agent (2)Asymmetric problem Collectives S 1 S 2 U(S1)= a(1-n/N) Agent Ai (n/N) (1-n/N) U(S2)= b S1 0 1 S2 1-q 1-q 9 Desirable Collective: Stability, Efficiency, Fairness Stability: Desirable collective need to be equilibrium of underlying games Efficiency: Desirable collective need to be efficient of underlying games Fairness Since there are many equilibria, the criteria of stability and efficiency are not enough, and fairness is evolution’s solution to the equilibrium selection problem UB Underlying asymmetric games Pure Nash Equilibrium (Payoff is not equal) Other agent 1 S1 S2 Agent S1 0 （1-θ） 1-q 0 1 S2 1 （1-θ） ( S1 : q , S2 : 1)- q （1-θ） （1-θ） 0 1-q 1 UA 10 Characterization of Learning Models (1) Learning models without coupling with others • Reinforcement learning ▲ Agents reinforce the strategy which gains the payoff • Evolutionary learning ▲ Agents evolve strategy of interaction (2) Learning models with coupling • Best-response learning ▲ Agents adapt based on the best-response strategy 11 Agents Make Choices without Coupling Most common learning model in minority games history 1 2 3 4 5 6 7 8 9 There is no coupling Status of collective 0 1 1 0 1 0 1 0 0 Memory of last m=3 history agent has several randomly Strategy generated strategies of memory m. •A each step, the player uses the strategy that would have maximized Next step, its gains over the entire history. choose 0 12 Coupling of Agents (1) Coupling with collectives Decision (2) Coupling with neighbors Information Action gathering collectives q1 q6 q3 qn-2 qn-1 qj qi q2 q4 q5 qn 13 Coupling Rule between Two Agents make choice based on the past two history actions past t-2 t-1 next Coupling rule pattern No. own opp own opp action between agents #1 0 0 0 1 # #2 0 0 1 0 # #3 0 0 1 1 # Decision Decision #4 0 1 0 0 # #5 0 1 0 1 # Information Information #6 0 1 1 0 # Action Action gathering gathering #7 0 1 1 1 # #8 1 0 0 0 # Symmetric games #9 1 0 0 1 # #10 1 0 1 0 # Other agent S S2 1 #11 1 0 1 1 # Agent Ai (n/N) (1-n/N) #12 1 1 0 0 # #13 1 1 0 1 # S1 0 q #14 1 1 1 0 # S2 1-q 0 #15 #16 1 0 1 0 1 0 1 0 # # 14 (# represents 0 or 1) The Performances of Evolutional Learning Noise=0% Max Ave Min Noise=5% Max Ave Min 15 What Agents Acquired with Evolutinary Learning ? 400 agents with different rules at the beginning evolved to share one of 15 coupling rules. The number of agents 16 Commonality of Acquired Rules The 15 meta-rules shared by all agents have the commonality 17 Coupling with Local Neighbors N N …相互作用（ 対戦） ) t( ip : The proportion of neighbors to choose1 S The behavioral rule as give-and-tale S )1 t( ia )niaG( 1S ) t( ia ,5.0 ) t( ip 2 S )1 t( ia )niaG( 2S ) t( ia ,5.0 ) t( ip 1 18 Simulation Results • Efficient and equitable dynamic orders are emerged with give-and-take S1 S2 Initial configuration (random) Neighborhood size = 4 Neighborhood size = 8 19 Coupling Agents with Collectives (1)The action variable of agent Ai , a1(t) = 1 : S1 (Go) a1(t) = 0 : S2 (Stay) (2)The Status of the Bar Coupling between agents and collective(ｆield) (t ) 1 The bar is crowded at time t (t ) 0 The bar is not crowded at time t (3) Rules of give-and-take <Gain> 0 )1 t( ia )1 ) t( ia( )0 ) t( ( ★If gain, then yields, ((t ) 1) (ai (t ) 0) ai (t 1) 1 if no gain, chooses randomly <No gain> ((t ) 1) (ai (t ) 1) ai (t 1) random ( (t ) 0) (ai (t ) 0) ai (t 1) random 20 Simulation Results （θ=0.5） All agents choose Nash strategies Payoff distribution Blue line;S1, Red line;S2 Give & Take Learning 21 Efficient Utilization of Limited Resource with Too Many Contestants •Market entry games The capacity of resource: q •El Farol bar problem The capacity of resource: q/2 Payoff 0 q /2 q 1 n/ N • How limited resource is maximally utilized under an Agents efficient and equitable situation? 22 How to Solve Inverse Problem? Desired (1)Design right behavioral rules Collective Interacting agents need to develop right Forward and behavioral rules for desirable collectives inverse problems Interaction (2) Design right utility functions Agents need to modify their endogenous Agent utility functions for desirable collectives. 23 Exogenous Design with Subsidy or Tax How should utility functions be redesigned with subsidy or tax? <Asymmetric problem> Nash equilibrium: n/N=q U(S1)=1-n/N U(S2)=1-q Pareto-optimal: n/N= q/2 U(S1)=1-n/N – (n/N)q/(2-q) Payoff Payoff U(S2)=1-q q/2 (n/N)q/(2-q): Tax Payoff q/2: Subsidy 1 1 - 2(1 - q ) /(2 - q ) 1-q / 2 1 -q 1 -q 0 q /2 q 1 n/ N 0 q /2 1 n/ N 24 Endogenous Design with Give&Take N 2(t ) The number of agent who stayed at time t (Case 1) N 2(t ) Nq • Agents who chose S2(Stay), choose S1(Enter) • A part of agents who chose S1(Enter), choose S1(Enter) again. x ( Nq - N 2(t )) / N 1(t ) Agents (Case 2) N 2(t ) Nq The capacity of resource • Agents who chose S1(enter), choose (bar) : Nq S2(stay) • A part of agents who chose S2(stay) choose S2(stay) again y Nq / N 2(t ) 25 Evolutionary Design with Agent-Based Simulation Solving Inverse Problems with Agent-based Simulation (ABS) ABS with coupling Co-evolution of coupling rules Evaluation of emergent collective Implementation of agents with designed coupling rules 26 Conclusion: Achieving Desired Collectives We showed that collective behavior with the logic of minority is much complex than that with the logic of majority! It is detrimental for large trait agents to behave with proper coupling rule in order to achieve efficient and equitable collective: Give-and-take Agents need to evolve their utility functions (internal models) for achieving desired collective 27

DOCUMENT INFO

Shared By:

Categories:

Tags:

Stats:

views: | 3 |

posted: | 2/3/2013 |

language: | Unknown |

pages: | 27 |

OTHER DOCS BY dfhdhdhdhjr

How are you planning on using Docstoc?
BUSINESS
PERSONAL

By registering with docstoc.com you agree to our
privacy policy and
terms of service, and to receive content and offer notifications.

Docstoc is the premier online destination to start and grow small businesses. It hosts the best quality and widest selection of professional documents (over 20 million) and resources including expert videos, articles and productivity tools to make every small business better.

Search or Browse for any specific document or resource you need for your business. Or explore our curated resources for Starting a Business, Growing a Business or for Professional Development.

Feel free to Contact Us with any questions you might have.