EPFL cole Polytechnique Fdrale de Lausanne by sanmelody

VIEWS: 14 PAGES: 50

									A Generic Mean Field Convergence Result
   for Systems of Interacting Objects

               From Micro to Macro

            Jean-Yves Le Boudec, EPFL

    Joint work with David McDonald, U. of Ottawa
            and Jochen Mundinger, EPFL




                                                   1
 The full text of my talk and this slide show are available from my
  web page

  http://people.epfl.ch/jean-yves.leboudec

   Direct access:

   Full text:
     http://infoscience.epfl.ch/getfile.py?recid=108827&mode=best


   Slide show:
      http://icawww1.epfl.ch/PS_files/mean-field-leb-vt07.ppt




                                                                       2
                           Contents

         1. Motivation

2. A Generic Model for a System
      of Interacting Objects

 3. Convergence to the Mean
            Field

      4. Fast Simulation

   5. Full Scale Example: A
       Reputation System

          6. Outlook
                                      E.L.

                                         3
                         Motivation
 Find re-usable approximations of large scale systems

   Examples from my field
     Performance of UWB impulse radio : many sensors, each has a MAC
     layer state
     Ad-Hoc networking
     Reputation Systems



 From microscopic description to macroscopic equations

 Understand fluid approximation and mean field approximation




                                                                       4
                  Example 1 : TCP/ECN
               ECN Feedback q(R(t))

    1

    n              ECN router
                queue length R(t)
    N
        N connections


 TCP connection n transmits at a         When connection n does not
  rate 2 {s0, …, si, …, sI}                receive an ECN, it increases its
 Queue length at router is R(t)           rate:
 With probability q(R(t)) connection          If rate == si, new rate := si+1 (i<I)
  i receives an Explicit Congestion       Else it decreases its rate:
  Notification (ECN) in next time slot         If rate == si, new rate := sd(i)



               The question is the behaviour when N is large

                                                                                       5
                Microscopic Description
 Time is discrete
 Connection n runs one Markov chain XNn(t);



                                                              no ECN received


      ECN received

 The transition probabilities of the Markov chain XNn(t) depend on
  global state R(t) (queue size)

 Global state R(t) depends on states of all connections
      let MNi(t) = nb of connections in state i at time t ,   C = service rate of router




                                                                                           6
               Macroscopic Description
 The fluid approximation is often given as a simplification of the
  previous model




 Combined with




  we have a macroscopic description of the system

 In [17], Tinna. and Makowski show that it holds as large N
  asymptotics
                                                                      7
         The Mean Field Approximation
 Assume we want to analyze one TCP connection in detail
 We can keep the microscopic description for this TCP connection,
  and use the fluid approximation for the others:
  We can call it fast simulation.




 i.e. pretend XN1(t) (one connection) and R(t) (global resource) are
  independent. This is similar to what is called the mean field
  approximation in physics

                                                                        8
Another Example:
  Robot Swarm



                    N robots
                    Robot has S = 2 possible states
                    Transition for one robot depends
                     on this robot’s state + how many
                     other robots are in search state
                    [11] uses the fluid approximation :




                                                           9
                         A few other Examples …

M.-D. Bordenave, D. McDonald and A. Proutière, A
particle system in interaction with a rapidly varying
environment: Mean field limits and applications
arXiv:math/0701363v2




                                                        10
  In these and other examples, some authors
  assume the validity of the fluid / mean field
  approximation and use the approximation to
  do performance evaluation, parameter
  identification, control…                        Never
                                                  again !
     … while, in contrast, others
spend most of the paper proving the
derivation and validity of the
approximations in their specific
setting




   papers in this latter class are intimidating
   cost of proof of one approximation result
   ¼ 1 PhD
   and not re-usable


                                                            11
Can we have answers of general applicability to:



  When are the fluid approximation and the mean field
  approximation valid ?



  Can we write them in a sound ( = mechanical) way ?




                                                        12
                           Contents

         1. Motivation

2. A Generic Model for a System
      of Interacting Objects

 3. Convergence to the Mean
            Field

      4. Fast Simulation

   5. Full Scale Example: A
       Reputation System

          6. Outlook
                                      E.L.

                                        13
          Mean Field Interaction Model
 A Generic Model, with generic results
 Does not cover all useful cases, but is a useful first step

 Time is discrete
 N objects
 Every object has a state in               .



 Informally: object n evolves depending only on
      Its own state
      A global resource whose evolution depends only on how many other
      objects are in each state




                                                                         14
                    Model Assumptions
XNn(t) : state of object n at time t

MNi(t)   = proportion of objects that are in state i
           MN is the “occupancy measure” ¼ the “mean field”

RN(t) = global resource =“history” of occupancy measure




Conditional to history up to time t, objects draws next state
  independent of each other according to




                                                                15
                Two Mild Assumptions
1. Continuity of the integration function g()




2. For large N, the transition matrix K becomes independent of N and
   is continuous




                                                                   16
TCP/ECN Example fits in this Framework
                  ECN Feedback q(R(t))        Function g() :
    1

    n                 ECN router
                   queue length R(t)
    N
          N connections                      thus

 Intuitively satisfies the conditions
        State of one connection depends
        only on buffer content
        Buffer contents depends only on
        how many connections are in each
        state
                                                g() is continuous
 Formally:
                                                Assumption 1 is satisfied
        One object = one TCP connection
        State of one object = index i of
        sending rate
        RN(t) = total buffer occupancy / N

                                                                            17
TCP/ECN Example fits in this Framework
                ECN Feedback q(R(t))      Transition matrix K
  1

 n                 ECN router              Let q(r) = proba of negative
                queue length R(t)          feedback when R==r
  N
      N connections


                       no ECN received



                                           K is independent of N thus
                                           Assumption 2 is is satisfied if q() is
 ECN received                              continuous




                                                                               18
                      A Multiclass Variant
 Take same as previous TCP/ECN      Also fits in our framework
  model but introduce multiclass

 Aggressive connections, normal
  connection                         Mean Field does not mean all
                                      objects are exchangeable !
 State of an object = (c, i)
       c : class
       i : sending rate



 Objects may change class or not




                                                                     19
                           Contents

         1. Motivation

2. A Generic Model for a System
      of Interacting Objects

 3. Convergence to the Mean
            Field

      4. Fast Simulation

   5. Full Scale Example: A
       Reputation System

          6. Outlook
                                      E.L.

                                        20
21
 Practical Application : Derivation of the Fluid
                Approximation
 The theorem replaces the stochastic system by a deterministic,
  dynamical system

 This gives a method to write and justify the fluid approximation in
  the large N regime
      Equation for the limiting occupancy measure  can be rewritten as




      where Ni(t) = N MNi(t) = number of objects in state i at time t


                                                                          22
                   Proof of Theorem
 Based on
    The next theorem (fast simulation)
    A coupling argument
    An ad-hoc version of the strong law of large numbers
    The Glivenko Cantelli lemma




                                                           23
                           Contents

         1. Motivation

2. A Generic Model for a System
      of Interacting Objects

 3. Convergence to the Mean
            Field

      4. Fast Simulation

   5. Full Scale Example: A
       Reputation System

          6. Outlook
                                      E.L.

                                        24
  Fast Simulation / Analysis of One Object
 Assume we are interested in one object in particular
      E.g. distribution of time until a TCP connection reaches maximum rate
 For large N, since mean field convergence holds, one may do the
  mean field approximation and replace the set of other objects by
  the deterministic dynamical system

 The next theorem says that, essentially, this is valid




                                                                          25
Fast Simulation Algorithm

                   State of one specific object




                  Returns next state for one object
                  When transition matrix is K



                 Replace true value by deterministic
                 limit


                  This is the mean field independence
                  approximation



                                                      26
Fast Simulation Result




                         27
                   Practical Application
 This justifies the mean field approximation for the stochastic
  evolution of one object in the large N regime

 Gives a method for fast simulation or analysis
      The state space for Y1 has S states, instead of SN




                                                                   28
                           Contents

         1. Motivation

2. A Generic Model for a System
      of Interacting Objects

 3. Convergence to the Mean
            Field

      4. Fast Simulation

   5. Full Scale Example: A
       Reputation System

          6. Outlook
                                      E.L.

                                        29
                  A Reputation System
 My original motivation for this work
 Illustrates the complete set of steps, including a few modelling
  tricks
 System
      N objects = N peers
      Peers observe one subject and rate it
      Rating is a number in (0,1)
      Direct observations and spreading of reputation
      Confirmation bias + forgetting




                                                                     30
   Operation of Reputation System: Forgetting
 Zn(t) = reputation rating held by peer n

 During a direct observation, subject is perceived as positive (with
  proba ) or negative (with proba 1-)




 In case of direct positive observation



 In case of direct negative observation




 w is the forgetting factor, close to 1 (0.9 in next slides)
                                                                        31
                   Confirmation Bias
 Peer also read other peer ratings
 If overheard rating is z:




  is the threshold of the confirmation bias




                                                32
                 Liars and Honest Peers
 Honest peer does as just explained
 Liar tries to bring the reputation down
      Uses different strategies, see later




                                             33
                      Example of exact simulation: N=100 peers
                        with maximal liars (always say Z=0)
proportion of peers


                                              Initially: peers have Z=0, 0.5 or 1

                                                                   = 0.9



                                              rating




                      Every time step: direct obs p=0.01, meet liar proba 0.30, meet honest proba 0.69
                                                                                                         34
   rating   3 particular peers, one of each type




 = 0.9




                                                   time


                                                     35
Can we study the system with 106
users instead of 100 ?




                                   36
    The problem fits in our framework…
 Assume discrete time
 At every time step a peer
      Makes a direct observation
      Or overhears a liar
      Or overhears some honest peer
      Or does nothing

 Object = honest peer

 Assume first that liars use strategy 1: maximal lying (always say
  Z=0)

 Transition of one honest peer depends on
      Own state
      Distribution of states of all other peers

   => Fits in our framework with memory R = occupancy measure M

                                                                      37
               Different Liar Strategies
 Strategy 1 (maximal lying): liars always say Z= 0
 Strategy 2 (infer): liar guesses your rating based on past
  experience
   Transition of one honest peer depends on
     Own state
     Distribution of states of all other peers
     What liars remember seeing in the past

   => Fits in our framework with memory R = occupancy measure of ratings
     at steps t and t-1

 Strategy 3 (side information): liars know your rating and is as
  negative as you accept
  not realistic but serves as benchmark (worst case)
      Similar to strategy 1, memory = occupancy measure M



                                                                           38
      We would like to apply the mean field
    convergence result to analyze very large N
 But model has continuous state space

 Discretize reputation ratings !
      Quantize Zn on ca. L bits; replace Zn by Xn = 2L ZN with



 Issue: small increments due to “forgetting” coefficient w (e.g. w =
  0.9) are set to 0




 Solution: use random rounding; replace previous equation by


      where RANDROUND(2.7) = 2 with proba 0.3 and 3 with proba 0.7
        E(RANDROUND(x)) = x

                                                                        39
                  Transition Matrix K
 The transition matrix KN is straightforward but tedious to describe.

 Unlike in the TCP/ECN example, it does depend on N

 It contains terms such as : the proba that an indirect observation
  with a honest peer is with someone who has rating equal to k. This
  proba is equal to




  It depends on N, but for large N it converges uniformly to MNk(t),
  with no term in N

 The limiting matrix K is polynomial in MNk(t), thus continuous, thus
  assumption 2 is satisfied

 Assumption 1 is trivially satisfied, by inspection
                                                                         40
Therefore we can apply the theorem and
derive the fluid approximation and the
mean field approximation




Both are true in the limit N = 1




                                         41
Discrete event simulation, N = 100            Fluid Approximation




                                     Limiting reputation ratings: 0.9 and 0.1

                                               Fast Simulation based on
                                               Mean Field Approximation




                                                                           42
 Fluid approximation
     Can be written using Theorem 4.1
     Is a deterministic recurrence with state vector the memory
     number of dimensions is 2 L+1, where L = number of quantization bits
     for reputation values (e.g. L=8)



 Mean Field Approximation = Fast Simulation
     Simulation of one Markov chain on state space with 2 L states, with time
     varying transition probability




                                                                            43
    Different
  Parameters
   (few liars)



Few liars
Final ratings converge to true value

Phase transition




                                       44
Different Initial Conditions




                               45
Liar Strategy 2
    (infer)




                     Peers starting after 512 time units

  Liar Strategy 3
(side information)




                                                     46
 Modelling Locality with Multiclass Model
 We can model spatial aspects
     Object = honest peer ; state = (c, x) with
        C = location (in a discrete set of locations)
        X = rating (same as before)
     This allows to account for locality of interaction




                                                          47
                           Contents

         1. Motivation

2. A Generic Model for a System
      of Interacting Objects

 3. Convergence to the Mean
            Field

      4. Fast Simulation

   5. Full Scale Example: A
       Reputation System

          6. Outlook
                                      E.L.

                                        48
                                   Outlook
 I have shown how a mean field convergence result can be used to write
  and validate
        the fluid approximation = macroscopic description
        the mean field approximation = fast simulation (or analysis)

 Applies to cases where objects interact such that
      Transition depends on state of this object + current and past distribution of
      states of all other objects
      Number of objects is large compared to number of states of one object

 Extensions
      birth and death of objects
      transitions that affect several objects simultaneously
      Continuous time limits
      Deterministic Approximations of Stochastic Evolution in Games, M. Benaïm and
      J.W. Weibull, Econometrica. (2003), 71, 3 873-903
      Quasi-stationary approximations
      [Bordenave, McDonald and Proutière, arXiv 2007]
      Gaussian approximations (central limit theorems)




                                                                                      49
… thank you for your attention




                                 E. L.




                                         50

								
To top