# EPFL cole Polytechnique Fdrale de Lausanne by sanmelody

VIEWS: 14 PAGES: 50

• pg 1
```									A Generic Mean Field Convergence Result
for Systems of Interacting Objects

From Micro to Macro

Jean-Yves Le Boudec, EPFL

Joint work with David McDonald, U. of Ottawa
and Jochen Mundinger, EPFL

1
 The full text of my talk and this slide show are available from my
web page

http://people.epfl.ch/jean-yves.leboudec

Direct access:

Full text:
http://infoscience.epfl.ch/getfile.py?recid=108827&mode=best

Slide show:
http://icawww1.epfl.ch/PS_files/mean-field-leb-vt07.ppt

2
Contents

1. Motivation

2. A Generic Model for a System
of Interacting Objects

3. Convergence to the Mean
Field

4. Fast Simulation

5. Full Scale Example: A
Reputation System

6. Outlook
E.L.

3
Motivation
 Find re-usable approximations of large scale systems

Examples from my field
Performance of UWB impulse radio : many sensors, each has a MAC
layer state
Reputation Systems

 From microscopic description to macroscopic equations

 Understand fluid approximation and mean field approximation

4
Example 1 : TCP/ECN
ECN Feedback q(R(t))

1

n              ECN router
queue length R(t)
N
N connections

 TCP connection n transmits at a         When connection n does not
rate 2 {s0, …, si, …, sI}                receive an ECN, it increases its
 Queue length at router is R(t)           rate:
 With probability q(R(t)) connection          If rate == si, new rate := si+1 (i<I)
i receives an Explicit Congestion       Else it decreases its rate:
Notification (ECN) in next time slot         If rate == si, new rate := sd(i)

The question is the behaviour when N is large

5
Microscopic Description
 Time is discrete
 Connection n runs one Markov chain XNn(t);

 The transition probabilities of the Markov chain XNn(t) depend on
global state R(t) (queue size)

 Global state R(t) depends on states of all connections
let MNi(t) = nb of connections in state i at time t ,   C = service rate of router

6
Macroscopic Description
 The fluid approximation is often given as a simplification of the
previous model

 Combined with

we have a macroscopic description of the system

 In [17], Tinna. and Makowski show that it holds as large N
asymptotics
7
The Mean Field Approximation
 Assume we want to analyze one TCP connection in detail
 We can keep the microscopic description for this TCP connection,
and use the fluid approximation for the others:
We can call it fast simulation.

 i.e. pretend XN1(t) (one connection) and R(t) (global resource) are
independent. This is similar to what is called the mean field
approximation in physics

8
Another Example:
Robot Swarm

 N robots
 Robot has S = 2 possible states
 Transition for one robot depends
on this robot’s state + how many
other robots are in search state
 [11] uses the fluid approximation :

9
A few other Examples …

M.-D. Bordenave, D. McDonald and A. Proutière, A
particle system in interaction with a rapidly varying
environment: Mean field limits and applications
arXiv:math/0701363v2

10
In these and other examples, some authors
assume the validity of the fluid / mean field
approximation and use the approximation to
do performance evaluation, parameter
identification, control…                        Never
again !
… while, in contrast, others
spend most of the paper proving the
derivation and validity of the
approximations in their specific
setting

papers in this latter class are intimidating
cost of proof of one approximation result
¼ 1 PhD
and not re-usable

11
Can we have answers of general applicability to:

When are the fluid approximation and the mean field
approximation valid ?

Can we write them in a sound ( = mechanical) way ?

12
Contents

1. Motivation

2. A Generic Model for a System
of Interacting Objects

3. Convergence to the Mean
Field

4. Fast Simulation

5. Full Scale Example: A
Reputation System

6. Outlook
E.L.

13
Mean Field Interaction Model
 A Generic Model, with generic results
 Does not cover all useful cases, but is a useful first step

 Time is discrete
 N objects
 Every object has a state in               .

 Informally: object n evolves depending only on
Its own state
A global resource whose evolution depends only on how many other
objects are in each state

14
Model Assumptions
XNn(t) : state of object n at time t

MNi(t)   = proportion of objects that are in state i
MN is the “occupancy measure” ¼ the “mean field”

RN(t) = global resource =“history” of occupancy measure

Conditional to history up to time t, objects draws next state
independent of each other according to

15
Two Mild Assumptions
1. Continuity of the integration function g()

2. For large N, the transition matrix K becomes independent of N and
is continuous

16
TCP/ECN Example fits in this Framework
ECN Feedback q(R(t))        Function g() :
1

n                 ECN router
queue length R(t)
N
N connections                      thus

 Intuitively satisfies the conditions
State of one connection depends
only on buffer content
Buffer contents depends only on
how many connections are in each
state
g() is continuous
 Formally:
Assumption 1 is satisfied
One object = one TCP connection
State of one object = index i of
sending rate
RN(t) = total buffer occupancy / N

17
TCP/ECN Example fits in this Framework
ECN Feedback q(R(t))      Transition matrix K
1

n                 ECN router              Let q(r) = proba of negative
queue length R(t)          feedback when R==r
N
N connections

K is independent of N thus
Assumption 2 is is satisfied if q() is

18
A Multiclass Variant
 Take same as previous TCP/ECN      Also fits in our framework
model but introduce multiclass

 Aggressive connections, normal
connection                         Mean Field does not mean all
objects are exchangeable !
 State of an object = (c, i)
c : class
i : sending rate

 Objects may change class or not

19
Contents

1. Motivation

2. A Generic Model for a System
of Interacting Objects

3. Convergence to the Mean
Field

4. Fast Simulation

5. Full Scale Example: A
Reputation System

6. Outlook
E.L.

20
21
Practical Application : Derivation of the Fluid
Approximation
 The theorem replaces the stochastic system by a deterministic,
dynamical system

 This gives a method to write and justify the fluid approximation in
the large N regime
Equation for the limiting occupancy measure  can be rewritten as

where Ni(t) = N MNi(t) = number of objects in state i at time t

22
Proof of Theorem
 Based on
The next theorem (fast simulation)
A coupling argument
An ad-hoc version of the strong law of large numbers
The Glivenko Cantelli lemma

23
Contents

1. Motivation

2. A Generic Model for a System
of Interacting Objects

3. Convergence to the Mean
Field

4. Fast Simulation

5. Full Scale Example: A
Reputation System

6. Outlook
E.L.

24
Fast Simulation / Analysis of One Object
 Assume we are interested in one object in particular
E.g. distribution of time until a TCP connection reaches maximum rate
 For large N, since mean field convergence holds, one may do the
mean field approximation and replace the set of other objects by
the deterministic dynamical system

 The next theorem says that, essentially, this is valid

25
Fast Simulation Algorithm

State of one specific object

Returns next state for one object
When transition matrix is K

Replace true value by deterministic
limit

This is the mean field independence
approximation

26
Fast Simulation Result

27
Practical Application
 This justifies the mean field approximation for the stochastic
evolution of one object in the large N regime

 Gives a method for fast simulation or analysis
The state space for Y1 has S states, instead of SN

28
Contents

1. Motivation

2. A Generic Model for a System
of Interacting Objects

3. Convergence to the Mean
Field

4. Fast Simulation

5. Full Scale Example: A
Reputation System

6. Outlook
E.L.

29
A Reputation System
 My original motivation for this work
 Illustrates the complete set of steps, including a few modelling
tricks
 System
N objects = N peers
Peers observe one subject and rate it
Rating is a number in (0,1)
Direct observations and spreading of reputation
Confirmation bias + forgetting

30
Operation of Reputation System: Forgetting
 Zn(t) = reputation rating held by peer n

 During a direct observation, subject is perceived as positive (with
proba ) or negative (with proba 1-)

 In case of direct positive observation

 In case of direct negative observation

 w is the forgetting factor, close to 1 (0.9 in next slides)
31
Confirmation Bias
 Peer also read other peer ratings
 If overheard rating is z:

  is the threshold of the confirmation bias

32
Liars and Honest Peers
 Honest peer does as just explained
 Liar tries to bring the reputation down
Uses different strategies, see later

33
Example of exact simulation: N=100 peers
with maximal liars (always say Z=0)
proportion of peers

Initially: peers have Z=0, 0.5 or 1

 = 0.9

rating

Every time step: direct obs p=0.01, meet liar proba 0.30, meet honest proba 0.69
34
rating   3 particular peers, one of each type

 = 0.9

time

35
Can we study the system with 106

36
The problem fits in our framework…
 Assume discrete time
 At every time step a peer
Makes a direct observation
Or overhears a liar
Or overhears some honest peer
Or does nothing

 Object = honest peer

 Assume first that liars use strategy 1: maximal lying (always say
Z=0)

 Transition of one honest peer depends on
Own state
Distribution of states of all other peers

=> Fits in our framework with memory R = occupancy measure M

37
Different Liar Strategies
 Strategy 1 (maximal lying): liars always say Z= 0
 Strategy 2 (infer): liar guesses your rating based on past
experience
Transition of one honest peer depends on
Own state
Distribution of states of all other peers
What liars remember seeing in the past

=> Fits in our framework with memory R = occupancy measure of ratings
at steps t and t-1

 Strategy 3 (side information): liars know your rating and is as
negative as you accept
not realistic but serves as benchmark (worst case)
Similar to strategy 1, memory = occupancy measure M

38
We would like to apply the mean field
convergence result to analyze very large N
 But model has continuous state space

 Discretize reputation ratings !
Quantize Zn on ca. L bits; replace Zn by Xn = 2L ZN with

 Issue: small increments due to “forgetting” coefficient w (e.g. w =
0.9) are set to 0

 Solution: use random rounding; replace previous equation by

where RANDROUND(2.7) = 2 with proba 0.3 and 3 with proba 0.7
E(RANDROUND(x)) = x

39
Transition Matrix K
 The transition matrix KN is straightforward but tedious to describe.

 Unlike in the TCP/ECN example, it does depend on N

 It contains terms such as : the proba that an indirect observation
with a honest peer is with someone who has rating equal to k. This
proba is equal to

It depends on N, but for large N it converges uniformly to MNk(t),
with no term in N

 The limiting matrix K is polynomial in MNk(t), thus continuous, thus
assumption 2 is satisfied

 Assumption 1 is trivially satisfied, by inspection
40
Therefore we can apply the theorem and
derive the fluid approximation and the
mean field approximation

Both are true in the limit N = 1

41
Discrete event simulation, N = 100            Fluid Approximation

Limiting reputation ratings: 0.9 and 0.1

Fast Simulation based on
Mean Field Approximation

42
 Fluid approximation
Can be written using Theorem 4.1
Is a deterministic recurrence with state vector the memory
number of dimensions is 2 L+1, where L = number of quantization bits
for reputation values (e.g. L=8)

 Mean Field Approximation = Fast Simulation
Simulation of one Markov chain on state space with 2 L states, with time
varying transition probability

43
Different
Parameters
(few liars)

Few liars
Final ratings converge to true value

Phase transition

44
Different Initial Conditions

45
Liar Strategy 2
(infer)

Peers starting after 512 time units

Liar Strategy 3
(side information)

46
Modelling Locality with Multiclass Model
 We can model spatial aspects
Object = honest peer ; state = (c, x) with
C = location (in a discrete set of locations)
X = rating (same as before)
This allows to account for locality of interaction

47
Contents

1. Motivation

2. A Generic Model for a System
of Interacting Objects

3. Convergence to the Mean
Field

4. Fast Simulation

5. Full Scale Example: A
Reputation System

6. Outlook
E.L.

48
Outlook
 I have shown how a mean field convergence result can be used to write
and validate
the fluid approximation = macroscopic description
the mean field approximation = fast simulation (or analysis)

 Applies to cases where objects interact such that
Transition depends on state of this object + current and past distribution of
states of all other objects
Number of objects is large compared to number of states of one object

 Extensions
birth and death of objects
transitions that affect several objects simultaneously
Continuous time limits
Deterministic Approximations of Stochastic Evolution in Games, M. Benaïm and
J.W. Weibull, Econometrica. (2003), 71, 3 873-903
Quasi-stationary approximations
[Bordenave, McDonald and Proutière, arXiv 2007]
Gaussian approximations (central limit theorems)

49
… thank you for your attention

E. L.

50

```
To top