Docstoc

Advanced Queueing Theory

Document Sample
Advanced Queueing Theory Powered By Docstoc
					                                                                 1
Networks of queues




• Networks of queues
  reversibility, output theorem, tandem networks, partial
  balance, product-form distribution, blocking, insensitivity,
  BCMP networks, mean-value analysis, Norton's theorem



                  Richard J. Boucherie

            Stochastic Operations Research
          department of Applied Mathematics
                 University of Twente
                                                  2

Networks of Queues
Nelson, sec 10.1
• Continuous time Markov chain
• Birth-death process
• Example: M/M/1 queue
• Birth-death process: equilibrium distribution
• Reversibility, stationarity
• Time reversed process
Continuous time Markov chain
                                                                                  3



• stochastic process X=(X(t), t≥0) evolution random variable

• countable or finite state space S={0,1,2,…}

• Markov property    P( X (t  s)  j | X (t )  i, X (tn )  jn ..., X (t1 )  j1 )
                       P( X (t  s)  j | X (t )  i)

• Markov chain: Stochastic process satisfying Markov property

• Transition probability         Pt (i, j )  P( X (t )  j | X (0)  i)
  time homogeneous

• Chapman-Kolmogorov equations
                                     Pt  s (i, j )   Pt (i, k ) Ps (k , j )
                                                       k
    Continuous time Markov chain                                                                   4



• Chapman-Kolmogorov equations Pt  s (i, j )   Pt (i, k ) Ps (k , j )
                                                                    k

• transition rates or jump rates q(i, j )  lim Ph (i, j )                        i j
                                                            h0         h
• Kolmogorov forward equations: (REGULAR)
                           Pt  h (i, j )   Pt (i, k ) Ph (k , j )
                                            k


               Pt  h (i, j )  Pt (i, j )   Pt (i, k ) Ph (k , j )  Pt (i, j )[Ph ( j , j )  1]
                                           k j


                                          [ Pt (i, k ) Ph (k , j )  Pt (i, j ) Ph ( j , k )]
                                           k j


                             Pt ' (i, j )   [ Pt (i, k )q(k , j )  Pt (i, j )q( j , k )]
                                           k j
                Continuous time Markov chain                         5


• Assume ergodic and regular

• global balance equations (equilibrium equations)

                    0   [ (k )q(k , j )   ( j )q( j, k )]
                        k j

• π is invariant measure (stationary measure)

• solution that can be normalised is stationary distribution

• if stationary distribution exists, then it is unique and is
  limiting distribution
                          lim P( X (t )  k | X (0)  j )   (k )
                          t 
    Markov jump chain                                                 6



• For Markov chain: holding time in state i is exponentially
  distributed; it is minimum of exponential times for each
  transition out of state i, say to state j with rate q(i,j)

• Minimum has exponential distribution with rate q(i) =-q(i,i)

• Probability that transition to state j occurs upon departure from
  state i is p(i,j)=q(i,j) / q(i)

• Markov jump chain : Markov chain with transition rates
  specified by q(i), and p(i,j) and is equivalent to original chain

• For Markov jump chain, we may generalise the holding time in
  the states to non-exponential times: this does not affect the
  equilibrium probability for being in state i
                                                    7

Networks of queues
Today (lecture 1):
•   Continuous time Markov chain
•   Birth-death process
•   Example: M/M/1 queue
•   Birth-death process: equilibrium distribution
•   Reversibility, stationarity
•   Time reversed process
                                                                                                           8
Birth-death process
 • State space S  Z
 • Markov chain, transition rates
                                ( j)          k  j  1 birth rate
                                ( j)          k  j  1 death rate
                          
              q( j, k )  
                            ( j )   ( j )    k j
                          
                                   0          otherwise
 • Kolmogorov forward equations
        dPt (i, j )
                     Pt (i, j  1) ( j  1)  Pt (i, j )[ ( j )   ( j )]  Pt (i, j  1)  ( j  1)
           dt
 • Global balance equations

                 0   ( j  1) ( j  1)   ( j )[ ( j )   ( j )]   ( j  1)  ( j  1)
                                                    9

Networks of queues
Today (lecture 1):
•   Continuous time Markov chain
•   Birth-death process
•   Example: M/M/1 queue
•   Birth-death process: equilibrium distribution
•   Reversibility, stationarity
•   Time reversed process
        M/M/1 queue
                                                                                               10




   • Poisson arrival process rate λ,
     single server, exponential service times, mean 1/μ
   • State space S={0,1,2,…}
   • Markov chain?
   • Assume initially empty: P(X(0)=0)=1,

   • Transition rates :

                                                                                   k  j 1
   P( X (t  h)  j  1 | X (t )  j )  h  o(h)                    
                                                                                 k  j  1, j  0
   P( X (t  h)  j  1 | X (t )  j )  h  o(h)       q( j, k )  
P( X (t  h)  j | X (t )  j )  1  [h  h]  o(h)                [    ] k  j , j  0
                                                                      
                                                                                  k  j, j  0
M/M/1 queue
                                                                                              11




                          k  j 1
                       k  j  1, j  0
            
q( j, k )  
             [    ] k  j , j  0
             
                         k  j, j  0

• Kolmogorov forward equations, j>0

                        dPt (i, j )
                                     Pt (i, j  1)  Pt (i, j )[   ]  Pt (i, j  1) 
                           dt
                        dPt (i,0)
                                      Pt (i,0)  Pt (i,1) 
                            dt
• Global balance equations, j>0
                       0   ( j  1)   ( j )[   ]   ( j  1) 
                       0   (0)   (1) 
                                                               12


M/M/1 queue

   λ       λ       λ
               j       j+1
  μ       μ
                                 ( j )   (0)( /  )   j

Equilibrium distribution: λ<μ
                                 (1   /  )( /  ) j
Stationary measure; summable  equilibrium distribution
Proof: Insert into global balance
Detailed balance!

         ( j )q( j, j  1)   ( j  1)q( j  1, j )
Theorem: A distribution that satisfies detailed balance is a
  stationary distribution
                                                    13

Networks of queues
Today (lecture 1):
•   Continuous time Markov chain
•   Birth-death process
•   Example: M/M/1 queue
•   Birth-death process: equilibrium distribution
•   Reversibility, stationarity
•   Time reversed process
Birth-death process
                                                                                      14


 • State space S  N  {0,1,2,...}
 • Markov chain, transition rates
                              ( j)                k  j 1          birthrate
                              ( j)             k  j  1, j  0
                                                                    deathrate
            q( j, k )  
                          ( j )   ( j )      k  j, j  0
                        
                              (0)              k  j, j  0

 • Definition: Detailed balance equations
      ( j )q( j, j  1)   ( j  1)q( j  1, j )
                                                                                 1
                                                       j
                                                              q ( r  1, r ) 
 • Theorem: Assume that              (0)                                        
                                             jS      r 1   q ( r , r  1) 
                         j
                               q (r  1, r )
   then    ( j )   (0)                   ,   jS
                        r 1   q (r , r  1)
   is the equilibrium distrubution of the birth-death process X.
                                                    15

Networks of queues
Today (lecture 1):
•   Continuous time Markov chain
•   Birth-death process
•   Example: M/M/1 queue
•   Birth-death process: equilibrium distribution
•   Reversibility, stationarity
•   Time reversed process
                Reversibility; stationarity                                               16


• Stationary process: A stochastic process is stationary if
  for all t1,…,tn,τ
   ( X (t1 ), X (t 2 ),..., X (t n )) ~ ( X (t1   ), X (t 2   ),..., X (t n   ))

• Theorem: If the initial distribution of a Markov chain is a
  stationary distribution, then a Markov chain is stationary

• Reversible process: A stochastic process is reversible if for
  all t1,…,tn,τ
    ( X (t1 ), X (t 2 ),..., X (t n )) ~ ( X (  t1 ), X (  t 2 ),..., X (  t n ))
            Reversibility; stationarity                         17



• Lemma: A reversible process is stationary.


• Theorem: A stationary Markov chain is reversible if and
  only if there exists a collection of positive numbers π(j),
  jεS, summing to unity that satisfy the detailed balance
  equations
           ( j )q( j, k )   (k )q(k , j ), j, k  S
  When there exists such a collection π(j), jεS, it is the
  equilibrium distribution.

• Proof
                                                                                         18
Proof
suppose process reversible
Let      ( j )  P( X (t )  j )
rev.:   P( X (t )  j , X (t  h)  k )  P( X (t )  k , X (t  h)  j )
then              P( X (t  h)  k | X (t )  j )          P( X (t  h)  j | X (t )  k )
            ( j)                                   (k )
                               h                                        h

Conversely: suppose there exists π satisfying detailed balance,
summing gives global balance.
                                                                                                                               19
  Proof (ctd.)
  Consider behaviour of X(t) for -T≤t ≤T
  starts at –T in j1, stays random time h1 before jumping to j2, stays
  random time h2, before jumping to j3, and so on, until it arrives at
  jm, where it stays until T
  Probability density for this path is
                     q ( j1 ) h1 q ( j1 , j2 )            q ( j2 ) h2 q ( j2 , j3 )     q( jm 1 , jm )  q ( jm ) hm
    ( j1 )q( j1 )e                             q ( j2 )e                                             e
                                    q( j1 )                               q ( j2 )         q( jm 1 )
                                                                                                                         (*)
      ( j1 )e    q ( j1 ) h1
                                  q( j1 , j2 )e    q ( j2 ) h2
                                                                  q ( j2 , j3 )    q( jm 1 , jm )e    q ( jm ) hm


  density with respect to h1,…hm. Integrate over h1,…hm such that
  h1+…+hm=2T.
   Detailed balance implies
 ( jm )q( jm , jm1 )q( jm1 , jm2 )    q( j2 , j1 )   ( j1 )q( j1 , j2 )q( j2 , j3 )    q( jm1 , jm )
  so that (*) is prob density for path starting –T in jm, stays random time
  hm before jumping to jm-1, and so on, until it arrives at j1, stays until T.
  Thus, also invoking stationarity,
        ( X (t1 ), X (t 2 ),..., X (t n )) ~ ( X (  t1 ), X (  t 2 ),..., X (  t n ))
                                                                                             20

  Kolmogorov’s criteria
• Theorem: A stationary Markov chain is reversible iff

  q( j1 , j2 )q( j2 , j3 )...q( jn1 , jn )q( jn , j1 )
                                                                      j1 , j2 ,..., jn  S
   q( j1 , jn )q( jn , jn1 )...q( j3 , j2 )q( j2 , j1 )
  for each finite sequence of states.

  Furthermore,
                     q (0, j1 )q ( j1 , j2 )q ( j2 , j3 )... q ( jn 1 , jn )q ( jn , j )
      ( j )   (0)
                     q ( j , jn )q ( jn , jn 1 )... q ( j3 , j2 )q ( j2 , j1 )q ( j1 ,0)


  for each sequence for which the denominator is positive.
                                                    21

Networks of queues
Today (lecture 1):
•   Continuous time Markov chain
•   Birth-death process
•   Example: M/M/1 queue
•   Birth-death process: equilibrium distribution
•   Reversibility, stationarity
•   Time reversed process
    Time reversed process
                                                                                                   22


• X(t) reversible Markov process  X(-t) also, but time
  homogeneity not inherited for non-stationary process

• Lemma: If X(t) is a time-homogeneous Markov process which is
  non stationary, then the reversed process X(τ-t) is a Markov
  process which is not even time-homogeneous.

• Proof. X(t) is a Markov process X(τ-t) easy
  non-time-homogeneous: observe
 P( X (t )  j ) P( X (t  h)  k | X (t )  j )  P( X (t  h)  k ) P( X (t )  j | X (t  h)  k )
 P( X (t  h)  k | X (t )  j )
                                            does not depend on t (time-hom)
 P( X (t  h)  k ), P( X (t )  j )        do depend on t at least for some t,j,k
  and so also the transition probabilities of the reversed process
                                      P( X (t )  j )
 P( X (t )  j | X (t  h)  k )                       P( X (t  h)  k | X (t )  j )
                                     P( X (t  h)  k )
   Time reversed process                                             23



• Theorem: If X(t) is a stationary Markov process with transition
  rates q(j,k), and equilibrium distribution π(j), jεS, then the
  reversed process X(τ-t) is a stationary Markov process with
  transition rates               (k )q(k , j )
                     q' ( j, k )                     j, k  S
                                       ( j)
  and the same equilibrium distribution.

• Theorem: Kelly’s lemma (Proposition 10.2)
  Let X(t) be a stationary Markov process with transition rates
  q(j,k). If we can find a collection of numbers q’(j,k) such that
  q’(j)=q(j), jεS, and a collection of positive numbers π(j), jεS,
  summing to unity, such that
          ( j ) q( j , k )   ( k ) q ' ( k , j )     j, k  S
  then q’(j,k) are the transition rates of the time-reversed process,
  and π(j), jεS, is the equilibrium distribution of both processes.
                                                    24

Networks of queues
Today (lecture 1):
•   Continuous time Markov chain
•   Birth-death process
•   Example: M/M/1 queue
•   Birth-death process: equilibrium distribution
•   Reversibility, stationarity
•   Time reversed process
                                                             25
                    Summary / next:
• Detailed balance or reversibility and their consequences
  Birth-death process
  M/M/1 queue
  Kolmogorov’s criteria

• Next on AQT…..
  Section 10.2 – 10.3.5




                      J
    ( j1,..., j J )   (1  i )  i ,  i   / i  1
                                        j

                      i1

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:10
posted:11/1/2011
language:English
pages:25