; Lecture8
Learning Center
Plans & pricing Sign in
Sign Out



  • pg 1
									Lecture 8
Basic probability laws of discrete random variables

Plan of the lecture:
   1. Bernoulli trial. The binomial law.
       1.1 Bernoulli trial
       1.2 The Bernoulli Random Variable
       1.3 The Binomial Random Variable
       1.4 The multinomial distribution
   2. The Poisson Random Variable
   3. Simplest flow of events (simple arrival)
       3.1 Flow of events
       3.2 Arrival process
       3.3 The Poisson process

Lecturer: A.S. Eremenko

       1 Bernoulli trial. The binomial law.

       1.1 Bernoulli trial

       In the theory of probability and statistics, a Bernoulli trial is an experiment whose
outcome is random and can be either of two possible outcomes, "success" and "failure".
       In practice it refers to a single experiment which can have one of two possible outcomes.
These events can be phrased into "yes or no" questions:
      Did the coin land heads?
      Was the newborn child a girl?
      Were a person's eyes green?
      Did a mosquito die after the area was sprayed with insecticide?
      Did a potential customer decide to buy a product?
      Did a citizen vote for a specific candidate?
      Did an employee vote pro-union?
       Therefore success and failure are labels for outcomes, and should not be construed
literally. Examples of Bernoulli trials include:
   1. Flipping a coin. In this context, obverse ("heads") conventionally denotes success and
       reverse ("tails") denotes failure. A fair coin has the probability of success         by
   2. Rolling a die, where a six is "success" and everything else a "failure".
   3. In conducting a political opinion poll, choosing a voter at random to ascertain whether
       that voter will vote "yes" in an upcoming referendum.

       1.2 The Bernoulli Random Variable

       Consider the toss of a biased coin, which comes up a head with probability , and a tail
with probability       . The Bernoulli random variable takes the two values      and , depending
on whether the outcome is a head or a tail:

       Its PMF is

       For all its simplicity, the Bernoulli random variable is very important. In practice, it is
used to model generic probabilistic situations with just two outcomes. Furthermore, by
combining multiple Bernoulli random variables, one can construct more complicated random
       Its mean, second moment, and variance are given by the following calculations:


       1.3 The Binomial Random Variable

       A biased coin is tossed    times. At each toss, the coin comes up a head with probability
 , and a tail with probability       , independently of prior tosses. Let    be the number of heads
in the -toss sequence. We refer to     as a binomial random variable with parameters        and .
The PMF of     consists of the binomial probabilities:

                                                                ,              .

       (Note that here and elsewhere, we simplify notation and use , instead of , to denote the
experimental values of integer-valued random variables.) The normalization property
              , specialized to the binomial random variable, is written as


       Some special cases of the binomial PMF are sketched in Fig. 1.

 Figure 1: The PMF of a binomial random variable. If              , the PMF is symmetric around
          . Otherwise, the PMF is skewed towards       if        , and towards   if       .

       Its mean, and variance are

       Mode:                or                  .
       Median:         or        .

       Skewness:            .

       Kurtosis:            .

                                     Figure 2: Binomial PMF and CDF

       The binomial distribution is of repeated use each time a trial involves superposing
independent elementary trials. In the field of traffic, the Engset problem introduces it as a
limiting distribution of the number of busy lines in subscriber concentrators.

       The binomial distribution enjoys the following property, concerning the sum of several
       THEOREM. If two discrete random variables              and       have binomial distributions with
parameters respectively         and          , the variable             has a binomial distribution with
parameters             .

       1.4 The multinomial distribution

       This is the generalization of the binomial law. Assume that            types can be distinguished
in the population, so that the population has a proportion      of elements of type      (with naturally
          ). The question is: what is the probability of observing, when extracting         items,    of
the type , etc.,    of type   , with                          . The result is:


       This result has numerous applications. Imagine for instance observing a network element
(e.g. a concentrator) to which various sources are connected. What is the distribution of the
traffic the sources generate? Individual traffic intensity, expressed in erlangs, is distributed
between      and    . One usually defines categories, according to the type of subscriber
(professional, residential, in urban area, etc.), corresponding to traffic intensities. With two
categories, the binomial distribution gives the answer. For several categories (typically: less than
    , between        and      , ..., higher than       ), the distribution of the number of customers
among the categories is given by the multinomial law. Finally, knowing the distribution of the
different customers among the categories allows dimensioning the subscriber concentrator, using
the multinomial distribution. More generally, this result holds whenever a population, composed
with different sub-populations, is observed.

       2 The Poisson Random Variable

       A Poisson random variable takes nonnegative integer values. Its PMF is given by

                                                   ,                ,

where       is a positive parameter characterizing the PMF, see Fig. 3. It is a legitimate PMF


         To get a feel for the Poisson random variable, think of a binomial random variable with
very small     and very large . For example, consider the number of typos in a book with a total
of      words, when the probability     that any one word is misspelled is very small (associate a
word with a coin toss which comes a head when the word is misspelled), or the number of cars
involved in accidents in a city on a given day (associate a car with a coin toss which comes a
head when the car has an accident). Such a random variable can be well-modeled as a Poisson
random variable.
         More precisely, the Poisson PMF with parameter             is a good approximation for a
binomial PMF with parameters          and , provided           ,   is very large, and    is very small,

                                                           ,                 .

         In this case, using the Poisson PMF may result in simpler models and calculations. For
example, let           and             . Then the probability of          successes in         trials is
calculated using the binomial PMF as


         Using the Poisson PMF with                                , this probability is approximated

Figure 3: The PMF             of the Poisson random variable for different values of . Note that if

        , then the PMF is monotonically decreasing, while if          , the PMF first increases and
                              then decreases as the value of   increases.

        The mean of the Poisson PMF can be calculated is follows:


        The last equality is obtained by noting that                                          is the

normalization property for the Poisson PMF.
        A similar calculation shows that the variance of a Poisson random variable is also .
        Mode:       and        if   is an integer.

        Median:                        .

        Skewness:         .
        Kurtosis:     .

                                    Figure 4: Poisson PMF and CDF

        THEOREM. Let          and     be two Poisson variables with parameters respectively   and .
Then,        obeys a Poisson distribution with parameter          .

       3 Simplest flow of events (simple arrival)

       Stemming from probability theory its application to the field of telecommunications for
the solving of traffic problems has given rise to a well-known discipline: teletraffic.
       A stochastic process is a mathematical model of a probabilistic experiment that evolves
in time and generates a sequence of numerical values. For example, a stochastic process can be
used to model:
       (a) the sequence of daily prices of a stock;
       (b) the sequence of scores in a football game;
       (c) the sequence of failure times of a machine;
       (d) the sequence of hourly traffic loads at a node of a communication network;
       (e) the sequence of radar measurements of the position of an airplane.
       Each numerical value in the sequence is modeled by a random variable, so a stochastic
process is simply a (finite or infinite) sequence of random variables and does not represent a
major conceptual departure from our basic framework. We are still dealing with a single basic
experiment that involves outcomes governed by a probability law, and random variables that
inherit their probabilistic properties from that law.

       3.1 Flow of events

       Suppose that in a time line randomly arise points – moments of appearance of some
homogeneous events (for example, calls on the telephone station etc). Sequence of those
moments is so called flow of events.

                                              Figure 5

                  Figure 6: Examples of flows of events in telecommunications

                                  Figure 7: The basic service station

       A lot of arrival processes can be described as so called Bernoulli and Poisson processes.

       3.2 Arrival process

       To advance in the study of traffic properties, it is necessary to look at the two components
on which it depends, i.e. client arrivals and their service.
       Arrivals of clients at the system input are observed. To describe the phenomenon, the
arrival law, the first idea is to use the time interval between successive arrivals (inter-arrival
time), or the number of arrivals in a given time interval.
       During the time ,            arrivals occur. The flow intensity is then expressed as a number,
the arrival rate, whose intuitive definition is:


       It is then also possible to estimate the average interval between arrivals, this being the
inverse of the previous quantity.
       Figure 7 illustrates random arrivals. Consider random arrivals in the time interval from
to   . The interval length is               . The arrival rate is a long term average of the number
of arrivals per unit time. Its mathematical symbol is     and its unit is        , and is given by the
following equation, where       is the number of arrivals in the interval of length :

                                               Figure 7

   Figure 8: Illustration of the terminology applied for a traffic process. Notice the difference
between time intervals and instants of time. We use the terms arrival and call synonymously. The
 inter-arrival time, respectively the inter-departure time, are the time intervals between arrivals,
                                       respectively departures.

       Flow of events is called the simplest flow if it has the properties of Poisson arrival.

       3.3 The Poisson process

       The Poisson process can be viewed as a continuous-time analog of the Bernoulli process
and applies to situations where there is no natural way of dividing time into discrete periods.
       We consider an arrival process that evolves in continuous time, in the sense that any real
number is a possible arrival time. We define


and assume that this probability is the same for all intervals of the same length . We also
introduce a positive parameter     to be referred to as the arrival rate or intensity of the process.
       Definition of the Poisson Process
       An arrival process is called a Poisson process with rate              if it has the following
       (a) (Time-homogeneity.) The probability                    of   arrivals is the same for all
intervals of the same length .
       (b) (Independence.) The number of arrivals during a particular interval is independent of
the history of arrivals outside this interval.
       (c) (Small interval probabilities.) The probabilities            satisfy



       Here,        and           are functions of that satisfy

                                                    ,                    .

       The first property states that arrivals are “equally likely” at all times. The arrivals during
any time interval of length         are statistically the same, in the sense that they obey the same
probability law.
       To interpret the second property, consider a particular interval                  , of length     .
The unconditional probability of        arrivals during that interval is                . Suppose now that
we are given complete or partial information on the arrivals outside this interval. Property (b)
states that this information is irrelevant: the conditional probability of          arrivals during
remains equal to the unconditional probability                    .
       The third property is critical. The              and           terms are meant to be negligible in
comparison to      , when the interval length      is very small. They can be thought of as the
terms in a Taylor series expansion of               . Thus, for small , the probability of a single
arrival is roughly        , plus a negligible term. Similarly, for small , the probability of zero
arrivals is roughly           .
       Note that the probability of two or more arrivals is


and is negligible in comparison to             as gets smaller and smaller.

                      Figure 9: Bernoulli approximation of the Poisson process

       Let us now start with a fixed time interval of length           and partition it into    periods of
length , where       is a very small number; see Fig. 6. The probability of more than two arrivals

during any period can be neglected, because of property (c) and the preceding discussion.
Different periods are independent, by property (b). Furthermore, each period has one arrival with
probability approximately equal to         , or zero arrivals with probability approximately equal to
        . Therefore, the process being studied can be approximated by a Bernoulli process, with
the approximation becoming more and more accurate the smaller                          is chosen. Thus the
probability              of      arrivals in time   , is approximately the same as the (binomial)
probability of        successes in            independent Bernoulli trials with success probability
           at each trial. While keeping the length      of the interval fixed, we let the period length
decrease to zero. We then note that the number           of periods goes to infinity, while the product
   remains constant and equal to          . Under these circumstances, we can see that the binomial
PMF converges to a Poisson PMF with parameter                       . We are then led to the important
conclusion that

                                                          ,                .

        Note that a Taylor series expansion of           , yields


consistent with property (c).
        Using our earlier formulas for the mean and variance of the Poisson PMF, we obtain

                                                    ,                  ,

where          stands for the number of arrivals during a time interval of length . These formulas are
hardly surprising, since we are dealing with the limit of a binomial PMF with parameters
           ,          , mean          , and variance                               .
        Let us now derive the probability law for the time            of the first arrival, assuming that the
process starts at time zero. Note that we have                if and only if there are no arrivals during the
interval          . Therefore,

                                                                                       ,      .

       We then differentiate the CDF          of , and obtain the PDF formula

                                                     ,            ,

which shows that the time until the first arrival is exponentially distributed with parameter . We
summarize this discussion in the table that follows. See also Fig. 9.
       Random Variables Associated with the Poisson Process and their Properties
            The Poisson with parameter        . This is the number                of arrivals in a Poisson
             process with rate , over an interval of length . Its PMF, mean, and variance are

                                                          ,                ,

                                                ,                     .

            The exponential with parameter . This is the time            until the first arrival. Its PDF,
             mean, and variance are

                                         ,      ,             ,                .

    Figure 10: View of the Bernoulli process as the discrete-time version of the Poisson. We
   discretize time in small intervals   and associate each interval with a Bernoulli trial whose
           parameter is       . The table summarizes some of the basic correspondences.

To top