Docstoc

Chapter 2 Random Variables

Document Sample
Chapter 2 Random Variables Powered By Docstoc
					    Chapter 2. Random Variables

        2.1 Discrete Random Variables
        2.2 Continuous Random Variables
        2.3 The Expectation of a Random Variable
        2.4 The Variance of a Random Variable
        2.5 Jointly Distributed Random Variables
        2.6 Combinations and Functions of Random Variables




NIPRL
 2.1 Discrete Random Variable
 2.1.1 Definition of a Random Variable (1/2)

 • Random variable
    – A numerical value to each outcome of a particular
      experiment
                                                          S




                                                              R


              -3   -2    -1     0    1     2    3

NIPRL
 2.1.1 Definition of a Random Variable (2/2)


 • Example 1 : Machine Breakdowns
    – Sample space : S  {electrical , mechanical , misuse}
    – Each of these failures may be associated with a repair cost
    – State space : {50, 200,350}
    – Cost is a random variable : 50, 200, and 350




NIPRL
 2.1.2 Probability Mass Function (1/2)


 • Probability Mass Function (p.m.f.)
    – A set of probability value pi assigned to each of the values
      taken by the discrete random variable xi
    – 0  pi  1 and i pi  1
        – Probability : P( X  xi )  pi




NIPRL
 2.1.2 Probability Mass Function (1/2)


 • Example 1 : Machine Breakdowns
    – P (cost=50)=0.3, P (cost=200)=0.2,
      P (cost=350)=0.5
    – 0.3 + 0.2 + 0.5 =1                   xi     50    200   350

                                                  0.3   0.2   0.5
                                           pi
        f ( x)
                                 0.5


                 0.3
                        0.2




                 50     200       350   Cost($)
NIPRL
 2.1.3 Cumulative Distribution Function (1/2)


 • Cumulative Distribution Function
    – Function : F ( x)  P( X  x) F ( x)     P( X
                                               y: y  x
                                                           y)
    – Abbreviation : c.d.f



    F ( x)
        1.0


        0.5


        0.3


              0   50           200                  350    x($cost)
NIPRL
 2.1.3 Cumulative Distribution Function (2/2)


 • Example 1 : Machine Breakdowns
          x  50  F ( x)  P(cost  x)  0
        50  x  200  F ( x)  P(cost  x)  0.3
        200  x  350  F ( x)  P(cost  x)  0.3  0.2  0.5
        350  x    F ( x)  P(cost  x)  0.3  0.2  0.5  1.0




NIPRL
 2.2 Continuous Random Variables
  2.2.1 Example of Continuous Random Variables (1/1)

 • Example 14 : Metal Cylinder Production
    – Suppose that the random variable X is the diameter of a
      randomly chosen cylinder manufactured by the company.
      Since this random variable can take any value between
      49.5 and 50.5, it is a continuous random variable.




NIPRL
 2.2.2 Probability Density Function (1/4)


 • Probability Density Function (p.d.f.)
    – Probabilistic properties of a continuous random variable

                      f ( x)  0
                  statespace
                                f ( x)dx  1




NIPRL
 2.2.2 Probability Density Function (2/4)

 • Example 14
    – Suppose that the diameter of a metal cylinder has a p.d.f
           f ( x)  1.5  6( x  50.2)2 for 49.5  x  50.5
           f ( x)  0, elsewhere


                                       f ( x)




                                                49.5          50.5   x
NIPRL
 2.2.2 Probability Density Function (3/4)


 • This is a valid p.d.f.
            50.5
        
        49.5
                   (1.5  6( x  50.0) 2 )dx  [1.5 x  2( x  50.0)3 ]50.5
                                                                       49.5

                                              [1.5  50.5  2(50.5  50.0)3 ]
                                                 [1.5  49.5  2(49.5  50.0)3 ]
                                              75.5  74.5  1.0




NIPRL
 2.2.2 Probability Density Function (4/4)


 • The probability that a metal cylinder has a diameter between
   49.8 and 50.1 mm can be calculated to be
               50.1
            49.8
                      (1.5  6( x  50.0) 2 )dx  [1.5 x  2( x  50.0)3 ]50.1
                                                                          49.8

                                                  [1.5  50.1  2(50.1  50.0)3 ]
                                                     [1.5  49.8  2(49.8  50.0)3 ]
        f ( x)                                    75.148  74.716  0.432




                      49.5   49.8     50.1    50.5      x

NIPRL
 2.2.3 Cumulative Distribution Function (1/3)


 • Cumulative Distribution Function
                                   x
         F ( x)  P( X  x)          f ( y )dy
                                   

                   dF ( x )
         f ( x) 
                    dx

         P ( a  X  b)  P ( X  b)  P ( X  a )
                        F (b)  F (a)

         P ( a  X  b)  P ( a  X  b)

NIPRL
 2.2.2 Probability Density Function (2/3)


 • Example 14
                              x
        F ( x)  P( X  x)   (1.5  6( y  50.0) 2 )dy
                              49.5

                            [1.5 y  2( y  50.0)3 ]49.5
                                                     x


                            [1.5 x  2( x  50.0)3 ]  [1.5  49.5  2(49.5  50.0) 3 ]
                           1.5 x  2( x  50.0)3  74.5
        P(49.7  X  50.0)  F (50.0)  F (49.7)
                                      (1.5  50.0  2(50.0  50.0)3  74.5)
                                       (1.5  49.7  2(49.7  50.0)3  74.5)
                                      0.5  0.104  0.396

NIPRL
 2.2.2 Probability Density Function (3/3)




                                    P(49.7  X  50.0)  0.396
                    1



   P( X  50.0)  0.5
                           F ( x)




   P( X  49.7)  0.104


                          49.5      49.7       50.0              50.5   x



NIPRL
 2.3 The Expectation of a Random Variable
  2.3.1 Expectations of Discrete Random Variables (1/2)

 • Expectation of a discrete random variable with p.m.f
                     P( X  xi )  pi

                    E( X )       px
                                   i
                                          i    i




 • Expectation of a continuous random variable with p.d.f f(x)

                   E( X )     state space
                                              xf ( x)dx


 • The expected value of a random variable is also called the
   mean of the random variable


NIPRL
 2.3.1 Expectations of Discrete Random Variables (2/2)


 • Example 1 (discrete random variable)
    – The expected repair cost is
        E (cost)  ($50  0.3)  ($200  0.2)  ($350  0.5)  $230




NIPRL
2.3.2 Expectations of Continuous Random Variables (1/2)


 • Example 14 (continuous random variable)
    – The expected diameter of a metal cylinder is
                             50.5
               E( X )             x(1.5  6( x  50.0)2 )dx
                            49.5
        – Change of variable: y=x-50
                            0.5
               E ( x)           ( y  50)(1.5  6 y 2 )dy
                        0.5
                            0.5
                                (6 y 3  300 y 2  1.5 y  75)dy
                        0.5

                      [3 y 4 / 2  100 y 3  0.75 y 2  75 y ]0.5
                                                                0.5

                      [25.09375]  [24.90625]  50.0

NIPRL
2.3.2 Expectations of Continuous Random Variables (2/2)


 • Symmetric Random Variables
    – If x has a p.d.f f ( x) that is
                                         f ( x)   E( X )  
      symmetric about a point 
      so that
            f (   x)  f (   x)

        – Then, E ( X )     (why?)

        – So that the expectation of
          the random variable is equal
          to the point of symmetry
                                                              x



NIPRL
        E ( X )   xf ( x)dx
                                
                  xf ( x)dx +  xf ( x)dx
                  -            

                        y  2  x
                                     
                  xf ( x)dx +   
                  -                  -
                                           yf ( y )dy

              


NIPRL
 2.3.3 Medians of Random Variables (1/2)


 • Median
    – Information about the “middle” value of the random variable
                        F ( x)  0.5
 • Symmetric Random Variable
    – If a continuous random variable is symmetric about a point    ,
      then both the median and the expectation of the random
      variable are equal to 




NIPRL
 2.3.3 Medians of Random Variables (2/2)


 • Example 14


        F ( x)  1.5 x  2( x  50.0)3  74.5  0.5

        x  50.0




NIPRL
 2.4 The variance of a Random Variable
  2.4.1 Definition and Interpretation of Variance (1/2)

 • Variance(  )
                        2

    – A positive quantity that measures the spread of the
      distribution of the random variable about its mean value
    – Larger values of the variance indicate that the distribution is
      more spread out

        – Definition:       Var( X )  E (( X  E ( X ))2 )
                                      E ( X 2 )  ( E ( X ))2

 • Standard Deviation
    – The positive square root of the variance
    – Denoted by 


NIPRL
 2.4.1 Definition and Interpretation of Variance (2/2)

        Var( X )  E (( X  E ( X )) 2 )
                  E ( X 2  2 XE ( X )  ( E ( X )) 2 )
                  E ( X 2 )  2 E ( X ) E ( X )  ( E ( X )) 2
                  E ( X 2 )  ( E ( X )) 2


                                f ( x)
    Two distribution with
    identical mean values but
    different variances




                                                                  x
NIPRL
 2.4.2 Examples of Variance Calculations (1/1)


 • Example 1

        Var( X )  E (( X  E ( X )) 2 )   pi ( xi  E ( X )) 2
                                            i

                   0.3(50  230) 2  0.2(200  230) 2  0.5(350  230) 2
                  17,100   2

          17,100  130.77




NIPRL
 2.4.3 Chebyshev’s Inequality (1/1)


 • Chebyshev’s Inequality
    – If a random variable has a mean  and a variance  2 then
                                                         ,
                                                      1
                    P (   c  X    c )  1 
                                                      c2
                                                      for c  1
        – For example, taking c  2 gives

                                                  1
                   P(   2  X    2 )  1  2  0.75
                                                 2




NIPRL
 • Proof
        
 2    
        
          ( x   )2 f ( x)dx       
                                     
                                  | x  | c
                                                ( x   )2 f ( x)dx  c 2 2      
                                                                                  
                                                                               | x  | c
                                                                                             f ( x)dx.



    P(| x   | c )  1/ c 2

    P(| x   | c )  1  P(| x   | c )  1  1/ c 2




NIPRL
 2.4.4 Quantiles of Random Variables (1/2)


 • Quantiles of Random variables
    – The pth quantile of a random variable X
                           F ( x)  p
        – A probability of p that the random variable takes a value
       less than the pth quantile
 • Upper quartile
    – The 75th percentile of the distribution
 • Lower quartile
    – The 25th percentile of the distribution
 • Interquartile range
    – The distance between the two quartiles



NIPRL
 2.4.4 Quantiles of Random Variables (2/2)


 • Example 14
             F ( x)  1.5 x  2( x  50.0)3  74.5 for 49.5  x  50.5
        – Upper quartile : F ( x)  0.75 x  50.17

        – Lower quartile : F ( x)  0.25    x  49.83

        – Interquartile range :   50.17  49.83  0.34




NIPRL
 2.5 Jointly Distributed Random Variables
  2.5.1 Jointly Distributed Random Variables (1/4)

 • Joint Probability Distributions
    – Discrete
                 P ( X  xi , Y  y j )  pij  0
                                     satisfying           p
                                                          i    j
                                                                   ij   1
        – Continuous


              f ( x, y)  0 satisfying   
                                          state space
                                                        f ( x, y)dxdx  1




NIPRL
 2.5.1 Jointly Distributed Random Variables (2/4)


 • Joint Cumulative Distribution Function
    – Discrete

                 F ( x, y )  P( X  xi , Y  y j )
        – Continuous

                 F ( x, y )      
                                i:xi  x j: y j  y
                                                      pij

                                   x            y
                F ( x, y )                           f ( w, z )dzdw
                                  w        z 




NIPRL
 2.5.1 Jointly Distributed Random Variables (3/4)


 • Example 19 : Air Conditioner Maintenance
    – A company that services air conditioner units in residences
      and office blocks is interested in how to schedule its
      technicians in the most efficient manner
    – The random variable X, taking the values 1,2,3 and 4, is the
      service time in hours
    – The random variable Y, taking the values 1,2 and 3, is the
      number of air conditioner units




NIPRL
 2.5.1 Jointly Distributed Random Variables (4/4)


                                           • Joint p.m.f
                   X=service time
 Y=
 number                                        p
                                               i   j
                                                         ij   0.12  0.18
 of units    1      2        3       4
                                                                    0.07  1.00

    1       0.12   0.08    0.07     0.05
                                           • Joint cumulative
                                             distribution function
    2       0.08   0.15    0.21     0.13
                                              F (2,2)  p11  p12  p21  p22
                                                        0.12  0.18  0.08  0.15
    3       0.01   0.01    0.02     0.07
                                                        0.43



NIPRL
 2.5.2 Marginal Probability Distributions (1/2)


 • Marginal probability distribution
    – Obtained by summing or integrating the joint probability
      distribution over the values of the other random variable
    – Discrete
                  P( X  i )  pi    pij
                                             j

        – Continuous
                                  
                   f X ( x)         f ( x, y ) dy
                              




NIPRL
 2.5.2 Marginal Probability Distributions (2/2)


 • Example 19
    – Marginal p.m.f of X
                           3
              P ( X  1)   p1 j  0.12  0.08  0.01  0.21
                          j 1

        – Marginal p.m.f of Y
                           4
              P(Y  1)   pi1  0.12  0.08  0.07  0.05  0.32
                          i 1




NIPRL
 • Example 20: (a jointly continuous case)

 • Joint pdf:    f ( x, y )
 • Marginal pdf’s of X and Y:


                f X ( x)   f ( x, y)dy
                fY ( y)   f ( x, y)dx




NIPRL
 2.5.3 Conditional Probability Distributions (1/2)


 • Conditional probability distributions
    – The probabilistic properties of the random variable X under
      the knowledge provided by the value of Y
    – Discrete
                                               P( X  i, Y  j ) pij
                  pi| j  P( X  i | Y  j )                   
                                                  P(Y  j )       p j
        – Continuous

                                f ( x, y )
              f X |Y  y ( x) 
                                 fY ( y )
        – The conditional probability distribution is a probability
          distribution.


NIPRL
 2.5.3 Conditional Probability Distributions (2/2)


 • Example 19
    – Marginal probability distribution of Y

           P(Y  3)  p3  0.01  0.01  0.02  0.07  0.11
        – Conditional distribution of X
                                            p13 0.01
            p1|Y 3    P ( X  1| Y  3)           0.091
                                            p3 0.11




NIPRL
 2.5.4 Independence and Covariance (1/5)


 • Two random variables X and Y are said to be independent if
    – Discrete
        pij  pi  p j for all values iof  X and jof Y
        – Continuous

            f ( x, y)  f X ( x) fY ( y) for all x and y

        – How is this independency different from the independence
          among events?




NIPRL
 2.5.4 Independence and Covariance (2/5)

 • Covariance
           Cov( X , Y )  E (( X  E ( X ))(Y  E (Y )))
                         E ( XY )  E ( X ) E (Y )

           Cov( X , Y )  E (( X  E ( X ))(Y  E (Y )))
                         E ( XY  XE (Y )  E ( X )Y  E ( X ) E (Y ))
                         E ( XY )  E ( X ) E (Y )  E ( X ) E (Y )  E ( X ) E (Y )
                         E ( XY )  E ( X ) E (Y )

        – May take any positive or negative numbers.
        – Independent random variables have a covariance of zero
        – What if the covariance is zero?

NIPRL
 2.5.4 Independence and Covariance (3/5)


 • Example 19 (Air conditioner maintenance)


        E ( X )  2.59,           E (Y )  1.79
                  4    3
        E ( XY )   ijpij
                 i 1 j 1

                (11 0.12)  (1 2  0.08)
                                 (4  3  0.07)  4.86
        Cov( X , Y )  E ( XY )  E ( X ) E (Y )
                     4.86  (2.59 1.79)  0.224


NIPRL
 2.5.4 Independence and Covariance (4/5)


 • Correlation:

                                Cov( X , Y )
               Corr( X , Y ) 
                               Var( X )Var(Y )

        – Values between -1 and 1, and independent random
          variables have a correlation of zero




NIPRL
 2.5.4 Independence and Covariance (5/5)


 • Example 19: (Air conditioner maintenance)


            Var( X )  1.162, Var(Y )  0.384
                             Cov( X , Y )
            Corr( X , Y ) 
                            Var( X )Var(Y )
                              0.224
                                         0.34
                           1.162  0.384




NIPRL
 • What if random variable X and Y have linear relationship, that is,
        Y  aX  b
    where     a0
            Cov( X , Y )  E[ XY ]  E[ X ]E[Y ]
             E[ X (aX  b)]  E[ X ]E[aX  b]
             aE[ X 2 ]  bE[ X ]  aE 2 [ X ]  bE[ X ]
             a ( E[ X 2 ]  E 2 [ X ])  aVar ( X )
                          Cov( X , Y )          aVar ( X )
        Corr ( X , Y )                    
                         Var ( X )Var (Y )   Var ( X )a 2Var ( X )
        That is, Corr(X,Y)=1 if a>0; -1 if a<0.


NIPRL
 2.6 Combinations and Functions of Random
 Variables
 2.6.1 Linear Functions of Random Variables (1/4)
 • Linear Functions of a Random Variable
    – If X is a random variable and Y  aX  b
      for some numbers a, b  R then E (Y )  aE( X )  b
      and Var(Y )  a 2 Var( X )

 • Standardization
   -If a random variable X has an expectation of  and a
   variance of  ,      X  1       
                 2

                    Y             X  
                                      
    has an expectation of zero and a variance of one.



NIPRL
 2.6.1 Linear Functions of Random Variables (2/4)


 • Example 21:Test Score Standardization
    – Suppose that the raw score X from a particular testing
      procedure are distributed between -5 and 20 with an
      expected value of 10 and a variance 7. In order to
      standardize the scores so that they lie between 0 and 100,
      the linear transformation Y  4 X  20is applied to the
      scores.




NIPRL
 2.6.1 Linear Functions of Random Variables (3/4)


        – For example, x=12 corresponds to a standardized score of
          y=(4ⅹ12)+20=68


                E (Y )  4E( X )  20  (4 10)  20  60

                Var(Y )  42 Var( X )  42  7  112

                 Y  112  10.58




NIPRL
 2.6.1 Linear Functions of Random Variables (4/4)


  • Sums of Random Variables
     – If X 1 and X 2 are two random variables, then
          E ( X1  X 2 )  E ( X1 )  E ( X 2 )  (why ?)
          and
         Var( X1  X 2 )  Var( X1 )  Var( X 2 )  2Cov( X1, X 2 )
        – If X 1 and X 2 are independent, so that Cov( X 1 , X 2 )  0
          then
             Var( X1  X 2 )  Var( X1 )  Var( X 2 )




NIPRL
 • Properties of       Cov( X 1 , X 2 )
   Cov( X 1 , X 2 )  E[ X 1 X 2 ]  E[ X 1 ]E[ X 2 ]

    Cov( X 1 , X 2 )  Cov( X 2 , X 1 )
        Cov( X 1 , X 1 )  Var ( X 1 ) Cov( X 2 , X 2 )  Var ( X 2 )
        Cov( X 1  X 2 , X 1 )  Cov( X 1 , X 1 )  Cov( X 2 , X 1 )
        Cov( X1  X 2 , X1  X 2 )  Cov( X1 , X1 )  Cov( X1 , X 2 ) 
        Cov( X 2 , X1 )  Cov( X 2 , X 2 )

    Var ( X 1  X 2 )  Var ( X 1 )  Var ( X 2 )  2Cov( X 1 , X 2 )

NIPRL
 2.6.2 Linear Combinations of Random Variables (1/5)


 • Linear Combinations of Random Variables
    – If X 1 , , X n is a sequence of random variables and a1 ,                , an
      and b are constants, then
           E (a1 X 1     an X n  b)  a1E ( X 1 )      an E ( X n )  b

        – If, in addition, the random variables are independent, then

          Var(a1 X 1     an X n  b)  a1 Var( X1 )       an Var( X n )
                                           2                    2




NIPRL
 2.6.2 Linear Combinations of Random Variables (2/5)

 • Averaging Independent Random Variables
    – Suppose that X 1 , , X n is a sequence of independent
      random variables with an expectation  and a variance  2.


        – Let         X1          Xn
                 X
                              n
        – Then
                  E( X )  
          and
                              2
                 Var( X ) 
                               n
        – What happened to the variance?



NIPRL
 2.6.2 Linear Combinations of Random Variables (3/5)


                   1      1       1                     1
        E( X )  E  X1     X n   E( X1)               E( X n )
                   n      n       n                     n
                 1      1
                       
                 n      n



                                                2                      2
                       1          1     1                    1
        Var( X )  Var  X 1      X n     Var( X 1 )        Var( X n )
                       n          n     n                    n
                                  1 2 
                      2               2
                 1
                                           2
                  2            
                 n              n     n



NIPRL
 2.6.2 Linear Combinations of Random Variables (4/5)


 • Example 21
    – The standardized scores of the two tests are
                     10             5      50
                 Y1  X1 and Y2  X 2 
                      3             3       3
    – The final score is
                    2    1     20     5      50
                 Z  Y1  Y2     X1  X 2 
                    3    3     9      9       9




NIPRL
 2.6.2 Linear Combinations of Random Variables (5/5)

        – The expected value of the final score is
                             20          5            50
                   E (Z )      E( X1 )  E ( X 2 ) 
                              9          9            9
                             20      5         50
                            18     30  
                             9       9         9
                           62.22

        – The variance of the final score is
                                    20     5     50 
                    Var( Z )  Var     X1  X 2  
                                    9      9     9 
                                    2                 2
                                20              5
                                 Var( X 1 )     Var( X 2 )
                                9               9
                                    2           2
                                20      5
                                 24     60  137.04
                                9       9

NIPRL
 2.6.3 Nonlinear Functions of a Random Variable (1/3)


 • Nonlinear function of a Random Variable
    – A nonlinear function of a random variable X is another
      random variable Y=g(X) for some nonlinear function g.
                    Y  X 2, Y  X , Y  eX

        – There are no general results that relate the expectation and
          variance of the random variable Y to the expectation and
          variance of the random variable X




NIPRL
 2.6.3 Nonlinear Functions of a Random Variable (2/3)
        – For example,                       f(x)=1
                                                              E(x)=0.5
            f X ( x)  1 for 0  x  1
            f ( x)  0 elsewhere

                                                          0              1   x
          FX ( x)  x for 0  x  1
                                         f(y)=1/y
                                                          E(y)=1.718
        – Consider

          Y  eX
                where 1  Y  2.718


        – What is the pdf of Y?
                                                    1.0                  2.718   y


NIPRL
 2.6.3 Nonlinear Functions of a Random Variable (3/3)

 • CDF methd

         FY ( y )  P(Y  y )  P(e X  y )  P( X  ln( y ))  Fx (ln( y ))  ln( y )

        – The p.d.f of Y is obtained by differentiation to be
                                       dFY ( y ) 1
                       fY ( y )                                 for 1  y  2.718
                                         dy       y
 • Notice that
                               2.718                     2.718
                  E (Y )             zfY ( z )dz             1dz  2.718  1  1.718
                            z 1                     z 1

                  E (Y )  e E ( X )  e0.5  1.649




NIPRL
 • Determining the p.d.f. of nonlinear relationship between r.v.s:
   Given f X ( x) and Y  g ( X ) ,what is fY ( y ) ?

    If    x1 , x2 ,    , xn    are all its real roots, that is,

             y  g ( x1 )  g ( x2 )          g ( xn )
   then                    n
                                f X ( xi )
              fY ( y )  
                         i 1 | g '( xi ) |

   where
                       dg ( x)
             g '( x) 
                        dx

NIPRL
 •   Example: determine   fY ( y )
     f X ( x)  1 for 0  x  1      Y  eX
     f ( x)  0 elsewhere                   where 1  Y  2.718

     y  g ( x)  e x
      ln y  x             -> one root is possible in   0  x 1
     dg
           ex  y
     dx
                  1   1
     fY ( y )      
                | y| y

NIPRL

				
DOCUMENT INFO