Multiple Regression by TBd92btU

VIEWS: 19 PAGES: 15

									                        Multiple Regression                              7.1

• More than one explanatory/independent variable

           yt  1   2 x2t   3 x3t  ...  k xkt  et
           yt  E ( yt )  et
• This makes a slight change to the interpretation of the coefficients
• This changes the measure of degrees of freedom
• We need to modify one of the assumptions

EXAMPLE: trt = 1 + 2 pt + 3 at + e

EXAMPLE: qdt = 1 + 2 pt + 3 inct + et

EXAMPLE: gpat = 1 + 2 SATt + 3 STUDYt + et
                       Interpretation of Coefficient             7.2

            yt  1   2 x2t   3 x3t  et
            dy
                        2
            dx2   x3

            dy
                        3
            dx3   x2

• 2 measures the change in Y from a change in X2, holding X3
  constant.
• 23 measures the change in Y from a change in X3, holding X2
  constant.
       Assumptions of the Multiple Regression Model                    7.3



1. The Regression Model is linear in the parameters and error term
       yt = 1 + 2 x2t + 3 x3t + … k xkt +et

2. Error Term has a mean of zero:
         E(e) = 0  E(y) = 1 + 2 x2t + 3 x3t + … k xkt
3. Error term has constant variance: Var(e) = E(e2) = 2
4. Error term is not correlated with itself (no serial correlation):
   Cov(ei,ej) = E(eiej) = 0 ij
5. Data on x’s are not random (and thus are uncorrelated with the
   error term: Cov(X,e) = E(Xe) = 0) and they are NOT exact linear
   functions of other explanatory variables.
6. (Optional) Error term has a normal distribution. E~N(0, 2)
         Estimation of the Multiple Regression Model                     7.4



• Let’s use a model with 2 independent variables:
        yt  1   2 x2t   3 x3t  et
• A scatterplot of points is now a scatter “cloud”. We want to fit the
  best “line” through these points. In 3 dimensions, the line becomes
  a plane.
• The estimated “line” and a residual are defined as before:

            yt  b1  b2 x2t  b3 x3t
            ˆ
            et  yt  yt
            ˆ         ˆ
• The idea is to choose values for b1, b2, and b3 such that the sum of
  squared residuals is minimized.
  et2   ( yt  yt ) 2
 ˆ              ˆ                                                            7.5

  ( yt  b1  b2 x2t  b3 x3t ) 2

From here, we minimize this expression with respect to b1,
b2, and b3. We set these three derivatives equal to zero and
Solve for b1, b2, b3. We get the following formulas:



 b2 
         yt* x2t  x3t2   yt* x3t  x2t x3t
               *     *            *     * *


              *
                  x22
                    t     x3t2
                           *
                                      * * 2
                                      x2t x3t

 b3 
       yt* x3t  x22   yt* x2t  x2t x3t
             *     *
                    t
                               *     * *                       Where:


          x2t  x3t   x2t x3t                                   ( yt  y )
                               * * 2                            *
                *2    *2                                       yt
                                                               x2t  ( x2t  x2 )
                                                                *

 b1  y  b2 x2  b3 x3
                                                               x3t  ( x3t  x3 )
                                                                *
[What is going on here? In the formula for b2, notice that if      7.6
x3 where omitted from the model, the formula reduces to the
familiar formula from Chapter 3.]




You may wonder why the multiple regression formulas on slide 7.5
aren’t equal to:

b2 
      ( yt  y )(x 2t  x2 )
           ( x 2t  x2 ) 2

b3 
      ( yt  y )(x3t  x3 )
           ( x 3t  x3 ) 2
We can use a Venn diagram to illustrate the idea of Regression as   7.7
Analysis of Variance

For Bivariate (Simple) Regression



    y
                        x


For Multiple Regression


        y
                        x2


            x3
               Example of Multiple Regression                           7.8


Suppose we want to estimate a model of home prices using data
on the size of the house (sqft), the number of bedrooms (bed) and the
number of bathrooms (bath). We get the following results:


  ˆ
pri cet  129.062  0.1548sqftt  21.588bedt 12.193batht

How does a negative coefficient estimate on bed and bath make
sense?
                                                                               7.9
                               Expected Value

 E (b1 )  1                      We will omit the proofs. The Least
 E (b2 )   2                     Squares estimator for multiple regression
                                   is unbiased, regardless of the number of
 E (b3 )   3                     independent variables


     Variance Formulas With 2 Independent Variables

                        2                   Where r23 is the correlation
Var(b2 ) 
             (1  r23 ) ( x2t  x2 ) 2
                   2                         between x2 and x3 and the
                                             parameter 2 is the variance
                        2                   of the error term.
Var(b3 ) 
             (1  r23 ) ( x3t  x3 ) 2
                   2




                         2
We need to estimate using the formula                  
                                                      2    ˆ
                                                              et2
This estimate has T-k degrees of freedom.                    T k
                     Gauss Markov Theorem                                 7.10



Under the assumptions 1-5 (the 6th assumption isn’t needed for the
  theorem to be true) of the linear regression model, the least squares
  estimators b1, b2, …bk have the smallest variance of all linear and
  unbiased estimators of 1 , 2,… k. They are the BLUE (Best,
  linear, unbiased, estimator)
         Confidence Intervals and Hypothesis Testing                          7.11

• The methods for constructing confidence intervals and conducting
  hypothesis tests are the same as they were for simple regression.

• The format for a confidence interval is:

           bi  tc se(bi )
   Where tc depends on the level of confidence and has T-k degrees of
   freedom. T is the number of observations and k is the number of
   independent variables plus one for the intercept.

• Hypothesis Tests:
                                                t
                                                  bi   i 
    Ho : i = c
    H1 : i  c
                                                   se(bi )
Use the value of c for i when calculating t. If t > tc or t < - tc  reject Ho
If c is 0, then we call it a test of significance.
                           Goodness of Fit                                  7.12

• R2 measures the proportion of the variance in the dependent
  variable that is explained by the independent variable. Recall that

                R 
                  2 SSR
                         1
                             SSE
                                  1
                                         et2ˆ
                    SST      SST       ( yt  y ) 2

• Least Squares chooses the line that produces the smallest sum of
  squared residuals, it also produces the line with the largest R2. It
  also has the property that the inclusion of additional independent
  variables will never increase and will often lower the sum of
  squared residuals, meaning that R2 will never fall and will often
  increase when new independent variables are added, even if the
  variables have no economic justification.
• Adjusted R2: adjust R2 for degrees of freedom

                                         R  1
                                           2            et2 /(T  k )
                                                         ˆ
                                                   ( yt  y ) 2 /(T  1)
                     Example: Grades at JMU                            7.13


 A sample of 55 JMU students was taken Fall 2002. Data on
    • GPA
    • SAT scores
    • Credit Hours Completed
    • Hours of Study per Week
    • Hours at a Job per week
    • Hours at Extracurricular Activites
Three models were estimated:
gpat = 1 + 2 SATt + et

gpat = 1 + 2 SATt + 3 CREDITSt + 4 STUDYt + 5JOBt + 6 ECt + et

gpat = 1 + 2 SATt + 3 CREDITSt + 4 STUDYt + 5JOBt + et
                                                                                                  7.14
                             Regression Statistics
Here is our simple   Multiple R             0.270081672
Regression model.    R Square               0.072944109
                     Adjusted R Square      0.055452489
                     Standard Error         0.455494651
                     Observations                    55



                                          Coefficients Standard Error   t Stat     P-value
                     Intercept            1.799734629    0.631013633 2.852132721 0.006178434
                     SAT Total            0.001057599    0.000517894 2.042114498 0.046129915

                                 Regression Statistics
Here is our multiple     Multiple R             0.465045986
regression model.        R Square               0.216267769
                         Adjusted R Square      0.136295092
Both R2 and              Standard Error            0.4355661
Adjusted R2 have         Observations                     55
increased with the
inclusion of                                   Coefficients Standard Error      t Stat       P-value
4 additional indep.      Intercept             1.538048132    0.686920438    2.239048435   0.029726041
                         SAT Total             0.000943615    0.000503684    1.873427614   0.066980023
variables.                                     0.003382201
                         Credit Hours (completed)             0.002664489    1.269361823   0.210308107
                         Study (Hrs/wk)        0.010762866    0.006782587    1.586837918   0.118982276
                         Job (Hrs/wk)           -0.00795843   0.006452798    -1.23333021    0.22333641
                         EC (Hrs/wk)           0.002606617    0.009216993    0.282805582   0.778517157
                                                                                                 7.15
                             Regression Statistics
                     Multiple R             0.463668569
Notice that the      R Square               0.214988542
                     Adjusted R Square      0.152187625
Exclusion of EC      Standard Error         0.431540195
increases adjusted   Observations                    55

R2 but reduces
R2                   Intercept
                                           Coefficients Standard Error
                                             1.53616637   0.680539354
                                                                            t Stat
                                                                         2.257277792
                                                                                         P-value
                                                                                       0.028390614
                     SAT Total             0.000951057    0.000498346    1.908425924   0.062085531
                                           0.003441283
                     Credit Hours (completed)             0.002631734    1.307610208   0.196986336
                     Study (Hrs/wk)        0.011267319    0.006483348    1.737885925   0.088386942
                     Job (Hrs/wk)             -0.0080306  0.006388155    -1.25710723   0.214555464

								
To top