Speciﬁcation Tests for Linear Regression Models
E-mail: ba firstname.lastname@example.org
(Note that this is a lecture note. Please refer to the textbooks suggested in the course outline for
details. Examples will be given and explained in the class)
We have learnt that the OLS estimators can provide consistent estimates under a certain set of
assumptions about the noise and the regressors in a linear regression equation. The ﬁrst objective
of this lecture is to show that if this set of assumptions is not fulﬁlled then the OLS estimators no
longer have desirable properties. The second objective is to provide some typical methods to check,
for a given data set, if a model satisﬁes the underlying assumptions required by the OLS estimator.
2 Speciﬁcation Error
The speciﬁcation of the linear model centers on the noise vector u and the regressors X. The
assumptions made in Lecture 2 were as follows:
y = Xβ + u, (2.1)
1. ut are IID (0, σ 2 ) or IID N (0, σ 2 ).
X u = 0 (orthogonality condition).
3. X is non-stochastic with full column rank K.
In reality, we face the problems listed below.
• u are heteroskedastic, i.e., E[u u] = diag(σ1 , . . . , σT ).
• E[ut ut−s ] = 0.
• Omitted variables.
• Irrelevant variables
Xu = 0.
Now, we analyze the consequences of the above problems on statistical inference.
2.1 Omitted variables
Suppose that the true data is generated by the process
y = X1 β1 + X2 β2 + u, (2.2)
T ×K1 T ×K2
where E[u] = 0 and E[u u] = σ 2 IT , and K1 + K2 = K. However, our econometrician estimates
the following equation:
y = X1 β1 + u1 .
In this equation, the variable X2 is omitted. The OLS estimate of β1 is
β1 = (X1 X1 )−1 X1 y.
Using y in Equation (2.2), it is straight-forward to prove that
β1 = β1 + (X1 X1 )−1 X1 X2 β2 + (X1 X1 )−1 X1 u,
E[β1 ] = β1 + (X1 X1 )−1 X1 X2 β2 .
The OLS estimate is unbiased iﬀ X1 X2 = 0.
u = y − X1 β1 = y − X1 (X1 X1 )−1 X1 y = M1 y,
where M1 = X1 (X1 X1 )−1 X1 and rank(M1 ) = T − K1 , we obtain
u1 u1 = y M1 y = u M1 u + β2 X2 M1 X2 β2 + 2β2 X2 M1 u.
E[u1 u1 ] = σ 2 (T − K1 ) + β2 X2 M1 X2 β2 .
E[σ 2 ] = E[u1 u1 ]/(T − K1 ) = σ 2 + β X M1 X2 β2 > σ 2 .
T − K1 2 2
The regression model with omitted variables always gives biased estimators.
2.2 Irrelevant variables
Conversely, we can assume that the true model is
y = X1 β1 + u, (2.3)
where E[u] = 0 and E[u u] = σ 2 IT . However, our econometrician estimate the following model:
y = X1 β1 + X2 β2 + e.
The OLS estimates are given by
β1 = (X1 M2 X1 )−1 X1 M2 y,
β2 = (X2 M1 X2 )−1 X2 M1 y,
where M2 = IT − X2 (X2 X2 )X2 and M1 = IT − X1 (X1 X1 )X1 .
We can show that
β1 = (X1 M2 X1 )−1 X1 M2 (X1 β1 + u)
= (X1 M2 X1 )−1 X1 M2 X1 β1 + (X1 M2 X1 )−1 X1 M2 u
= β1 + (X1 M2 X1 )−1 X1 M2 u. (2.4)
By the assumption of u, we can see that E[β1 ] = β1 . Thus, the OLS estimate is unbiased.
Let X = X1 , X2 and M = IT − X(X X)−1 X , the sum of squared residuals is
T ×K1 T ×K2
T ×(K1 +K2 )
e e = y My
= (X1 β1 + u) M (X1 β1 + u)
∗ ∗ ∗
= u M u + β1 X1 M X1 β1 + 2β1 X1 M u
= u M u,
the last equation comes from the fact that M X = 0. Hence, we can show that
E[σ 2 ] = E[ ] = σ2.
T − (K1 + K2 )
The OLS esimate of σ 2 is unbiased.
Including irrelevant regressors still yields unbiased OLS estimators.
3 Regression Speciﬁcation Test
We can test whether u in the regression y = X β +u is uncorrelated with X by testing whether
T ×K K×1
γ in the regression equation y = Xβ + γy + is zero. This can be done in 3 steps:
1. Run the OLS estimate of y = X β.
2. Run the OLS estimate of y = X β + γ y 2 + .
u u−b b
3. Construct a F test, i.e., F = b b/(T −K−1)
∼ F (1, T − K − 1), where u = y − y.
4 Chow’s Test for Parameter Constancy
Suppose that β is not constant throughout the time, i.e.,
y 1 X1 0 β1
T ×1 T ×K K×1 u1
1 = 1 + ,
y2 0 X2 β2 u2
T2 ×1 T2 ×K K×1
where T1 + T2 = T . The null hypothesis is H0 : β1 = β2 ; and the alternative hypothesis is
H1 : β1 = β2 .
The unrestricted (i.e, under H1 ) OLS estimate of β1 and β2 is
β1 X1 X1 0 X1 y (X1 X1 ) X1 y
= = .
β2 0 X2 X2 X2 y (X2 X2 )−1 X2 y
The sum of squared residuals of the unrestricted OLS equation is u u = u1 u1 + u2 u2 with u1 =
y − X1 β1 and u2 = y − X2 β2 .
The restricted (i.e., under H0 ) OLS estimate of β1 and β2 is
β = (X X)−1 X y,
X = ( X1 , X2 ).
T ×K T ×K1 T ×K2
The sum of squared residuals of the restricted OLS equation is u u with u = y − X β.
If T1 and T2 are both greater than K, then
F(β1 =β2 ) = (T − 2K)/K ∼ F (K, T − 2K).
If T1 > K while T2 < K, then
u u − u1 u1
F(β1 =β2 ) = (T1 − K)/T2 ∼ F (T2 , T1 − K).
5 Recursive Estimation for Parameter Constancy
yt = xt β + yt , ∀ t = 1, . . . , T,
where xt = (1, x2t , . . . , xKt ).
The idea is very simple. Fit the model to the ﬁrst K observations. Next increase the number of
observations by K + 1 (i.e., add 1 more observation to the ﬁrst sample) and compute the coeﬃcient
vector β again. By repeating this process till the end of the sample, we can obtain a sequence of
vectors β1 , β2 , . . . , βT −K . Finally, using visual graphics to check for the constancy of this sequence.
6 Dummy Variables
Since the material of the section is rather elementary, you can refer to Section 4.6 in JD97. This
will not be in the exams.
7 Exercises to Practise
1. In the model
yt = βxt + ut , (7.1)
for t = 1, . . . , T , where xt is non-stochastic and ut ∼ N (0, σ 2 ), the parameter β is assumed to
change at a certain point in the sample, i.e.,
β1 , t = 1, . . . , s
β , t = s + 1, . . . , T.
Suppose an econometrician ignores the change in β and assume that β is constant throughout
the sample. Derive an expression for the bias of the OLS estimate β as an estimate of β1 .
2. Consider the multiple regression model
y = X1 β1 + X2 β2 + u,
where u ∼ N (0, σ 2 IT ); and X1 and X2 are K1 × T and K2 × T matrices, respectively.
(a) Using the normal equations, show that the OLS estimate of β1 is β1 = (X1 X1 )−1 X1 (y −
X2 β2 ), where β2 is the OLS estimate of β2 .
(b) Show that, under the restriction β2 = 0, the restricted least-squares estimate of β =
is , where β1 is the OLS estimator of the parameter in the regression of y
on X1 .
3. Consider the regression model
y t = xt β + u t ,
where xt = (1, x1t , . . . , xKt ) is non-stochastic, E[ut ] = 0 and
σ , t = 1, 2, . . . , T1
E[ut ] =
σ 2 , t = T + 1, . . . , T.
Assuming that ut is independently and normally distributed (NID), derive the Likelihood
2 2 2
Ratio (LR) and Lagrange Multiplier (LM) statistics for tesing H0 : σ1 = σ2 against H1 : σ1 =
4. Given the model
yt = β0 + β1 X1t + β2 X2t + ut ,
where ut ∼ N ID(0, σ 2 ), for t = 1, . . . , T , a sample size T = 17 gives
(yt − y)2 = 386, (X1t − X 1 )2 = 2100, (X2t − X 2 )2 = 1272,
(X1t −X 1 )(X2t −X 2 ) = −1521, (X1t −X 1 )(yt −y) = −859, (X2t −X 2 )(yt −y) = 682,
where X 1 = 100, X 1 = 55 and y = 43 denote the sample means of X1t , X2t and yt , respec-
tively; and the all the summations run from 1 to 17.
Use the LR, LM and W test statistics to test H0 : 2β1 + β2 = 0 against H1 : 2β1 + β2 = 0.
5. Problems 4.1 and 4.10 in JD97.