Elementary properties of the covariance matrix

Document Sample
Elementary properties of the covariance matrix Powered By Docstoc
					Peter B. Lerner Homework #3

Prof. Kao’s class

Elementary spectral properties of the covariance matrix

~ 1. “While S T is consistent, in need not be positive semi-definite in any finite sample when m is not
zero”. 1 This statement follows from spectral representation of the covariance matrix (equivalent result in probability theory is called Wiener-Khinchine theorem, see 2) and the following observation on the properties of Fourier transform. Example 1

~ Let f ( ) be the Fourier transform of the function f (t ) . Then the following is true for any point of ~ continuity of the function f ( )
~ ~ ~ lim 2f (  i ) f  (  i ) | f ( ) | 2
 0

(1)

~ ~ ~ yet 2f (  i ) f  (  i ) may not even be real for any ε. For analytic f ( ) , we can write
~ ~ ~  2 ~" f (  i )  f ( )  if ' ( )  f ( )  ... 2 ~ ~ ~  2 ~"  f (  i )  f  ( )  i ( f ' )  ( )  ( f ) ( )  ... 2

(2)

and the above product is indeed semi-definite in any order of ε, but analyticity of the Fourier transform requires that the function f (t ) falls off faster than any power for t   , which is not always the case. Moreover, one can argue that we introduce the cut-off kernel specifically to enforce that property.

2. Let
1

Here we provide a proof of Wiener-Khinchine theorem in an elementary setting.

W. K. Newey, K. D. West, Econometrica, 55 (1987), p. 703

1

Peter B. Lerner Homework #3

Prof. Kao’s class

ht 

 h e  
 

 

i t

(3)

Then
j  j      j     

j  

 E (ht ht  j ) 
j  j    j   j     

j  

  
2

    

 E (h h  )e i t i (t  j ) 
 lim
 
N 2 N    N

j    i j

   E (h h )e
      2

i (   ) t ij


(4)



   | h |

e

 i ( j  j  )

 | h |  e 

sin 2 ( N )  lim  | h | 2  2 | h0 | 2 N  sin 2 ( )   
 

Third equality sign is the consequence of definition Eq. (3), in the fourth equality we changed an order of summation and the last equality sign is the result of convergence of sinusoidal-square (Völler) kernel to the 2 ( ) . This reasoning is not strict but each step can be justified by rigorous estimates.

3.

We expand the variation of the sum of random variables:

m m  m  m Var   ht  j    Var (ht  j )  2 Cov (ht ht  j )    j 1 t 1  j 1  j 1

 E (h j h Tj )  2
j 1 j m

m

j   m , t 1 j 0



m

m j

 E (ht ht  j )  E (h j h Tj ) 
j 1

m

j  m, j 0

 (m  j ) E (h h
m t

T t j

)  (m  j ) E (ht htT j ) 



(5)

j m  | j | T (m | j |)E (ht htT j )  m  1  E (ht ht  j ) m m  j  j  m 

We can approximate expectation in the last equation by its sample average (Newey and West, Theorem 1).

4. Repeating the argument of the previous paragraph:
j m

j  m

 w( j, m)(

 m  1 T ht ht  j )  Varsam ple  ht  j   0 (6)    T t 1  j 1 

5.

We have AR(1) process:

2

Peter B. Lerner Homework #3

Prof. Kao’s class

ht  ht 1   t

(7)

Expansion of the ht provides:
ht   t   t 1   2  t  2  ...(a) ht  j   t  j    jt 1   2  t  j  2  ...(b)
(8)

Taking variances of both sides of Eq. (8a) and using the i.i.d. property of  t , t ,Var ( t )   2 we obtain:

Var[ht ]   2   2 2   4 2  ... 

 2 1  2

(9)

For computation of the covariance we notice, that, because of i.i.d. property, series under the expectation sign begin with the j-th term in regression (8a) and the first term in regression (8b) and
E[ht ht  j ]   j  E[  j  t  j  t  j   j 1 t  j 1 t  j 1  ...], j  0  2 ,j0    (1   )  0 0, j  0  
(10)

 E[ht ht  j ]   2 
j 0

j 

j 

j 1   2 (1   ) 2 j  0 (1   )

(11)

Q.E.D.

6. Mixing is a property of random processes, which is weaker than ergodicity but is used essentially for the same purpose: namely to assure interchangeability between a limit sign and probability measure. This allows proofs of uniform convergence in the law of large numbers and CLT.

3

Peter B. Lerner Homework #3

Prof. Kao’s class

There are many non-equivalent definitions of mixing. The naïve idea of mixing is the following2: consider an shift operator of the random process ξ(t):
S u (t )   (t  u )

Then, mixing means that for t   the points ω of the set StA1 are uniformly distributed over the entire space Ω, in the sense that for any set in this space, the points ω of StA1 have a measure, which is proportional to the measure of this set:
lim P( S t A1 )  A2   P( A1 ) P( A2 )

t 

(12)

This property is called “uniform mixing”. Formal definitions for the most popular types of mixing provided in (Zhenguan, Chuanzhong, 1996) are listed below.3 First, one defines the distance between two subfields A and B of algebra F in one of the following fashions:

 ( A, B)  sup | P( AB)  P( A) P( B) |
A, B

(13)

 ( A, B)  sup
A, B

| E ( AB)  E ( A) E ( B) | VarAVarB

(14)

 ( A, B)  sup | P( B | A)  P( B) |
A, B

(15)

and

Fab   ( X i , a  i  b)
Then α- (respectively (ρ, φ))-mixing means that
2 3

Yu. V. Prokhorov, Ya. A. Rozanov, Probability Theory. Lin Zhenguan, Lu Chuanzhong, Limit Theory for Mixing Dependent Random Variables, (Kluwer, Dordrecht, 1996).

4

Peter B. Lerner Homework #3

Prof. Kao’s class

sup  ( F1k , Fn k )  0, n   
k

(16)

Mixing property assures that serial correlations between two random processes decrease with time. This property is important for convergence as one can observe from the following example. 4

Example 2 We estimate the coefficient  n for two scalar processes X t , Z t and E ( X t2 )  1 :

 n  n 1  E ( X t Z t )
t 1

n

(17)

Suppose a sequence for estimation of X t , Z t begins with:
1,3,1,1,3,3,1,1,1,1,...

and repeats indefinitely. Then the coefficient  n will oscillate between 1.5 and 2 no matter how long the sample is. This example illustrates that the mixing sequence is the sequence, which sweeps the probability space more or less uniformly. In their Theorem 2, Condition (iii), Newey and West assume mixing sequences  (l ) ~ O(l   ) ,

 (l ) ~ O(l  ) where   2r /(2r  1) , r  1 . Otherwise, there will be no assurance that the bandwidth will
have appropriate growth rate. In particular, inequality of Eq. (11), Newey and West can hold only if m(T) grows slower than T1/4.

4

Domowitz and White. Journal of Economics, 20 (1982) 35-58.

5


				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:7
posted:1/7/2010
language:English
pages:5