Document Sample

Connexions module: m13500 1 Maximum Likelihood Estimation - ∗ Examples Ewa Paszek This work is produced by The Connexions Project and licensed under the † Creative Commons Attribution License Abstract This course is a short series of lectures on Introductory Statistics. Topics covered are listed in the Table of Contents. The notes were prepared by Ewa Paszek and Marek Kimmel. The development of this course has been supported by NSF 0203396 grant. 1 MAXIMUM LIKELIHOOD ESTIMATION - EXAMPLES 1.1 EXPONENTIAL DISTRIBUTION Let X1 , X2 , ..., Xn be a random sample from the exponential distribution with p.d.f. 1 −x/θ f (x; θ) = e , 0 < x < ∞, θ ∈ Ω = {θ; 0 < θ < ∞}. θ The likelihood function is given by n 1 −x1 /θ 1 −x2 /θ 1 −xn /θ 1 − i=1 xi L (θ) = L (θ; x1 , x2 , ..., xn ) = e e ··· e = exp , 0 < θ < ∞. θ θ θ θn θ The natural logarithm of L (θ) is n 1 lnL (θ) = − (n) ln (θ) − xi , 0 < θ < ∞. θ i=1 Thus, n d [lnL (θ)] −n i=1 xi = + = 0. dθ θ θ2 The solution of this equation for θ is n 1 θ= xi = x. n i=1 ∗ Version 1.3: Oct 8, 2007 3:44 pm GMT-5 † http://creativecommons.org/licenses/by/2.0/ http://cnx.org/content/m13500/1.3/ Connexions module: m13500 2 Note that, d [lnL (θ)] 1 nx = −n + > 0, θ < x, dθ θ θ d [lnL (θ)] 1 nx = −n + = 0, θ = x, dθ θ θ d [lnL (θ)] 1 nx = −n + < 0, θ > x, dθ θ θ Hence, lnL (θ) does have a maximum at x, and thus the maximum likelihood estimator for θ is n ^ 1 θ= X = Xi . n i=1 This is both an unbiased estimator and the method of moments estimator for θ. 1.2 GEOMETRIC DISTRIBUTION Let X1 , X2 , ..., Xn be a random sample from the geometric distribution with p.d.f. x−1 f (x; p) = (1 − p) p, x = 1, 2, 3, .... The likelihood function is given by P x1 −1 x2 −1 xn −1 xi −n L (p) = (1 − p) p(1 − p) p · · · (1 − p) p = pn (1 − p) , 0 ≤ p ≤ 1. The natural logarithm of L (θ) is n lnL (p) = nlnp + xi − n ln (1 − p) , 0 < p < 1. i=1 Thus restricting p to 0<p<1 so as to be able to take the derivative, we have n dlnL (p) n xi − n i=1 = − = 0. dp p 1−p Solving for p, we obtain n 1 p= n = . i=1 xi x So the maximum likelihood estimator of p is ^ n 1 p= n = i=1 Xi X Again this estimator is the method of moments estimator, and it agrees with the intuition because, in n n observations of a geometric random variable, there are n successes in the i=1 xi trials. Thus the estimate of p is the number of successes divided by the total number of trials. http://cnx.org/content/m13500/1.3/ Connexions module: m13500 3 1.3 NORMAL DISTRIBUTION Let X1 , X2 , ..., Xn be a random sample from N (θ1 , θ2 ), where Ω = ((θ1 , θ2 ) : −∞ < θ1 < ∞, 0 < θ2 < ∞) . That is, here let θ1 = µ and θ2 = σ 2 . Then n 2 1 (xi − θ1 ) L (θ1 , θ2 ) = √ exp − , i−1 2πθ2 2θ2 or equivalently, n n 2 1 − i=1 (xi − θ1 ) L (θ1 , θ2 ) = √ exp − , (θ1 , θ2 ) ∈ Ω. 2πθ2 2θ2 The natural logarithm of the likelihood function is n 2 n − i=1 (xi − θ1 ) lnL (θ1 , θ2 ) = − ln (2πθ2 ) − . 2 2θ2 The partial derivatives with respect to θ1 and θ2 are n ∂ (lnL) 1 = (xi − θ1 ) ∂θ1 θ2 i=1 and n ∂ (lnL) −n 1 2 = + 2 (xi − θ1 ) . ∂θ2 2θ2 2θ2 i=1 ∂(lnL) ∂(lnL) The equation ∂θ1 =0 has the solution θ 1 = x. Setting ∂θ2 =0 and replacing θ1 by x yields n 2 1 θ2 = (xi − x) . n i=1 By considering the usual condition on the second partial derivatives, these solutions do provide a maxi- mum. Thus the maximum likelihood estimators µ = θ1 and σ 2 = θ2 are ^ θ1 = X and n 2 ^ 1 θ2 = Xi − X . n i=1 Where we compare the above example with the introductory one, we see that the method of moments estimators and the maximum likelihood estimators for µ and σ 2 are the same. But this is not always the case. If they are not the same, which is better? Due to the fact that the maximum likelihood estimator of θ has an approximate normal distribution with mean θ and a variance that is equal to a certain lower bound, thus at least approximately, it is unbiased minimum variance estimator. Accordingly, most statisticians prefer the maximum likelihood estimators than estimators found using the method of moments. http://cnx.org/content/m13500/1.3/ Connexions module: m13500 4 1.4 BINOMIAL DISTRIBUTION Observations:k successes in n Bernoulli trials. n! n−x f (x) = px (1 − p) x! (n − x)! n n n n! n−xi n! n− n xi P L (p) = f (xi ) = pxi (1 − p) = pxi (1 − p) i=1 i=1 i=1 xi ! (n − xi )! x ! (n − xi )! i=1 i n n lnL (p) = xi lnp + n− xi ln (1 − p) i=1 i=1 n n dlnL (p) 1 1 = xi − n− xi =0 dp p i=1 i=1 1−p ^ n n ^ 1− p i=1 xi − (n − i=1 xi ) p =0 ^ ^ p 1− p n n n ^ ^ ^ xi − p xi − n p + xi p = 0 i=1 i=1 i=1 n ^ i=1 xi k p= = n n 1.5 POISSON DISTRIBUTION Observations: x1 , x2 , ..., xn , λx e−λ f (x) = , x = 0, 1, 2, ... x! n n λxi e−λ λ i=1 xi L (λ) = = e−λn n i=1 xi ! i=1 xi n n lnL (λ) = −λn + xi lnλ − ln xi i=1 i=1 n dl 1 = −n + xi dλ i=1 λ n 1 −n + xi =0 i=1 λ n ^ i=1 xi λ= n http://cnx.org/content/m13500/1.3/

DOCUMENT INFO

Shared By:

Categories:

Stats:

views: | 12 |

posted: | 9/22/2012 |

language: | Latin |

pages: | 4 |

OTHER DOCS BY tmkpe18

How are you planning on using Docstoc?
BUSINESS
PERSONAL

By registering with docstoc.com you agree to our
privacy policy and
terms of service, and to receive content and offer notifications.

Docstoc is the premier online destination to start and grow small businesses. It hosts the best quality and widest selection of professional documents (over 20 million) and resources including expert videos, articles and productivity tools to make every small business better.

Search or Browse for any specific document or resource you need for your business. Or explore our curated resources for Starting a Business, Growing a Business or for Professional Development.

Feel free to Contact Us with any questions you might have.