VIEWS: 221 PAGES: 11 CATEGORY: Other POSTED ON: 5/30/2010 Public Domain
QM1 – Week 5 Statistical Significance (II) Dr Alexander Moradi University of Oxford, Dept. of Economics & GPRG/CSAE Email: alexander.moradi@economics.ox.ac.uk 1 7 Steps in Hypothesis Testing 1. Specify the hypothesis in an appropriate form for statistical testing 2. Set a level of probability on the basis of which the hypothesis should be rejected 3. Select the relevant test statistic 4. Calculate the relevant test statistic and compare it with the critical value from a theoretical probability distribution for all possible outcomes. Reach a decision: to reject or not to reject the hypothesis 5. Interpret the results of the decision 2 1 7.1 Step 1: Null and Alternative Hypothesis • The first step is set up a null hypothesis (H0) that can be rejected • The null hypothesis is presumed to be true until the data strongly suggests otherwise (like a defendant on trial) • The alternative hypothesis H1 specifies the opposite • Examples: – H0: The defendant is innocent H1: The defendant is guilty – H0: Unemployment rates had no impact on Nazi votes H1: Unemployment had an impact on Nazi votes – H0: bUnemployment=0 H1: bUnemployment≠0 3 7.2 Step 2: Type I and Type II error • In hypothesis testing we can make two kinds of mistakes – Type I error: Rejecting the null hypothesis when it is in fact true – Type II error: Failing to reject the null hypothesis when it is actually false Statistical decision True state of nature Reject H0 Do not reject H0 H0 is true Type I error Correct H0 is false Correct Type II error • What is the risk we are willing to take of making a Type I error? 4 2 7.2 Step 2: Significance Level • Significance level, α, is the probability of making a Type I error • ⇒ A small probability of a type I error is preferred • To what extent are we willing to take a risk of making wrong conclusion? • Common choices for α are – 10% – 5% (most common) – 1% • 5% level means that we are taking a risk of being wrong five times per 100 trials • Trade-off: If we reduce the probability of a type I error, the probability for a type II error will increase 5 7.3 Step 3: Test Statistics and Critical Values • What is the probability, that H0 is true given the observed outcome? • For every sample statistic there is a corresponding sampling distribution • Test statistics and critical values in order to test a null hypothesis against an alternative 6 3 7.3 Test Statistic for Regression Coefficients • The error term (residuals) is a random variable • If we could repeat the “social” experiment, we would obtain different values for the error term and consequently for the dependent variable ⇒ impact on the slope coefficient • How reliable is it to conclude that there is a relationship? • Example: y 2nd sample 1st sample b<b 7 x 7.3 Test Statistics for a Regression Coefficient • Is b significantly different from 0? • H0: b=0 • ⇒ t-statistics b tb = σ e /σ x • The nominator b is the regression coefficient derived from the OLS method • The denominator corresponds to the estimated standard deviation of the sampling distribution of the coefficient ⇒ SE(b) 8 4 7.3 Step 4: Statistical Significance: t-statistics • The t-statistic indicates how many standard deviations the sample regression coefficient is from 0 (we can also express it as deviation from some certain value of interest if we want) • Central Limit Theorem applies – Shape of the sampling distribution becomes normal – With increasing sample size the t-distribution approximates a standard normal distribution • ⇒ If t-value >1.96, the estimated regression coefficient is more than 1.96 standard deviations from 0 and the probability for such an outcome is less than 5%, if H0 is true 9 7.3 Step 4: Sampling Distribution and Critical Value 95% of cases 2.5% of cases 2.5% of cases -1.96 +1.96 Reject H0 Accept H0 Reject H0 Confidence level α=0.95 10 5 7.4 Step 4: Significance Level and p-Value • p-value is the probability that the outcome observed would be present if the null hypothesis is true • Small p-values are evidence against H0 • p-value<α ⇒ Reject H0 , accept H1 • Failing to reject the null hypothesis does not necessarily constitute support for H0; it just means that the data or pattern is not sufficiently strong to reject H0 11 7.5 One- and Two-Tailed Tests • Two-tailed test: The parameter value is calculated for both tails of the sampling distribution • ⇒ The critical region is divided equally between the left- and the right-hand tails • If the hypothesis is about the directions: • One-tailed test: H0: µ1 = µ2; H1: µ1 > µ2 • Critical region is in the left- or the right-hand tail • ⇒ a one-tailed test increases the critical region at one tail • Strong a priori reasons must exist to justify a one- tailed test 12 6 7.5 Step 5: Statistical versus Historical Significance • Statistical significance refers to the probability of type I error (rejecting the null hypothesis when it is in fact true) • Statistical significance is influenced by – magnitude of the parameter – magnitude of the standard error, i.e. sample size (because the standard error decreases with N everything else being equal) • Historical significance: What is the practical significance of rejecting the hypothesis? • Example: H0: bUnemployment=0.55 rejected, estimated bUnemployment=0.52 • ⇒ The aim is to find statistically significant results that are relevant from a historical perspective 13 7 Exercise: Statistical Significance 1. F&Ts‘ exercises for chapter 6 (p. 181-184): 1, 3, 7 14 7 7 Exercise: Statistical Significance 15 7 Exercise: Statistical Significance 16 8 7 Exercise: Statistical Significance Data set: weimar_election.dta 2. Regress Nazi votes on unemployment rates. Interpret the STATA output 3. Test whether unemployment had a “similar” impact on the electoral outcome for the communist party (p_kpd) 4. What is the historical significance of this H0? 5. Explore the relationship between the explanatory variables. What correlations are statistically significant? 17 7 STATA commands µ Mean comparison tests (µ is the true population mean) • ttest varname == # H0: µvarname = #; H1: µvarname ≠ # • ttest var1 == var2 H0: µvar1 = µvar2; H1: µvar1 ≠ µvar2 • ttest varname, by(groupvar) Compares the mean of two groups in varname that are distinguished by groupvar When specifying the option “unequal”, the separate sample variances are used to calculate the SE, i.e. if H0 of equal variances can be rejected Variance comparison test (see p. 172 in F&T) • sdtest varname1 == varname2 H0: sd(varname1) = sd(varname2) • sdtest var, by (groupvar) • test tests for simple and composite linear hypotheses about the parameters of the most recently fitted model (after regress), e.g. you can test H0 that the regression coefficient is equal to a certain value 18 9 7 Homework Exercises –Week 5 1. Read Chapter 6 of F&T 2. Read the following article on the Great Depression: Eichengreen, B. and J. Sachs (1985). Exchange Rates and Economic Recovery in the 1930s. Journal of Economic History 45(4): 925-946. [Don‘t worry if you do not fully understand the economics or the model on page 934; the QMs are rather easy, Hint: in this paper, a dummy variable for Germany is equivalent to treating Germany as an outlier and excluding this observation] a) Give a short summary (max 250 words) b) Interpret Table 3 (page 937) 3. Use the dataset Depression.dta. A manual can be downloaded from http://www.economics.ox.ac.uk/Members/alexander.moradi/teaching.html. a) Replicate Figure 1 (p. 936), Figure 2 (p. 938), and Figure 3 (p. 939) of E&S [Hint: Depression.dta includes more countries & years than the study of E&S. Use the ‚filter‘ variable in the Depression.dta file to select the countries that were chosen by E&S ⇒ if filter==1] b) What is wrong with the title of the horizontal axis in figure 2 (p.938)? c) E&S took „wage data from Mitchell“ (p. 937), whereas Depression.dta used a different data source (League of Nations). This makes a difference. Point to discrepancies between your scatterplot of REALWAGE and PROD and the one of E&S‘ 19 7 Homework Exercises –Week 5 d) Replicate E&S‘ table 3, regressions 1 to 4 (p. 937). Use all countries (but not years) for which data is available. Commands to be used: • regress prod exchange if year==1935 & country!=„USA“ • reg prod exchange if year==1935 • reg realwage exchange if year==1935 In regression 3, the base year is 1932 (instead of 1929). Therefore, the exchange rate and industrial production must be adjusted, so that they represent an index where (1932=100). An index is calculated by Yt IP1935 Yt (base year = 100) = *100 IP1935 (1932 = 100) = *100 YBase year IP1932 Commands to be used: • tsset country_id year [The tsset command tells STATA that the data set is a panel, where country_id refers to the cross-sectional component (country) and year refers to the time component (year). This allows calculations among different cells in a column – because they have an explicit 20 order ] 10 7 Homework Exercises –Week 5 • generate prod35_b32=(prod/L3.prod)*100 if year==1935 [This command generates a new variable named prod35_b32 by dividing prod (IP1935) by the value of prod which is found 3 cells before (IP1932 or L3.prod)] • generate exchange35_b32=(exchange/L3.exchange)*100 if year==1935 • reg prod35_b32 exchange35_b32 if country!=“Germany” e) Compare your results with those of E&S. Interpret the differences. Is it reasonable to restrict the analysis to a smaller sample of countries? Explain f) E&S used wholesale prices (instead of retail prices) as price deflator for the real wage variable. Which price deflator would you prefer? Explain g) What are the reasons to choose 1935 as the year for comparison instead of say 1932 or 1936? 21 11