7 Steps in Hypothesis Testing by ubb16013

VIEWS: 221 PAGES: 11

									              QM1 – Week 5
        Statistical Significance (II)

                        Dr Alexander Moradi
      University of Oxford, Dept. of Economics & GPRG/CSAE
           Email: alexander.moradi@economics.ox.ac.uk


   7 Steps in Hypothesis Testing
1. Specify the hypothesis in an appropriate form
   for statistical testing
2. Set a level of probability on the basis of which
   the hypothesis should be rejected
3. Select the relevant test statistic
4. Calculate the relevant test statistic and compare
   it with the critical value from a theoretical
   probability distribution for all possible
   outcomes. Reach a decision: to reject or not to
   reject the hypothesis
5. Interpret the results of the decision

7.1 Step 1: Null and Alternative Hypothesis
  • The first step is set up a null hypothesis (H0) that
    can be rejected
  • The null hypothesis is presumed to be true until
    the data strongly suggests otherwise (like a
    defendant on trial)
  • The alternative hypothesis H1 specifies the
  • Examples:
     – H0: The defendant is innocent
       H1: The defendant is guilty
     – H0: Unemployment rates had no impact on Nazi votes
       H1: Unemployment had an impact on Nazi votes
     – H0: bUnemployment=0
       H1: bUnemployment≠0                                     3

  7.2 Step 2: Type I and Type II error
• In hypothesis testing we can make two kinds of
   – Type I error: Rejecting the null hypothesis when it is in
     fact true
   – Type II error: Failing to reject the null hypothesis when
     it is actually false
                                Statistical decision
   True state of nature    Reject H0        Do not reject H0
       H0 is true         Type I error          Correct
       H0 is false          Correct          Type II error

• What is the risk we are willing to take of making a
  Type I error?                                                4

    7.2 Step 2: Significance Level
 • Significance level, α, is the probability of making a
   Type I error
 • ⇒ A small probability of a type I error is preferred
 • To what extent are we willing to take a risk of
   making wrong conclusion?
 • Common choices for α are
    – 10%
    – 5% (most common)
    – 1%
 • 5% level means that we are taking a risk of being
   wrong five times per 100 trials
 • Trade-off: If we reduce the probability of a type I
   error, the probability for a type II error will increase

 7.3 Step 3: Test Statistics and Critical
• What is the probability, that H0 is true given the
  observed outcome?
• For every sample statistic there is a
  corresponding sampling distribution
• Test statistics and critical values in order to test
  a null hypothesis against an alternative


7.3 Test Statistic for Regression Coefficients
• The error term (residuals) is a random variable
• If we could repeat the “social” experiment, we would obtain
  different values for the error term and consequently for the
  dependent variable ⇒ impact on the slope coefficient
• How reliable is it to conclude that there is a relationship?
• Example:
         y                                    2nd sample
                                              1st sample


7.3 Test Statistics for a Regression Coefficient
 • Is b significantly different from 0?
 • H0: b=0
 • ⇒ t-statistics
                    tb =
                           σ e /σ x
• The nominator b is the regression coefficient
  derived from the OLS method
• The denominator corresponds to the estimated
  standard deviation of the sampling distribution of
  the coefficient ⇒ SE(b)                                  8

  7.3 Step 4: Statistical Significance: t-statistics

• The t-statistic indicates how many standard
  deviations the sample regression coefficient is
  from 0 (we can also express it as deviation from
  some certain value of interest if we want)
• Central Limit Theorem applies
   – Shape of the sampling distribution becomes normal
   – With increasing sample size the t-distribution
     approximates a standard normal distribution
• ⇒ If t-value >1.96, the estimated regression
  coefficient is more than 1.96 standard deviations
  from 0 and the probability for such an outcome is
  less than 5%, if H0 is true

7.3 Step 4: Sampling Distribution and Critical Value

                                     95% of cases

2.5% of cases                                 2.5% of cases

            -1.96                       +1.96
     Reject H0          Accept H0          Reject H0

                Confidence level α=0.95

7.4 Step 4: Significance Level and p-Value

• p-value is the probability that the outcome
  observed would be present if the null
  hypothesis is true
• Small p-values are evidence against H0
• p-value<α ⇒ Reject H0 , accept H1
• Failing to reject the null hypothesis does not
  necessarily constitute support for H0; it just
  means that the data or pattern is not
  sufficiently strong to reject H0


  7.5 One- and Two-Tailed Tests
 • Two-tailed test: The parameter value is calculated
   for both tails of the sampling distribution
 • ⇒ The critical region is divided equally between
   the left- and the right-hand tails
 • If the hypothesis is about the directions:
 • One-tailed test: H0: µ1 = µ2; H1: µ1 > µ2
 • Critical region is in the left- or the right-hand tail
 • ⇒ a one-tailed test increases the critical region at
   one tail
 • Strong a priori reasons must exist to justify a one-
   tailed test


7.5 Step 5: Statistical versus Historical Significance
• Statistical significance refers to the probability of type I
  error (rejecting the null hypothesis when it is in fact true)
• Statistical significance is influenced by
   – magnitude of the parameter
   – magnitude of the standard error, i.e. sample size
      (because the standard error decreases with N everything
      else being equal)
• Historical significance: What is the practical significance
  of rejecting the hypothesis?
• Example: H0: bUnemployment=0.55 rejected, estimated
• ⇒ The aim is to find statistically significant results that are
  relevant from a historical perspective

     7 Exercise: Statistical Significance
1.    F&Ts‘ exercises for chapter 6 (p. 181-184): 1, 3, 7


7 Exercise: Statistical Significance


7 Exercise: Statistical Significance


         7 Exercise: Statistical Significance
    Data set: weimar_election.dta
    2. Regress Nazi votes on unemployment rates.
       Interpret the STATA output
    3. Test whether unemployment had a “similar”
       impact on the electoral outcome for the
       communist party (p_kpd)
    4. What is the historical significance of this H0?
    5. Explore the relationship between the
       explanatory variables. What correlations are
       statistically significant?


                 7 STATA commands
               Mean comparison tests (µ is the true population mean)
• ttest varname == #             H0: µvarname = #; H1: µvarname ≠ #
• ttest var1 == var2             H0: µvar1 = µvar2; H1: µvar1 ≠ µvar2
• ttest varname, by(groupvar)    Compares the mean of two groups in varname that are
                                 distinguished by groupvar
                                 When specifying the option “unequal”, the separate
                                 sample variances are used to calculate the SE, i.e. if H0
                                 of equal variances can be rejected

                     Variance comparison test (see p. 172 in F&T)
• sdtest varname1 == varname2    H0: sd(varname1) = sd(varname2)
• sdtest var, by (groupvar)

• test                           tests for simple and composite linear hypotheses about
                                 the parameters of the most recently fitted model (after
                                 regress), e.g. you can test H0 that the regression
                                 coefficient is equal to a certain value

     7 Homework Exercises –Week 5
1.      Read Chapter 6 of F&T

2.       Read the following article on the Great Depression: Eichengreen, B. and J. Sachs
         (1985). Exchange Rates and Economic Recovery in the 1930s. Journal of
         Economic History 45(4): 925-946.
         [Don‘t worry if you do not fully understand the economics or the model on page
         934; the QMs are rather easy, Hint: in this paper, a dummy variable for Germany
         is equivalent to treating Germany as an outlier and excluding this observation]
       a)     Give a short summary (max 250 words)
       b) Interpret Table 3 (page 937)

3.       Use the dataset Depression.dta. A manual can be downloaded from
       a)    Replicate Figure 1 (p. 936), Figure 2 (p. 938), and Figure 3 (p. 939) of E&S
             [Hint: Depression.dta includes more countries & years than the study of
             E&S. Use the ‚filter‘ variable in the Depression.dta file to select the
             countries that were chosen by E&S ⇒ if filter==1]
       b) What is wrong with the title of the horizontal axis in figure 2 (p.938)?
       c)    E&S took „wage data from Mitchell“ (p. 937), whereas Depression.dta used
             a different data source (League of Nations). This makes a difference. Point
             to discrepancies between your scatterplot of REALWAGE and PROD and
             the one of E&S‘                                                            19

     7 Homework Exercises –Week 5
      d)    Replicate E&S‘ table 3, regressions 1 to 4 (p. 937). Use all countries (but
            not years) for which data is available.
            Commands to be used:
           •    regress prod exchange if year==1935 & country!=„USA“
           •    reg prod exchange if year==1935
           •    reg realwage exchange if year==1935
            In regression 3, the base year is 1932 (instead of 1929). Therefore, the
            exchange rate and industrial production must be adjusted, so that they
            represent an index where (1932=100). An index is calculated by

                                 Yt                                            IP1935
     Yt (base year = 100) =                *100        IP1935 (1932 = 100) =          *100
                              YBase year                                       IP1932
            Commands to be used:
           •   tsset country_id year
               [The tsset command tells STATA that the data set is a panel, where
               country_id refers to the cross-sectional component (country) and
               year refers to the time component (year). This allows calculations
               among different cells in a column – because they have an explicit
               order ]

7 Homework Exercises –Week 5
    •    generate prod35_b32=(prod/L3.prod)*100 if year==1935
         [This command generates a new variable named prod35_b32 by
         dividing prod (IP1935) by the value of prod which is found 3
         cells before (IP1932 or L3.prod)]
   • generate exchange35_b32=(exchange/L3.exchange)*100 if
   • reg prod35_b32 exchange35_b32 if country!=“Germany”
e) Compare your results with those of E&S. Interpret the differences.
    Is it reasonable to restrict the analysis to a smaller sample of
    countries? Explain
f) E&S used wholesale prices (instead of retail prices) as price
    deflator for the real wage variable. Which price deflator would you
    prefer? Explain
g) What are the reasons to choose 1935 as the year for comparison
    instead of say 1932 or 1936?



To top