7 Steps in Hypothesis Testing by ubb16013

VIEWS: 221 PAGES: 11

• pg 1
```									              QM1 – Week 5
Statistical Significance (II)

University of Oxford, Dept. of Economics & GPRG/CSAE

1

7 Steps in Hypothesis Testing
1. Specify the hypothesis in an appropriate form
for statistical testing
2. Set a level of probability on the basis of which
the hypothesis should be rejected
3. Select the relevant test statistic
4. Calculate the relevant test statistic and compare
it with the critical value from a theoretical
probability distribution for all possible
outcomes. Reach a decision: to reject or not to
reject the hypothesis
5. Interpret the results of the decision
2

1
7.1 Step 1: Null and Alternative Hypothesis
• The first step is set up a null hypothesis (H0) that
can be rejected
• The null hypothesis is presumed to be true until
the data strongly suggests otherwise (like a
defendant on trial)
• The alternative hypothesis H1 specifies the
opposite
• Examples:
– H0: The defendant is innocent
H1: The defendant is guilty
– H0: bUnemployment=0
H1: bUnemployment≠0                                     3

7.2 Step 2: Type I and Type II error
• In hypothesis testing we can make two kinds of
mistakes
– Type I error: Rejecting the null hypothesis when it is in
fact true
– Type II error: Failing to reject the null hypothesis when
it is actually false
Statistical decision
True state of nature    Reject H0        Do not reject H0
H0 is true         Type I error          Correct
H0 is false          Correct          Type II error

• What is the risk we are willing to take of making a
Type I error?                                                4

2
7.2 Step 2: Significance Level
• Significance level, α, is the probability of making a
Type I error
• ⇒ A small probability of a type I error is preferred
• To what extent are we willing to take a risk of
making wrong conclusion?
• Common choices for α are
– 10%
– 5% (most common)
– 1%
• 5% level means that we are taking a risk of being
wrong five times per 100 trials
• Trade-off: If we reduce the probability of a type I
error, the probability for a type II error will increase
5

7.3 Step 3: Test Statistics and Critical
Values
• What is the probability, that H0 is true given the
observed outcome?
• For every sample statistic there is a
corresponding sampling distribution
• Test statistics and critical values in order to test
a null hypothesis against an alternative

6

3
7.3 Test Statistic for Regression Coefficients
• The error term (residuals) is a random variable
• If we could repeat the “social” experiment, we would obtain
different values for the error term and consequently for the
dependent variable ⇒ impact on the slope coefficient
• How reliable is it to conclude that there is a relationship?
• Example:
y                                    2nd sample
1st sample
b<b

7
x

7.3 Test Statistics for a Regression Coefficient
• Is b significantly different from 0?
• H0: b=0
• ⇒ t-statistics
b
tb =
σ e /σ x
• The nominator b is the regression coefficient
derived from the OLS method
• The denominator corresponds to the estimated
standard deviation of the sampling distribution of
the coefficient ⇒ SE(b)                                  8

4
7.3 Step 4: Statistical Significance: t-statistics

• The t-statistic indicates how many standard
deviations the sample regression coefficient is
from 0 (we can also express it as deviation from
some certain value of interest if we want)
• Central Limit Theorem applies
– Shape of the sampling distribution becomes normal
– With increasing sample size the t-distribution
approximates a standard normal distribution
• ⇒ If t-value >1.96, the estimated regression
coefficient is more than 1.96 standard deviations
from 0 and the probability for such an outcome is
less than 5%, if H0 is true
9

7.3 Step 4: Sampling Distribution and Critical Value

95% of cases

2.5% of cases                                 2.5% of cases

-1.96                       +1.96
Reject H0          Accept H0          Reject H0

Confidence level α=0.95
10

5
7.4 Step 4: Significance Level and p-Value

• p-value is the probability that the outcome
observed would be present if the null
hypothesis is true
• Small p-values are evidence against H0
• p-value<α ⇒ Reject H0 , accept H1
• Failing to reject the null hypothesis does not
necessarily constitute support for H0; it just
means that the data or pattern is not
sufficiently strong to reject H0

11

7.5 One- and Two-Tailed Tests
• Two-tailed test: The parameter value is calculated
for both tails of the sampling distribution
• ⇒ The critical region is divided equally between
the left- and the right-hand tails
• If the hypothesis is about the directions:
• One-tailed test: H0: µ1 = µ2; H1: µ1 > µ2
• Critical region is in the left- or the right-hand tail
• ⇒ a one-tailed test increases the critical region at
one tail
• Strong a priori reasons must exist to justify a one-
tailed test

12

6
7.5 Step 5: Statistical versus Historical Significance
• Statistical significance refers to the probability of type I
error (rejecting the null hypothesis when it is in fact true)
• Statistical significance is influenced by
– magnitude of the parameter
– magnitude of the standard error, i.e. sample size
(because the standard error decreases with N everything
else being equal)
• Historical significance: What is the practical significance
of rejecting the hypothesis?
• Example: H0: bUnemployment=0.55 rejected, estimated
bUnemployment=0.52
• ⇒ The aim is to find statistically significant results that are
relevant from a historical perspective
13

7 Exercise: Statistical Significance
1.    F&Ts‘ exercises for chapter 6 (p. 181-184): 1, 3, 7

14

7
7 Exercise: Statistical Significance

15

7 Exercise: Statistical Significance

16

8
7 Exercise: Statistical Significance
Data set: weimar_election.dta
2. Regress Nazi votes on unemployment rates.
Interpret the STATA output
3. Test whether unemployment had a “similar”
impact on the electoral outcome for the
communist party (p_kpd)
4. What is the historical significance of this H0?
5. Explore the relationship between the
explanatory variables. What correlations are
statistically significant?

17

7 STATA commands
µ
Mean comparison tests (µ is the true population mean)
• ttest varname == #             H0: µvarname = #; H1: µvarname ≠ #
• ttest var1 == var2             H0: µvar1 = µvar2; H1: µvar1 ≠ µvar2
• ttest varname, by(groupvar)    Compares the mean of two groups in varname that are
distinguished by groupvar
When specifying the option “unequal”, the separate
sample variances are used to calculate the SE, i.e. if H0
of equal variances can be rejected

Variance comparison test (see p. 172 in F&T)
• sdtest varname1 == varname2    H0: sd(varname1) = sd(varname2)
• sdtest var, by (groupvar)

• test                           tests for simple and composite linear hypotheses about
the parameters of the most recently fitted model (after
regress), e.g. you can test H0 that the regression
coefficient is equal to a certain value
18

9
7 Homework Exercises –Week 5
1.      Read Chapter 6 of F&T

2.       Read the following article on the Great Depression: Eichengreen, B. and J. Sachs
(1985). Exchange Rates and Economic Recovery in the 1930s. Journal of
Economic History 45(4): 925-946.
[Don‘t worry if you do not fully understand the economics or the model on page
934; the QMs are rather easy, Hint: in this paper, a dummy variable for Germany
is equivalent to treating Germany as an outlier and excluding this observation]
a)     Give a short summary (max 250 words)
b) Interpret Table 3 (page 937)

a)    Replicate Figure 1 (p. 936), Figure 2 (p. 938), and Figure 3 (p. 939) of E&S
[Hint: Depression.dta includes more countries & years than the study of
E&S. Use the ‚filter‘ variable in the Depression.dta file to select the
countries that were chosen by E&S ⇒ if filter==1]
b) What is wrong with the title of the horizontal axis in figure 2 (p.938)?
c)    E&S took „wage data from Mitchell“ (p. 937), whereas Depression.dta used
a different data source (League of Nations). This makes a difference. Point
to discrepancies between your scatterplot of REALWAGE and PROD and
the one of E&S‘                                                            19

7 Homework Exercises –Week 5
d)    Replicate E&S‘ table 3, regressions 1 to 4 (p. 937). Use all countries (but
not years) for which data is available.
Commands to be used:
•    regress prod exchange if year==1935 & country!=„USA“
•    reg prod exchange if year==1935
•    reg realwage exchange if year==1935
In regression 3, the base year is 1932 (instead of 1929). Therefore, the
exchange rate and industrial production must be adjusted, so that they
represent an index where (1932=100). An index is calculated by

Yt                                            IP1935
Yt (base year = 100) =                *100        IP1935 (1932 = 100) =          *100
YBase year                                       IP1932
Commands to be used:
•   tsset country_id year
[The tsset command tells STATA that the data set is a panel, where
country_id refers to the cross-sectional component (country) and
year refers to the time component (year). This allows calculations
among different cells in a column – because they have an explicit
20
order ]

10
7 Homework Exercises –Week 5
•    generate prod35_b32=(prod/L3.prod)*100 if year==1935
[This command generates a new variable named prod35_b32 by
dividing prod (IP1935) by the value of prod which is found 3
cells before (IP1932 or L3.prod)]
• generate exchange35_b32=(exchange/L3.exchange)*100 if
year==1935
• reg prod35_b32 exchange35_b32 if country!=“Germany”
e) Compare your results with those of E&S. Interpret the differences.
Is it reasonable to restrict the analysis to a smaller sample of
countries? Explain
f) E&S used wholesale prices (instead of retail prices) as price
deflator for the real wage variable. Which price deflator would you
prefer? Explain
g) What are the reasons to choose 1935 as the year for comparison
instead of say 1932 or 1936?

21

11

```
To top