# Lecture 10 Semester 2 2007 Standard Error of the Mean and

Document Sample

```					    Lecture 10 Semester 2
2007
Standard Error of the Mean and
Hypothesis Testing

David Morrison
203
Today’sTopics
• Sampling from a population and defining
oddness to infer reliable differences
• Hypothesis testing – the process of using
statistics for support
Sampling from a Population
• When we sample from a population and measure the
characteristics of that sample we are usually trying to
make an assessment that applies to the population.
• Like measurement error in individual assessment we
also have sampling error when it comes to group
assessment.
• No two samples are ever likely to be the same
• The funadmental question is –
– when we compare samples, are the observed differences
between samples what we should expect by chance or are
they real differences due to …
– some intervention (or experimental manipulation) to which
we have exposed the sample.
Generating Sample Means
• In order to know what we should expect by
chance we have to generate a distribution
of sample means
Differences between samples

– Samples only provide estimates (information)
about the population from whence they came (e.g.
Means and standard deviations).
– Often samples we take will be similar but
sometimes they are very different even if the
samples come from the same population.
– Our question “Do the samples come from the
same or different populations?” is hard to answer
because …
– the expected difference between samples is
linked to the size of the sample
– Lets look at distributions of means generated
from samples of different sizes
What the previous slides show
• There is more variability when the sample size is small
• Larger differences between means are observed with
small samples
• So, simply looking at mean differences does not give us
the information we need to draw inferences about the
presence or absence of an experimental effect
• We need to know what the distribution of sample means
looks like and therefore what the expected frequency is
of those mean differences amongst the set of possible
mean differences generated from repeated sampling
• The distribution of sample means and the expected size
of mean differences happens to vary depending on the
size of the sample.
The distribution of sample means for random samples of size
(a) n = 1, (b) n = 4, and (c) n = 100 obtained from a normal
population with µ = 80 and σ = 20.
How likely are observed differences in sample
means across groups likely to have occurred
by chance?
• We don’t usually try and answer this directly but rely on
statistical tests to answer the question “do these groups
differ”
• Inferential Statistical Tests tell us how likely it is that an
observed set of data departs from expectation.
• The tests examine how likely an observed difference
between sample means is had the data all been drawn
from the same underlying population
• This is a test of the null hypothesis that the samples
were taken from the same population.
• We can proxy a statistical test by looking at what are
referred to as confidence intervals for the samples we
collect
Confidence Intervals
• A confidence interval is calculated from
the data collected for an individual sample
• It uses the Standard Deviation of the
sampling distribution of the means and it
tells us, for any given sample, where the
true population mean is likely to lie.
Calculating the Standard Error of
the Mean
• The formula for standard deviation of the
sample means is:
SEM= sd/n
• SEM= Standard Error of the Mean
• Not to be confused with SEM Standard error
of Measurement
• SEM is a standard deviation like any other
standard deviation which means we can
make statements about where a score lies in
a distribution of possible scores.
What is the SEM
• We know then that if we take a random sample
that we have a 68% chance that the sample
mean will be less than or equal to one SEM from
the true population mean.
• Lets have a look at the normal curve again
• The sample mean plus or minus the value of the
SEM gives us the 68% confidence interval
• The sample mean plus or minus 1.96 * SEM
gives us the 95% confidence interval
Some more about the Standard Error of
the Mean
• The SEM IS affected by sample size as the formula
suggests and as we saw in an earlier slide
• As the sample gets larger our estimate of the true
mean gets better and we can be more confident
about where the true mean lies.
• The practical consequence of this is that even small
differences can appear “statistically odd” denoting a
probable experimental effect rather than sampling
error
• Lets have a look at the confidence intervals for
samples of different sizes
Sampling Distribution of the Mean (SEM)
Summary
• Repeated sampling from a population produces a distribution
of means which is bell shaped.
• The properties of the distribution are that most samples yield
means close to the true population mean but give us a normal
distribution – irrespective of the shape of the original
distribution (Central Limit Theorem).
• The standard deviation of the sample means is known as the
standard error of the mean (SEM)
• Normal distributions have certain properties that are useful and
we can use this to define confidence intervals.
• We can specify a criterion defining “oddness” where be
become convinced the data are more likely to have come from
different distributions.
• When we infer this we say that the difference between means is
statistically significant.
95% Confidence Intervals when n=10 note
how the Dark Green Sample is “odd”
95% confidence intervals when the sample is larger
Apparently no odd samples here
Hypothesis Testing
• Is intended to help researchers differentiate real
and random patterns
• In discussing hypothesis testing you will realise it
is a logical process that uses
• Standard (z) scores
• Probability
• Standard Error
• If you understand these concepts this topic is
easy!
The Logic of Hypothesis Testing
• Hypothesis testing is a procedure that uses sample data
to evaluate an hypothesis about a population parameter
(e.g., mean, Std Deviation)
• We start from the position that the null hypothesis is True
– that is there is no difference between the sample and
the population against which we are comparing it.
• In the simplest case we perform what is known as the z
test
• Different tests are used for different comparisons. For
example we apply a different test when we compare two
sample means (t-test) and a different test again when we
compare multiple sample means (analysis of variance)
(More of these later)
Steps in HypothesisTesting for the
z test - An EXAMPLE
• State the hypothesis – e.g. Living close to a nuclear
power plant affects the rate of cancer experienced by
the population
• Before selecting the sample we use the hypothesis to
help identify what the sample should be.
• If we don’t know the incidence of cancer in the wider
population we need to identify the cancer base rate
or find some other indication of likelihood of
developing cancer in the population
• In other words we need a comparison population
• We use the z test when we know the population
parameters (mean & SD)
Steps in HypothesisTesting II
• For this example we obtain a random sample of
individuals living close to nuclear power plants (not just
one)
• Compare the obtained sample data with the prediction
that was made from the hypothesis
• The question we ask is “Does the incidence of cancer
depart from our expectations given what we know about
the cancer rate in the wider community?”
• A problem with this particular example is that we don’t have the
ability to randomly assign people to living close to a nucelar power
plant - there are lots of things we didn’t control that might lead to
alternative explanations of the results (e.g. social class, exposure to
other toxins etc.)
Exposure

Exposed
Population

σ=4
We rely on the
Exposed sample to
tell us what the
Exposed population
will look like with                   Exposed
Sample
respect to the                          n=16
Dep.Var.
The hypothesis test is concerned with the unknown population that
would exist if the treatment (living close to a nuclear power plant)
were administered to the entire population.
Summary
• State the hypothesis about the unknown population
– The null hypothesis (H0) – Being exposed to background
radiation equivalent to that found close to a power station
has no effect on X (the measure of cancer).
– The alternative hypothesis (H1) - Being exposed to
background radiation equivalent to that found close to a
power station WILL have an affect on X (the measure of
cancer).
• Note at this stage we have not said whether X will go
up or down (known as a 2 – tailed test).
– We need to distinguish between directional and non-
directional hypotheses
– A directional test is slightly different from a non-directional
test in that it uses only one half of the normal distribution of
scores.
What will convince us that the null
hypothesis is false?
• Rejection of the null hypothesis in favour of the
alternative (H1) is a purely statistical decision
and it is ARBITRARY.
• By convention we reject the null hypothesis
when the probability of it being true is less than 1
in 20 or p<.05
• The decision criterion is to accept H1 when the
sample data violate our expectation at a given
pre-defined level when p<.05
• Let’s have a look at the normal distribution again
The Decision Criterion
Then if a mean
from a new
sample falls at
If this was the                 the extremities
distribution of values            we say we have a
from a known                      statistically
population with                 significant effect
sample Means X                     Where do we
and SD Y                   place the decision
criterion?
The Decision Criterion

•Alpha Level –
Alpha Level            (Level of
significance) is a
probability value
used to define
very unlikely
outcomes if the
Null Hypothesis is
true. By
convention alpha
is set where p<.05
The Decision Criterion

•Critical Region –
Boundaries
determined by
Critical Region           the alpha level. If
sample data fall
in the critical
region the Ho is
rejected in favour
of H1
For the Z test
• Boundaries for the critical region are defined by the Z
score location. With α≤.05 the boundaries separate the
extreme 5% from the middle 95%
• As the extreme is split across 2 tails of the distribution Z=
± 1.96
• For α=.01 Z=± 2.58; for α=.001 Z=± 3.30
Z=-1.64 α=.05 for a one       Z=1.64 α=.05 for a one
tailed test                   tailed test

For a directional
prediction we use a one
tailed test.
The decision criterion
for the one tailed test is
less extreme as we are
using the top or bottom
5% of data points rather
than 2.5% at either end
of the distribution

The locations of the critical region boundaries for three
different levels of significance:  = .05,  = .01, and  = .001.
Step 3: Collect data and compute
sample statistics
• Obtain random sample and measure sample on
key dependent variable
• Summarise the data (mean)
• Compare the sample mean (say its 29 derived
from our sample of 16 people living close to the
power plant) with the null hypothesis
• (mean(μ) = 26 Std Deviation (σ)=4)
Compare Mean to Null Hypothesis

Compute z-score of
M 
where sample mean
z

is located relative to
the hypothesised
population mean
M


As σ and μ are known
population paramaters
   M

n
σM= SEM is defined by
Make a Decision
M               Then first we need to
z
If           M
calculate the Std
Error for the given
sample size

M n
                              4        This exceeds
gives          M 16
          the critical
value and we
infer living
29  26                 next to power
z                  =3         plants causes
1                     cancer
Area under the standard normal curve.
Uncertainty and Error in Hypothesis
Testing
• There is always a chance that the selected
sample can mislead us (however small)
• A type 1 error occurs when Ho is rejected
when it is true.
• The alpha level (α=.05) is the probability of
making a type 1 error.
• A type II error occurs when we fail to reject
Ho when it is really false. (β)
Assumptions of the z test
• Random Sampling – important with respect to equality
and representativeness.
• Independent Observations in the sample – if this is
violated the standard error is under estimated and the
type 1 error rate may be a lot larger than you think.
• σ is unchanged by the treatment. We assume the
treatment adds a constant. It is perfectly possible that
there is a person by treatment effect.
• Normal Sampling Distribution – to evaluate hypotheses
with z scores we use a unit normal table to identify the
critical region.
FINISH
A sample is selected, then the sample mean is
computed and placed in a frequency distribution.
This process is repeated over and over until all the
possible random samples are obtained and the
complete set of sample means is in the distribution.
Sampling Distribution of Means from 20 samples
of ten subjects
Sampling distribution of means from 100 samples of 10
subjects
Sampling Distribution of Means from 500 Samples of
10
Sampling Distribution of Means from 1000 samples
Sampling distribution of means from an infinite
number of samples
An example of a normal distribution. Each of the small
boxes represents a single data point obtained for one
sample.

The mean from                              Is different from
this sample                               this one but the
data points all
come from the
same population
Example where the estimated sample mean does not
always overlap with the population mean

Confidence Interval

Confidence Interval
The distribution of sample means for random samples of size
(a) n = 1, (b) n = 4, and (c) n = 100 obtained from a normal
population with µ = 80 and σ = 20.
Mean Estimate N=10

A lot of variability in
sample means due to
the small sample size

The true population
mean
Mean Estimate: N=100

Variability of sample
means is a lot less
when the sample size
increases
Sampling distribution of the mean when sample size =5
Sampling distribution of the mean when sample size=10
Sampling distribution of the mean when sample size=100
80% Confidence Interval
90% Confidence Interval
95% Confidence Interval
Area under the standard normal curve.

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 10 posted: 3/13/2012 language: pages: 53