# Power

Document Sample

```					Power
The Four Components to a
Statistical Conclusion

Sample size The number of units (e.g.,
people) accessible to study
Effect size The salience of the program
relative to the noise
Alpha level The odds the observed result is
due to chance
Power The odds you’ll observe a
treatment effect when it
occurs
The Four Components to a
Statistical Conclusion

Sample size Amount of information

Effect size Salience of program

Alpha level Willingness to risk

Power Ability to see effect that’s there
The Effect Size
Is a ratio of...
The Effect Size
Is a ratio of...

Signal

Noise
The Effect Size
Is a ratio of...

Signal

Noise

Difference between groups

Standard error of the difference
Given Values for Any Three, You
Can Compute the Fourth.

   n = f(effect size, a, power)
   effect size = f(n, a, power)
   a = f(n, effect size, power)
   power = f(n, effect size, a)
The Decision Matrix
In reality

What
we conclude
The Decision Matrix
In reality       Null true
Alternative false
In reality...
•
What             •
There is no real program effect
There is no difference, gain
•
we conclude           Our theory is wrong
The Decision Matrix
In reality               Null true
Alternative false
In reality...
•
What                          •
There is no real program effect
There is no difference, gain
•
we conclude                        Our theory is wrong

Accept null
Reject alternative
We say...

•   There is no real
program effect
•   There is no difference,
gain
•   Our theory is wrong
The Decision Matrix
In reality               Null true
Alternative false
In reality...
•
What                          •
There is no real program effect
There is no difference, gain
•
we conclude                        Our theory is wrong

Accept null                                      1-
Reject alternative                 THE CONFIDENCE LEVEL
We say...
The odds of saying there is
•   There is no real
no effect or gain when in
program effect                    fact there is none
•   There is no difference,
# of times out of 100 when
gain
there is no effect, we’ll say
•   Our theory is wrong
there is none
The Decision Matrix
In reality              Null true
Alternative false
In reality...
•
What                          •
There is no real program effect
There is no difference, gain
•
we conclude                        Our theory is wrong

Reject null
Accept alternative
We say...
•   There is a real program
effect
•   There is a difference,
gain
•   Our theory is correct
The Decision Matrix
In reality              Null true
Alternative false
In reality...
•
What                          •
There is no real program effect
There is no difference, gain
•
we conclude                        Our theory is wrong

Reject null                                      
Accept alternative                       TYPE I ERROR
We say...                     The odds of saying there is
an effect or gain when in
•   There is a real program
effect
fact there is none
•   There is a difference,           # of times out of 100 when
gain                             there is no effect, we’ll say
•   Our theory is correct                    there is one
The Decision Matrix
Null false
In reality
Alternative true
In reality...
What                     •
•
There is a real program effect
There is a difference, gain
we conclude              •    Our theory is correct
The Decision Matrix
Null false
In reality
Alternative true
In reality...
•
What                               •
There is a real program effect
There is a difference, gain
•
we conclude                             Our theory is correct

Accept null
Reject alternative
We say...

•   There is no real
program effect
•   There is no difference,
gain
•   Our theory is wrong
The Decision Matrix
Null false
In reality
Alternative true
In reality...
•
What                               •
There is a real program effect
There is a difference, gain
•
we conclude                             Our theory is correct

Accept null                                             
Reject alternative                           TYPE II ERROR
We say...
The odds of saying there is
•   There is no real
no effect or gain when in
program effect                          fact there is one
•   There is no difference,
# of times out of 100 when
gain
there is an effect, we’ll say
•   Our theory is wrong
there is none
The Decision Matrix
Null false
In reality
Alternative true
In reality...
•
What                              •
There is a real program effect
There is a difference, gain
•
we conclude                            Our theory is correct

Reject null
Accept alternative
We say...
•   There is a real program
effect
•   There is a difference,
gain
•   Our theory is correct
The Decision Matrix
Null false
In reality
Alternative true
In reality...
•
What                              •
There is a real program effect
There is a difference, gain
•
we conclude                            Our theory is correct

Reject null                                            1-
Accept alternative                              POWER
We say...                         The odds of saying there is
an effect or gain when in
•   There is a real program
effect
fact there is one
•   There is a difference,              # of times out of 100 when
gain                                there is an effect, we’ll say
•   Our theory is correct                       there is one
The Decision Matrix
Null false
In reality               Null true
Alternative false                      Alternative true
In reality...                         In reality...
•                                      •
What                           •
There is no real program effect
There is no difference, gain      •
There is a real program effect
There is a difference, gain
•                                      •
we conclude                         Our theory is wrong                    Our theory is correct

Accept null                                       1-                                      
Reject alternative                  THE CONFIDENCE LEVEL                        TYPE II ERROR
We say...
The odds of saying there is            The odds of saying there is
•    There is no real
no effect or gain when in              no effect or gain when in
program effect                    fact there is none                      fact there is one
•    There is no difference,
# of times out of 100 when             # of times out of 100 when
gain
there is no effect, we’ll say          there is an effect, we’ll say
•    Our theory is wrong
there is none                          there is none

Reject null                                                                               1-
Accept alternative                        TYPE I ERROR                              POWER
We say...                      The odds of saying there is            The odds of saying there is
an effect or gain when in              an effect or gain when in
•    There is a real program            fact there is none                     fact there is one
effect
•    There is a difference,           # of times out of 100 when            # of times out of 100 when
gain                             there is no effect, we’ll say         there is an effect, we’ll say
•    Our theory is correct                    there is one                          there is one
The Decision Matrix
Null false
In reality               Null true
Alternative false                      Alternative true
In reality...                         In reality...
•                                      •
What                           •
There is no real program effect
There is no difference, gain      •
There is a real program effect
There is a difference, gain
•                                      •
we conclude                         Our theory is wrong                    Our theory is correct

Accept null                                      1-                                       
Reject alternative             THE CONFIDENCE LEVEL                           TYPE II ERROR
We say...

•    There is no real
program effect
•    There is no difference,
gain
•    Our theory is wrong

Reject null                                                                               1-
Accept alternative                      TYPE I ERROR                                   POWER
We say...

•    There is a real program
effect
•    There is a difference,
gain
•    Our theory is correct
The Decision Matrix
Null false
In reality               Null true
Alternative false                      Alternative true
In reality...                         In reality...
•                                      •
What                           •
There is no real program effect
There is no difference, gain      •
There is a real program effect
There is a difference, gain
•                                      •
we conclude                         Our theory is wrong                    Our theory is correct

Accept null                                      1-                                       
Reject alternative             THE CONFIDENCE LEVEL                           TYPE II ERROR
We say...
CORRECT
•    There is no real
program effect
•    There is no difference,
gain
•    Our theory is wrong

Reject null                                                                               1-
Accept alternative                      TYPE I ERROR                                   POWER
We say...

•    There is a real program                                                        CORRECT
effect
•    There is a difference,
gain
•    Our theory is correct
The Decision Matrix
Null false
In reality               Null true
Alternative false                      Alternative true
In reality...                         In reality...
•                                      •
What                           •
There is no real program effect
There is no difference, gain      •
There is a real program effect
There is a difference, gain
•                                      •
we conclude                         Our theory is wrong                    Our theory is correct

Accept null                                       1-                                      
Reject alternative                  THE CONFIDENCE LEVEL                        TYPE II ERROR
We say...
The odds of saying there is            The odds of saying there is
•          If you try to increase power, you increase
There is no real
no effect or gain when in              no effect or gain when in
program effect                    fact there is none                      fact there is one
•                              winding up
the chance of times out of 100 when in the bottom when
There is no difference,
# of                              # of times out of 100
gain
•
there is no effect, we’ll say error.
row and of none I
Our theory is wrong
there is
Type              there is an effect, we’ll say
there is none

Reject null                                                                               1-
Accept alternative                        TYPE I ERROR                              POWER
We say...                      The odds of saying there is            The odds of saying there is
an effect or gain when in              an effect or gain when in
•    There is a real program            fact there is none                     fact there is one
effect
•    There is a difference,           # of times out of 100 when            # of times out of 100 when
gain                             there is no effect, we’ll say         there is an effect, we’ll say
•    Our theory is correct                    there is one                          there is one
The Decision Matrix
Null false
In reality           Null true
Alternative false                      Alternative true
In reality...
If you try to         • There is no real program effect            •
In reality...
What                                                                     There is a real program effect
decrease Type I •• There is no is wrong gain
difference,                   •
•
There is a difference, gain
we conclude               Our theory                                     Our theory is correct
errors, you
Accept null                             1-                                              
increase the
Reject alternative       THE CONFIDENCE LEVEL                                 TYPE II ERROR
chance of winding The odds of saying there is
We say...
The odds of saying there is
There is top
up •in theno real row no effect or gain when in                            no effect or gain when in
program effect            fact there is none                             fact there is one
• There is Type II
and of no difference,      # of times out of 100 when                      # of times out of 100 when
gain
error.
• Our theory is wrong
there is no effect, we’ll say
there is none
there is an effect, we’ll say
there is none

Reject null                                                                            1-
Accept alternative                     TYPE I ERROR                              POWER
We say...                  The odds of saying there is            The odds of saying there is
an effect or gain when in              an effect or gain when in
•    There is a real program        fact there is none                     fact there is one
effect
•    There is a difference,       # of times out of 100 when            # of times out of 100 when
gain                         there is no effect, we’ll say         there is an effect, we’ll say
•    Our theory is correct                there is one                          there is one

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 1 posted: 9/1/2012 language: English pages: 23