Docstoc

Power

Document Sample
Power Powered By Docstoc
					Power
        The Four Components to a
          Statistical Conclusion

Sample size The number of units (e.g.,
               people) accessible to study
  Effect size The salience of the program
               relative to the noise
 Alpha level The odds the observed result is
               due to chance
      Power The odds you’ll observe a
               treatment effect when it
               occurs
        The Four Components to a
          Statistical Conclusion

Sample size Amount of information

 Effect size Salience of program

Alpha level Willingness to risk

     Power Ability to see effect that’s there
              The Effect Size
Is a ratio of...
              The Effect Size
Is a ratio of...


                    Signal

                    Noise
              The Effect Size
Is a ratio of...


                      Signal

                       Noise




             Difference between groups

          Standard error of the difference
    Given Values for Any Three, You
       Can Compute the Fourth.

   n = f(effect size, a, power)
   effect size = f(n, a, power)
   a = f(n, effect size, power)
   power = f(n, effect size, a)
         The Decision Matrix
    In reality

What
we conclude
         The Decision Matrix
    In reality       Null true
                     Alternative false
                       In reality...
                 •
What             •
                      There is no real program effect
                      There is no difference, gain
                 •
we conclude           Our theory is wrong
                   The Decision Matrix
         In reality               Null true
                                  Alternative false
                                    In reality...
                              •
What                          •
                                   There is no real program effect
                                   There is no difference, gain
                              •
we conclude                        Our theory is wrong

Accept null
Reject alternative
 We say...

•   There is no real
    program effect
•   There is no difference,
    gain
•   Our theory is wrong
                   The Decision Matrix
         In reality               Null true
                                  Alternative false
                                    In reality...
                              •
What                          •
                                   There is no real program effect
                                   There is no difference, gain
                              •
we conclude                        Our theory is wrong

Accept null                                      1-
Reject alternative                 THE CONFIDENCE LEVEL
 We say...
                                  The odds of saying there is
•   There is no real
                                   no effect or gain when in
    program effect                    fact there is none
•   There is no difference,
                                    # of times out of 100 when
    gain
                                    there is no effect, we’ll say
•   Our theory is wrong
                                            there is none
                  The Decision Matrix
          In reality              Null true
                                  Alternative false
                                    In reality...
                              •
What                          •
                                   There is no real program effect
                                   There is no difference, gain
                              •
we conclude                        Our theory is wrong




Reject null
Accept alternative
    We say...
•   There is a real program
    effect
•   There is a difference,
    gain
•   Our theory is correct
                  The Decision Matrix
          In reality              Null true
                                  Alternative false
                                    In reality...
                              •
What                          •
                                   There is no real program effect
                                   There is no difference, gain
                              •
we conclude                        Our theory is wrong




Reject null                                      
Accept alternative                       TYPE I ERROR
    We say...                     The odds of saying there is
                                   an effect or gain when in
•   There is a real program
    effect
                                       fact there is none
•   There is a difference,           # of times out of 100 when
    gain                             there is no effect, we’ll say
•   Our theory is correct                    there is one
         The Decision Matrix
                             Null false
    In reality
                             Alternative true
                              In reality...
What                     •
                         •
                              There is a real program effect
                              There is a difference, gain
we conclude              •    Our theory is correct
                   The Decision Matrix
                                       Null false
         In reality
                                       Alternative true
                                        In reality...
                                   •
What                               •
                                        There is a real program effect
                                        There is a difference, gain
                                   •
we conclude                             Our theory is correct

Accept null
Reject alternative
 We say...

•   There is no real
    program effect
•   There is no difference,
    gain
•   Our theory is wrong
                   The Decision Matrix
                                       Null false
         In reality
                                       Alternative true
                                        In reality...
                                   •
What                               •
                                        There is a real program effect
                                        There is a difference, gain
                                   •
we conclude                             Our theory is correct

Accept null                                             
Reject alternative                           TYPE II ERROR
 We say...
                                       The odds of saying there is
•   There is no real
                                        no effect or gain when in
    program effect                          fact there is one
•   There is no difference,
                                         # of times out of 100 when
    gain
                                         there is an effect, we’ll say
•   Our theory is wrong
                                                 there is none
                  The Decision Matrix
                                      Null false
          In reality
                                      Alternative true
                                       In reality...
                                  •
What                              •
                                       There is a real program effect
                                       There is a difference, gain
                                  •
we conclude                            Our theory is correct




Reject null
Accept alternative
    We say...
•   There is a real program
    effect
•   There is a difference,
    gain
•   Our theory is correct
                  The Decision Matrix
                                      Null false
          In reality
                                      Alternative true
                                       In reality...
                                  •
What                              •
                                       There is a real program effect
                                       There is a difference, gain
                                  •
we conclude                            Our theory is correct




Reject null                                            1-
Accept alternative                              POWER
    We say...                         The odds of saying there is
                                       an effect or gain when in
•   There is a real program
    effect
                                           fact there is one
•   There is a difference,              # of times out of 100 when
    gain                                there is an effect, we’ll say
•   Our theory is correct                       there is one
                    The Decision Matrix
                                                                          Null false
          In reality               Null true
                                   Alternative false                      Alternative true
                                     In reality...                         In reality...
                               •                                      •
What                           •
                                    There is no real program effect
                                    There is no difference, gain      •
                                                                           There is a real program effect
                                                                           There is a difference, gain
                               •                                      •
we conclude                         Our theory is wrong                    Our theory is correct

Accept null                                       1-                                      
Reject alternative                  THE CONFIDENCE LEVEL                        TYPE II ERROR
 We say...
                                   The odds of saying there is            The odds of saying there is
•    There is no real
                                    no effect or gain when in              no effect or gain when in
     program effect                    fact there is none                      fact there is one
•    There is no difference,
                                     # of times out of 100 when             # of times out of 100 when
     gain
                                     there is no effect, we’ll say          there is an effect, we’ll say
•    Our theory is wrong
                                             there is none                          there is none

Reject null                                                                               1-
Accept alternative                        TYPE I ERROR                              POWER
    We say...                      The odds of saying there is            The odds of saying there is
                                    an effect or gain when in              an effect or gain when in
•    There is a real program            fact there is none                     fact there is one
     effect
•    There is a difference,           # of times out of 100 when            # of times out of 100 when
     gain                             there is no effect, we’ll say         there is an effect, we’ll say
•    Our theory is correct                    there is one                          there is one
                    The Decision Matrix
                                                                          Null false
          In reality               Null true
                                   Alternative false                      Alternative true
                                     In reality...                         In reality...
                               •                                      •
What                           •
                                    There is no real program effect
                                    There is no difference, gain      •
                                                                           There is a real program effect
                                                                           There is a difference, gain
                               •                                      •
we conclude                         Our theory is wrong                    Our theory is correct

Accept null                                      1-                                       
Reject alternative             THE CONFIDENCE LEVEL                           TYPE II ERROR
 We say...

•    There is no real
     program effect
•    There is no difference,
     gain
•    Our theory is wrong

Reject null                                                                               1-
Accept alternative                      TYPE I ERROR                                   POWER
    We say...

•    There is a real program
     effect
•    There is a difference,
     gain
•    Our theory is correct
                    The Decision Matrix
                                                                          Null false
          In reality               Null true
                                   Alternative false                      Alternative true
                                     In reality...                         In reality...
                               •                                      •
What                           •
                                    There is no real program effect
                                    There is no difference, gain      •
                                                                           There is a real program effect
                                                                           There is a difference, gain
                               •                                      •
we conclude                         Our theory is wrong                    Our theory is correct

Accept null                                      1-                                       
Reject alternative             THE CONFIDENCE LEVEL                           TYPE II ERROR
 We say...
                                            CORRECT
•    There is no real
     program effect
•    There is no difference,
     gain
•    Our theory is wrong

Reject null                                                                               1-
Accept alternative                      TYPE I ERROR                                   POWER
    We say...

•    There is a real program                                                        CORRECT
     effect
•    There is a difference,
     gain
•    Our theory is correct
                    The Decision Matrix
                                                                          Null false
          In reality               Null true
                                   Alternative false                      Alternative true
                                     In reality...                         In reality...
                               •                                      •
What                           •
                                    There is no real program effect
                                    There is no difference, gain      •
                                                                           There is a real program effect
                                                                           There is a difference, gain
                               •                                      •
we conclude                         Our theory is wrong                    Our theory is correct

Accept null                                       1-                                      
Reject alternative                  THE CONFIDENCE LEVEL                        TYPE II ERROR
 We say...
                                   The odds of saying there is            The odds of saying there is
•          If you try to increase power, you increase
     There is no real
                                    no effect or gain when in              no effect or gain when in
     program effect                    fact there is none                      fact there is one
•                              winding up
             the chance of times out of 100 when in the bottom when
     There is no difference,
                          # of                              # of times out of 100
     gain
•
                          there is no effect, we’ll say error.
                     row and of none I
     Our theory is wrong
                                 there is
                                          Type              there is an effect, we’ll say
                                                                    there is none

Reject null                                                                               1-
Accept alternative                        TYPE I ERROR                              POWER
    We say...                      The odds of saying there is            The odds of saying there is
                                    an effect or gain when in              an effect or gain when in
•    There is a real program            fact there is none                     fact there is one
     effect
•    There is a difference,           # of times out of 100 when            # of times out of 100 when
     gain                             there is no effect, we’ll say         there is an effect, we’ll say
•    Our theory is correct                    there is one                          there is one
                        The Decision Matrix
                                                                           Null false
               In reality           Null true
                                    Alternative false                      Alternative true
                                      In reality...
    If you try to         • There is no real program effect            •
                                                                            In reality...
   What                                                                     There is a real program effect
 decrease Type I •• There is no is wrong gain
                                         difference,                   •
                                                                       •
                                                                            There is a difference, gain
   we conclude               Our theory                                     Our theory is correct
     errors, you
   Accept null                             1-                                              
   increase the
   Reject alternative       THE CONFIDENCE LEVEL                                 TYPE II ERROR
chance of winding The odds of saying there is
     We say...
                                                                           The odds of saying there is
       There is top
 up •in theno real row no effect or gain when in                            no effect or gain when in
       program effect            fact there is none                             fact there is one
    • There is Type II
  and of no difference,      # of times out of 100 when                      # of times out of 100 when
       gain
          error.
    • Our theory is wrong
                             there is no effect, we’ll say
                                     there is none
                                                                             there is an effect, we’ll say
                                                                                     there is none

    Reject null                                                                            1-
    Accept alternative                     TYPE I ERROR                              POWER
         We say...                  The odds of saying there is            The odds of saying there is
                                     an effect or gain when in              an effect or gain when in
     •    There is a real program        fact there is none                     fact there is one
          effect
     •    There is a difference,       # of times out of 100 when            # of times out of 100 when
          gain                         there is no effect, we’ll say         there is an effect, we’ll say
     •    Our theory is correct                there is one                          there is one

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:1
posted:9/1/2012
language:English
pages:23