Power

							i ni yi

1
Control

2
Intervention

Power (1 - β)

1.00

0.0000 0.0050 0.0100
0.0150 0.0200 0.0250 0.0300 0.0350

0.0000 0.6221 0.7124
0.7632 0.7976 0.8230 0.8428 0.8588

50

100

0.0400
0.0450 0.0500 0.0550 0.0600 0.0650

0.8720
0.8832 0.8929 0.9012 0.9086 0.9151

50.00

100.00

0.80

Size effect

50.00

0.0700
0.0750 0.0800 0.0850 0.0900 0.0950

0.9209
0.9261 0.9308 0.9351 0.9389 0.9425

σi deff ei

50.00

100.00

0.60

0.1000
0.1050 0.1100 0.1150 0.1200 0.1250

0.9458
0.9488 0.9516 0.9541 0.9565 0.9588

2.00

2.00
0.40

0.1300
0.1350 0.1400 0.1450 0.1500 0.1550

0.9608
0.9628 0.9646 0.9663 0.9679 0.9694

10.00

14.14

0.1650 0.9721
0.1700 0.1750 0.1800 0.9734 0.9745 0.9757

0.1600

0.9708

1
-sided test Significance level

0.20
-1.30

0.1850 0.9767
0.1900
0.1950 0.2000 0.2050 0.2100

0.9777
0.9787 0.9796 0.9804 0.9813

α 1-β

0.0500
Power

-2.00 0.2150 0.9820

0.00
0.10 0.25

0.70 2.10 2.00 10.00 0.01

0.2200 0.2250 0.2300 0.2350 0.2400

0.9828 0.9835 0.9841 0.9848 0.9854 0.9860 0.9865

0.8929

0.000.2450
0.2500

0.05

0.10

0.15

0.20

0.25

Significance level (α)

State of the world Null hypothesis H0 true H0: y2 - y1 = 0 Z ≤ 1.64 P (correctly accepting the null hypothesis) = ##### H0 acepted if ŷ2 - ŷ1 ≤ 20.15 1-α= Z > 1.64 P (Type I error) = α = 5.00% H0 rejected if ŷ2 - ŷ1 > 20.15 Significance level of the test = There are two types of errors that can be made in statistical testing. Type I error is rejecting the null when it is true and Type II is accepting when it is false and some alternative hypothesis is true. The probability of making a Type I error is called the significance level of the test. probability of making a Type II error is called the power of the test. It is equivalent to the probability of correctly rejecting the null hypothe alternative when the alternative hypothesis is true. For reference, The level of significance (α) gives the probability of rejecting the null hypothesis when it is true (type I error). Research findings

Decision rule Z= ŷ2 - ŷ1 √(σ1²/n1+σ2²/n2) based on ( this is a one -sided test )

The power gives the probability of rejecting the null hypothesis when an alternative hypothesis is true (which equals 1 minus the p II error (1-β)). A desirable property of a test is that it minimizes the probability of making both type I and type II errors (i.e. low significance level, high p sample size, there is generally a tradeoff between significance level and power. A higher significance level comes at the expense of lower p versa. Both α can be decreased and 1-β increased by increasing the sample size. Assume that the goal is to determine the sample sizes required to carry out statistical tests that attain a sufficient level of significance and p variables that will be of interest. If a desired level of significance is specified, as well as the type of test to be carried out, the alternative hy desired power, then one can calculate the required sample size. Let us illustrate how to do this with a simple example of a one-sided test of between two means.

Let y2 denote the average outcome measure for the treated group and y1 the average outcome measure for the control group. Suppose we co alternative that treatment increased mean treatment group outcomes above control group outcomes by Δ units. The null and alternative hyp

H0: y1 = y2 HA: y2 - y1 = Δ By a central limit theorem, the difference between two sample means, appropriately standardized follows a normal distribution. The power to
      1    1   1      sides    

 12
n1



2 2

n2

      

Function SIZE_FOR_POWER(Y1, Y2, S1, S2, ALPHA, SIDES, POWER) NA = 1: PA = POWER_FOR_SIZE(Y1, Y2, S1, S2, ALPHA, SIDES, NA) NB = 1E+15: PB = POWER_FOR_SIZE(Y1, Y2, S1, S2, ALPHA, SIDES, NB) Do NC = (NA + NB) / 2: PC = POWER_FOR_SIZE(Y1, Y2, S1, S2, ALPHA, SIDES, NC) If PC > POWER Then NB = NC: PB = PC Else NA = NC: PA = PC Loop Until Abs(PC - POWER) < 1E-16 SIZE_FOR_POWER = NC End Function Function POWER_FOR_SIZE(Y1, Y2, S1, S2, ALPHA, SIDES, SIZE) POWER_FOR_SIZE = 1 - Application.WorksheetFunction.NormDist( _ -Application.WorksheetFunction.NormInv(ALPHA / SIDES, 0, 1) _ - (Y2 - Y1) / Sqr(S1 ^ 2 / SIZE + S2 ^ 2 / SIZE), 0, 1, True) End Function

Where σ1 and σ2 are the standard deviations in the outcomes in the control and treatment group samples and Φ is the standard normal cumu function. To compute the required sample sizes, n1 and n2, we need to specify the desired level of significance (α), the power (1-β), the alte standard deviations (σ1 and σ2). Also, since different combinations of n1 and n2 can solve the equation, we need to know the desired ratio o sizes. Calculations in the "Power" spreadsheet can also be done with Stata's sampsi command: sampsi 50 100, sd1(50) sd2(100) n1(50) n2(100) alpha(0.05) onesided sampsi 50 100, sd1(50) sd2(100) power(0.8929) alpha(0.05) onesided or with the VBA macros:

State of the world Alternative hypothesis HA true HA: y2 - y1 = Δ = 50.00 P (Type II error) = β = #####

P (correctly rejecting the null hypothesis) = ##### 1 - β = Power of the test = ng the null when it is true and Type II is accepting the null hypothesis I error is called the significance level of the test. 1 minus the probability of correctly rejecting the null hypothesis in favor of the

when it is true (type I error).

ve hypothesis is true (which equals 1 minus the probability of a type

d type II errors (i.e. low significance level, high power). For a given ignificance level comes at the expense of lower power and vice

that attain a sufficient level of significance and power with regard to he type of test to be carried out, the alternative hypothesis and the this with a simple example of a one-sided test of the equality

me measure for the control group. Suppose we consider the outcomes by Δ units. The null and alternative hypotheses are:

ardized follows a normal distribution. The power of the test is equal

group samples and Φ is the standard normal cumulative distribution level of significance (α), the power (1-β), the alternative (Δ) and the the equation, we need to know the desired ratio of the n1/n2 sample

onesided nesided

NC)

, 0, 1) _ 1, True)