# fishing

Document Sample

```					                                 Fishing Expeditions
Purpose

This model allows the user to generate Type I error through the process of multiple testing
(fishing.) The user will define twenty binominal factors (i.e can only have two states: present
and absent) and then the computer will random assign observations form these parent
populations into two samples. Next, the user will have the opportunity to discover if there any
real or apparent differences between the two samples. Keep in mind that since the members
of both samples came from the same parent distribution, any such differences were due to

Directions

Define the names of the twenty variables in the shaded column labeled "Factor" You may
also choose to use the default values.

In the shaded cells besides each of these variable names, indicate the proportion of the
population that has that Factor present. This should be a number from 0.00 to 1.00. Note
that the computer will automatically calculate and display the proportion of the population who

Press the F9 key. This will automatically randomly draw members from the defined
population and assign them to Sample A and Sample B. This process will actually take place
twice. At that top of the screen, the computer will draw samples with 10 observations each
and at the bottom of the spreadsheet it will draw samples of size 50. The computer will place
the expected number of observations with the factor in Sample A in the column labeled "#

If you scroll to the right, you can see the actual observations in each sample. A "1" indicates
the factor is present and a "0" indicates it is absent.

The computer will then report on summary statistics which include: a) the actual number of
observation in sample A that has the factor (column labeled "# in Sample A"), b) the actual
number of observations in sample B that has the factor (column labeled "# in Sample B"), c)
the difference in these two number of observations (column labeled "diff"), and d) the ratio of
the # with the factor in sample A versus the # in sample B (column labeled "ratio").

The second set of two samples (n=50 each) also reports on whether the difference found
between the empirical proportions observed in the random samples is statistically significant.
These are reported as either significant as a one tailed or two tailed test.

You may change the values of any shaded cell and/or press F9 to draw new samples.

Questions
Define the population factor proportions as described in the directions and press F9. Look at
the top table (in which sample sizes are 10 each.) Were there any cases in wiich there is a
ratio of 2 to 1 or higher between the two samples? If yes, why? If no, press F9 until that

Scroll down to the results of the samples of size 50 can be seen. Look at the column labeled
"Statisical significance." Press F9 until there is significance. How can there be a statistically
significant difference between the groups if the observations in both samples were drawn

Continue to press F9 for about a dozen times. What can you say about the pattern of
statistically significant results?

Developed by Dr. Scott Wetstone, (860) 679-4440
n = 10 for each sample
Observed Sample Statistics
probability   probability      #          # in            # in
Factor              present        absent       Expected    sample A        sample B    diff
1                        Male     0.50           0.5           5            5               8        3
2                    Married      0.50           0.5           5            3               6        3
3            Wears glasses        0.50           0.5           5            3               8        5
4            Electric blanket     0.20           0.8           2            1               1        0
5                  Lead paint     0.60           0.4           6            6               8        2
6          >1 hour TV daily       0.70           0.3           7            6               8        2
7          >2 meals out/wk        0.80           0.2           8            7               8        1
8                   Has a cat     0.40           0.6           4            4               3        1
9                  Has a dog      0.33          0.67           3            2               3        1
10                  Speeding      0.20           0.8           2            3               2        1
11      Childbearing years        0.90           0.1           9            9               9        0
12     >2 cups coffee/day         0.60           0.4           6            7               7        0
13     Plays video games          0.10           0.9           1            0               2        2
14          Cancer in family      0.30           0.7           3            2               4        2
15            Visited Mexico      0.10           0.9           1            0               2        2
16     Exercises regularly        0.30           0.7           3            2               3        1
17   Full-time employment         0.60           0.4           6            6               6        0
18              Daily vitamin     0.40           0.6           4            5               4        1
19    Leo astrological sign       0.08          0.92           1            0               1        1
20    8+ hours sleep/night        0.50           0.5           5            2               5        3

n = 50 for each sample
Observed Sample Statistics
probability probability             #         # in            # in
Factor           present      absent              Exp       sample A        sample B    diff
1                      Male    0.50         0.5                25          24              20        4
2                    Married    0.50    0.5   25   33   26   7
3            Wears glasses      0.50    0.5   25   27   25   2
4            Electric blanket   0.20    0.8   10    8    6   2
5                  Lead paint   0.60    0.4   30   28   33   5
6          >1 hour TV daily     0.70    0.3   35   32   35   3
7          >2 meals out/wk      0.80    0.2   40   44   39   5
8                   Has a cat   0.40    0.6   20   13   21   8
9                  Has a dog    0.33   0.67   17   13   21   8
10                  Speeding    0.20    0.8   10   15    7   8
11      Childbearing years      0.90    0.1   45   42   48   6
12     >2 cups coffee/day       0.60    0.4   30   24   31   7
13     Plays video games        0.10    0.9    5    4    5   1
14          Cancer in family    0.30    0.7   15   18   15   3
15            Visited Mexico    0.10    0.9    5   10    4   6
16     Exercises regularly      0.30    0.7   15   21   21   0
17   Full-time employment       0.60    0.4   30   28   36   8
18              Daily vitamin   0.40    0.6   20   15   22   7
19    Leo astrological sign     0.08   0.92    4    3    5   2
20    8+ hours sleep/night      0.50    0.5   25   24   25   1
A zero means the trait is absent and a one means it is present.
Sample A
<-------------10 observations ----------------------------------------------------------------------------------------------------
ratio               1            2           3              4              5              6              7
1.6                 0            1          0              1              1              1              1
2.0                 0            0          1              0              0              0              0
2.7                 0            0          1              0              0              1              0
1.0                 0            1          0              0              0              0              0
1.3                 1            1          0              1              1              1              0
1.3                 1            0          1              1              1              0              1
1.1                 1            0          1              0              0              1              1
1.3                 1            0          0              1              0              0              1
1.5                 0            0          1              0              0              0              0
1.5                 0            0          0              0              1              1              0
1.0                 1            1          1              0              1              1              1
1.0                 1            0          1              1              1              0              1
***                 0            0          0              0              0              0              0
2.0                 0            0          0              0              1              0              0
***                 0            0          0              0              0              0              0
1.5                 1            0          0              0              0              0              0
1.0                 0            0          0              1              1              1              1
1.3                 0            1          0              1              1              0              0
***                 0            0          0              0              0              0              0
2.5                 0            0          0              0              0              0              0

Sample A
Statistical        <-------------50 observations ---------------------------------------
ratio    Za         Zb           Za-b        Signif. pooled p      1            2           3
1.2    -0.1       -1.4           0.8                  0.4          0            1          0
1.3    2.4    0.3    1.4            0.6   1   1   1
1.1    0.7    0.0    0.4            0.5   1   1   1
1.3   -0.5   -1.4    0.6            0.1   0   0   0
1.2   -0.4    0.9   -1.0            0.6   1   0   0
1.1   -0.8    0.0   -0.6            0.7   1   1   1
1.1    1.6   -0.4    1.3            0.8   1   0   1
1.6   -1.9    0.3   -1.7            0.3   0   1   0
1.6   -0.9    1.4   -1.7            0.3   1   1   0
2.1    1.9   -1.1    1.9   1 tail   0.2   1   0   0
1.1   -1.2    1.4   -2.0   2 tail   0.9   1   1   1
1.3   -1.6    0.3   -1.4            0.6   0   0   1
1.3   -0.2    0.0   -0.3            0.1   0   0   0
1.2    1.1    0.0    0.6            0.3   0   1   1
2.5    2.6   -0.5    1.7   1 tail   0.1   0   0   0
1.0    2.0    1.9    0.0            0.4   0   1   0
1.3   -0.4    1.7   -1.7            0.6   0   0   1
1.5   -1.3    0.6   -1.4            0.4   1   1   0
1.7   -0.3    0.5   -0.7            0.1   0   0   0
1.0   -0.1    0.0   -0.2            0.5   0   1   0
Sample A                     Sample B
------------------------------------------------------------------>          <-------------10 observations ----------------------------------------------------------------------
8              9              10                           1            2           3              4              5
0               0              0                            1            1          1              0              1
1               1              0                            1            0          1              1              0
0               1              0                            1            1          0              0              1
0               0              0                            0            0          1              0              0
0               1              0                            1            1          1              1              0
0               1              0                            1            1          0              1              1
1               1              1                            1            0          1              1              1
0               1              0                            0            0          0              1              0
0               1              0                            0            0          1              1              0
1               0              0                            0            1          0              0              0
1               1              1                            1            0          1              1              1
1               0              1                            1            0          1              1              0
0               0              0                            0            0          1              0              0
0               1              0                            1            0          0              1              0
0               0              0                            0            1          0              0              0
0               0              1                            0            1          0              1              1
1               0              1                            1            0          1              1              1
1               0              1                            0            0          1              1              0
0               0              0                            0            0          0              0              0
0               1              1                            0            1          0              0              0

ations -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
4              5              6              7              8              9              10             11             12
0              0              0              0              1               1              0              1              0
1   1   1   0   1   1   1   0   0
1   1   1   0   0   0   1   1   1
1   0   0   1   0   0   0   1   1
1   1   1   0   0   0   0   1   1
1   0   1   1   0   1   1   0   1
1   1   1   1   1   0   1   1   1
0   1   0   0   0   0   0   0   1
1   0   0   0   0   0   0   0   0
0   1   1   0   0   0   0   0   0
1   1   1   1   1   0   1   1   1
0   0   1   1   0   0   1   1   1
0   0   0   0   0   0   0   0   0
1   0   1   0   0   1   0   0   1
0   0   0   0   0   0   1   0   0
1   1   0   0   0   0   1   0   0
1   1   1   0   0   1   1   1   0
0   0   0   1   1   0   1   0   0
0   0   0   0   0   0   0   0   0
0   1   1   1   0   0   0   0   1
Sample B
------------------------------------------------------------------------------------------------>
6              7              8              9              10
1              1              0               1              1
1              0              1               0              1
1              1              1               1              1
0              0              0               0              0
1              0              1               1              1
1              0              1               1              1
1              1              1               0              1
1              0              1               0              0
0              0              1               0              0
0              0              0               1              0
1              1              1               1              1
1              1              0               1              1
0              1              0               0              0
0              1              1               0              0
0              0              0               0              1
0              0              0               0              0
0              0              1               0              1
0              1              0               1              0
1              0              0               0              0
0              1              1               1              1

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
13             14             15             16             17             18             19              20             21
1              0              0              0              1              1               0              0              0
0   1   0   1   1   0   1   0   1
0   0   0   0   1   1   0   0   0
0   0   0   0   0   0   0   1   0
0   0   0   1   0   1   0   0   0
1   0   1   0   1   1   0   1   0
1   0   1   1   1   0   1   1   1
1   0   0   0   0   0   0   0   0
0   0   0   0   0   1   0   0   0
0   1   0   1   0   1   0   0   0
0   1   1   1   0   1   0   1   1
1   0   1   1   0   1   1   0   1
0   0   0   0   0   0   0   1   0
1   0   0   0   0   0   0   0   0
1   1   1   0   0   0   0   0   0
1   0   1   0   0   1   1   0   0
0   1   0   0   0   1   1   1   1
1   0   1   0   0   1   0   0   0
0   0   1   0   0   0   0   0   0
0   1   0   1   0   1   1   1   1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------
22             23             24             25             26             27             28             29             30
0              0              1              1              1              1              1               1              1
0   1   0   1   1   0   0   1   0
0   1   0   0   1   1   1   1   0
0   0   0   0   1   1   0   0   0
1   1   0   0   0   1   0   1   1
1   0   1   0   1   1   1   1   1
1   1   1   1   1   1   1   1   0
1   1   0   1   0   0   0   0   0
1   0   1   0   0   1   0   0   0
1   0   0   0   0   0   1   0   1
1   0   0   1   1   1   1   1   1
0   1   0   1   0   0   1   0   0
0   0   1   0   0   0   1   0   0
0   0   0   0   1   0   1   0   0
0   0   0   0   0   0   0   1   0
0   0   0   1   0   1   1   0   1
0   0   0   0   0   1   1   0   1
0   0   0   1   0   1   0   0   0
0   0   0   0   0   0   0   0   0
1   1   0   0   0   1   0   0   1
-------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------
31             32             33             34             35            36             37             38             39
0              1              0              0              0             0              0              1              1
1   1   1   1   1   1   0   1   0
1   0   0   0   0   0   1   0   0
0   0   0   0   0   0   0   0   1
0   1   1   1   1   0   0   1   0
1   0   0   1   0   0   0   1   1
1   1   1   1   1   1   1   1   1
0   0   0   0   0   0   0   0   0
1   1   0   0   1   1   0   1   0
0   0   0   0   1   1   1   0   0
1   1   1   1   1   1   1   1   1
0   0   0   0   1   0   1   1   1
0   0   0   0   1   0   0   0   0
1   0   1   1   0   0   1   0   0
0   1   1   0   0   1   0   0   0
0   0   1   1   1   0   1   0   1
1   1   1   0   1   1   0   0   1
0   0   0   0   0   1   0   0   0
0   0   0   0   0   0   0   0   0
1   0   1   0   0   0   1   1   1
----------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------------------------
40             41             42              43             44             45             46             47            48
0               1              1              1              0              1              0              1             1
1   1   1   1   1   1   1   1   0
0   1   1   1   1   1   0   1   1
0   0   0   0   0   0   0   0   0
0   1   1   1   1   1   1   1   1
1   1   1   1   0   1   0   1   1
1   1   1   1   1   1   0   1   1
1   0   1   1   1   0   1   0   0
0   0   0   0   0   0   0   0   0
0   0   1   1   1   0   0   0   0
1   1   0   1   0   1   1   1   1
1   1   0   0   0   0   1   0   1
0   0   0   0   0   0   0   0   0
0   0   1   1   1   0   0   0   0
0   0   0   1   0   0   1   0   0
0   1   0   0   0   1   0   1   0
1   0   1   1   0   1   1   0   0
0   1   0   1   0   0   0   1   0
0   0   0   1   0   0   0   1   0
0   0   0   1   0   1   1   0   0
Sample A           Sample B
------------------------------------------------>   <-------------50 observations -------------------------------------------------------------------------------------
49             50                  1            2           3              4              5              6
1              0                   0            0          1              1              0              0
0   0   0   0   0   1   1   1
1   1   1   0   0   1   1   1
0   0   0   0   0   0   0   0
1   1   0   1   1   0   1   1
0   0   0   1   1   1   0   1
1   1   1   1   1   0   1   1
1   0   1   0   0   0   0   0
0   1   0   0   0   1   0   1
0   0   0   0   0   0   0   0
1   1   1   1   1   1   1   1
0   1   1   1   0   1   1   1
0   0   0   0   0   0   0   0
1   1   0   1   1   1   0   0
0   0   0   1   0   0   0   0
0   1   1   0   0   0   0   0
1   1   1   0   1   1   1   1
1   0   0   0   0   1   0   1
0   0   0   1   0   1   1   0
0   1   0   0   0   1   1   0
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
7              8              9              10             11             12             13             14             15
0              0               0              1              0              1              0              1              0
0   1   0   0   1   1   0   1   1
1   1   1   0   0   1   0   0   1
0   0   0   0   0   0   0   0   1
1   0   1   1   1   1   0   1   1
1   1   1   1   1   1   0   0   1
0   1   1   1   1   1   0   1   1
0   0   0   0   0   1   0   0   0
0   0   0   0   0   1   0   1   1
0   0   0   0   0   1   0   0   0
1   1   1   1   1   1   1   1   1
0   1   0   0   1   1   1   1   1
1   0   0   0   0   0   0   0   0
1   1   1   0   1   1   0   0   1
0   0   0   0   0   0   0   0   0
0   1   1   1   1   1   1   1   0
1   1   1   0   0   0   1   1   1
1   1   1   0   1   1   0   0   1
0   0   0   0   0   0   0   0   0
1   0   0   1   0   1   1   1   0
----------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------
16             17             18             19              20             21             22             23             24
1              0              1               0              1              0              1              1              1
1   0   0   1   0   0   1   1   0
1   0   1   1   1   1   1   0   1
0   0   1   0   1   0   0   0   0
1   1   0   1   1   0   1   0   1
1   1   1   1   0   0   0   1   0
1   1   1   1   0   1   1   1   1
0   0   1   0   0   1   0   1   1
1   1   0   0   0   1   1   0   0
0   0   0   0   0   0   1   0   0
1   1   1   1   1   1   1   1   1
0   0   0   1   0   1   0   1   0
0   1   0   0   0   0   0   0   0
0   0   0   0   0   0   0   1   0
0   0   0   0   0   0   1   0   0
0   0   0   1   0   1   0   0   0
1   0   1   0   1   0   1   1   1
0   0   0   0   1   0   1   1   1
0   0   0   0   0   0   0   0   0
0   1   0   0   1   1   1   1   0
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
25             26             27             28             29             30              31             32             33
0              1              0              0               1              1              1              0              1
1   0   1   0   0   1   1   1   0
0   0   0   0   0   0   0   1   0
1   0   0   0   0   0   0   0   0
0   0   1   0   1   0   1   1   1
1   1   1   1   0   1   0   0   1
1   1   1   1   1   0   1   0   0
0   1   1   1   0   0   0   1   1
0   0   0   0   0   0   1   0   1
0   0   1   0   0   1   0   0   0
1   1   1   1   1   1   1   1   1
1   0   1   0   1   1   1   0   1
0   0   0   0   0   1   0   0   0
1   1   0   0   0   0   0   0   0
0   0   0   0   1   1   0   0   0
1   0   0   0   1   0   0   0   0
0   1   1   1   1   0   1   1   1
0   0   1   0   0   1   0   0   0
0   0   0   0   0   0   0   0   0
0   1   1   1   0   1   0   0   1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------
34             35             36             37             38             39             40             41             42
0              0              1              0              1              0              0               0              0
1   1   1   1   0   0   0   1   0
0   0   0   0   1   0   1   0   0
0   0   0   0   0   0   0   0   0
0   0   1   1   1   1   1   1   1
1   1   1   1   1   1   0   0   1
1   1   1   1   1   0   1   1   1
0   0   1   1   1   0   1   0   1
1   0   0   0   0   1   0   1   1
0   0   0   0   0   0   0   0   0
1   1   1   1   1   1   1   1   1
1   1   1   0   1   0   1   0   1
0   0   1   0   0   0   0   0   0
0   0   0   0   0   0   0   1   1
0   0   0   0   0   0   0   0   0
0   1   1   0   0   1   1   0   0
1   1   1   1   1   1   1   0   0
1   0   0   0   0   1   1   0   0
0   0   0   0   0   0   0   0   0
1   0   1   1   0   0   0   0   1
Sample B
---------------------------------------------------------------------------->
------------------------------------------------------------------------------------
43             44             45             46             47            48             49             50
1              0              0              1              0             0              0              0
1   1   0   1   1   0   0   0
1   1   0   1   1   1   0   1
0   0   0   0   1   0   0   1
1   0   1   1   0   0   1   0
1   1   1   0   0   1   1   1
0   0   1   1   0   1   1   1
1   0   1   0   0   1   1   1
0   1   1   1   0   1   1   1
0   1   0   0   1   0   1   0
0   1   1   1   1   1   0   1
1   0   1   0   1   0   1   1
0   0   0   0   1   0   0   0
1   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0
0   1   1   1   1   0   1   0
0   1   0   1   1   0   1   1
0   1   1   1   1   0   1   0
0   1   0   0   0   0   0   1
1   0   0   0   0   1   1   1

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 3 posted: 12/7/2011 language: pages: 28
How are you planning on using Docstoc?