sample size margin of error

Document Sample
sample size margin of error Powered By Docstoc
					                Discussion on Sample Sizes and the Margin of Error
Reliable versus Valid

Often, clients ask me what I think a valid sample size would be, and always my retort is a sample
size of 1. The two terms, valid, or validity, and reliable or reliability tend to get used
interchangeably when in fact they really mean very different things. Why is 1 sample point
valid? Well, because the definition of valid is well-grounded, sound. For one person this may be
true, it certainly is the truth as they know it. However, if one wants a reliable sample size then
we need to understand the definition of reliable; that is, something that can be relied on,
dependable. Therefore, sample that is reliable and dependable and that can be relied upon and
generalized to the larger universe from which it was drawn is the key to successful sample

Confidence Level

The confidence level or Level of Confidence, to which it is sometimes referred, is the confidence
one can have on the reliability or direction of the responses. For example, if a survey were
replicated 100 times with the same amount of sample each time, then 95 times one would expect
the results to fall within the limits of the margin of error, and 5 times one would expect the
results to fall outside the limits of the margin of error.

Margin of Error

The margin of error, sometimes referred to as the range around the result, is dependent upon
several factors: the result itself, and the sample size. The sample size required is a function of
several factors as well: the margin of error one is willing to tolerate and the budget available to
conduct the study.

Determining the Sample Size

First, let’s discuss the ideal world and then we can move to a practical discussion that will assist
the reader on determining the sample size that best meets the needs of the business.

Suppose you wish to have sufficient enough sample that will give one a fairly tight margin of
error. Further, let’s suppose you are willing to tolerate no more than a margin of error of + or – 3
percentage points around the result from your study. If you cannot estimate or don’t wish to
hazard a guess on the percent satisfied from the study, use 50%. Given the 50% result and the
requirement for a sample sufficient to not exceed + or – 3 percentage points, the required number
of completed interviews in the study would be 1,050. The margin or error can be reduced by
introducing one of several factors – increasing the sample size or increasing the estimated
percent satisfied.

For example, if previous studies have indicated that you should really achieve an 80%
satisfaction result, then the required sample with a margin of error of + or – 3 percentage points
would be 670. If you cannot guess the percent satisfied and can live with a larger margin of

error; let’s say 4.5 percentage points [+ or -], then the sample required would only have to be 470
completed interviews.

Practical Applications

Most clients are happy to have a margin of error no greater than + or – 5 percentage points at the
95% level of confidence. Using the several scenarios above, the sample requirement would look
as follows:

               Required Sample Size                               Percent Satisfied
                     380                                               50%
                     370                                               60%
                     320                                               70%
                     250                                               80%
                     140                                               90%

These sample sizes, of course, are for each segment of a sampled population that one wishes to
view. If only one national sample is required, and assuming it is randomly drawn, then the above
example works fine. If, on the other hand, there are two segments in the sampled universe, and
one wishes the same precision in each of the segments, then each segment would have to have
the required sample as in the above example. This also means that at the national level,
assuming one will aggregate the two segments, the sample size will be twice that of an individual
segment. This will, in effect, improve the margin of error at the national level, but the
improvement is contingent upon the levels of satisfaction in the two segments.

Budget Matters

Of course, the above discussion is good for making a case with a scientifically drawn and
represented sample. Ultimately, it comes down to the budget available for the study as it relates
to the margin of error one is willing to tolerate. The dollars for any one particular study is also
dependent upon the length of the survey, the respondent eligibility requirements and the
incidence (those that qualify to complete the survey) of the sample listing. For discussions on
these matters, please read the Customer Opinion Research, Inc. paper on Survey Design and

What next?

If you would like to calculate a margin of error based on your sample size and satisfaction rate
data, visit our Margin of Error calculation webpage at


Shared By: