Learning Center
Plans & pricing Sign in
Sign Out



									            Survey Research

AD700              College of Advancing Studies
13 October, 2004   Brendan Rapple
  This presentation owes much to the American
  Statistical Association brochure series on survey
   Little Cards on Restaurant Tables:
                 Was the service good?

   Telephone:
                 Is president doing a good job?
                 The most popular programs on public radio?
   Census:
                 How many bathrooms do you have?

   Magazines:
                 How is your romantic life?

   Market Researchers:
                 Brand X or Brand Y?
                 How many drinks last week?
     Surveys Provide Important Knowledge

   Economists, psychologists, health professionals,
    political scientists, and sociologists conduct surveys to
    study such topics as:

       Income and expenditure patterns among households;
       Roots of ethnic or racial prejudice;
       Implications of health problems on people’s lives;
       Voting behavior;
       Effects on family life of women working outside the home, etc.
           Specific Purpose Essential

   Objectives of a survey should be as
          Specific
          Clear-cut
          Unambiguous as possible

   "Men's Health Practices" is a very nebulous topic.

   How often do African-American males aged 40-49 visit the
Steps in Conducting A Survey

    Define precise purpose

    Specify population

    Specify appropriate sample

    How to administer survey?

    Draft of survey instrument

    Pretest it

    Revise it

    Administer survey to sample

    Analyze, write it up, and communicate the results

    Use results meaningfully
Decide on Mode of Data Collection

    Mail

    Telephone

    In Person Interview

    Computer

   Critical for identifying questionnaire problems.

   Main problems revolve about:

      Question content, e.g. confusion with overall meaning of question as
       well as misinterpretation of individual terms or concepts

      Formatting, e.g. problems with how to skip or navigate from question
       to question may result in missing data and frustration for both
       interviewers and respondents.

   Individuals

   Larger units, e.g. families
     Sometimes Difficult to Specify

   e.g. female faculty members at BC: do we include
    part-time profs?
   Must be representative of population.

   Are the distributions of attributes, opinions, and beliefs in the
    sample the same as in the population?

   You want to be able to make inferences about the population as a
    whole based on what you find to be true of the sample.

   Often difficult to find representative sample.

   Always a danger of sampling error or bias.
           Quality of Sample Important

   The quality of the sample – whether it is up-to-date and
    complete – is probably the dominant feature for
    ensuring adequate coverage of the desired population
    to be surveyed.

   Variability is large, then sample should be large

   Converse also true
       2 Barrels of Apples

   Barrel A (low variability) -- all apples about 3 ins.
    in diameter (range 3.1 to 2.9 ins.)

   Barrel B (high variability) -- apples range from 2
    to 5 ins. in diameter

   Picking 3 apples from Barrel B might give result
    well below (above) average.
      Size of Sample Isn't Everything

   Large numbers do not, in and of themselves, increase
    the representativeness of a sample.

   Most professional survey conductors hold that a
    moderate sample size is enough statistically and
             Whole Population and Sample
              Sometimes the Same

   – Small companies in the paper recycling industry in LA.

    –   Unit of Analysis: a company

    –   You define "small company" as a private co. with turnover of
        less than $2,000,000 per annum

    –   Research shows that there are 34 relevant companies

    –   Therefore, manageable to use ALL in sample
   Results will only relate to small paper recycling
   companies in LA -- difficult to generalize about
   other types of company in other parts of the
    Population Often Not Feasible Due to Size

             •   Welfare Recipients

             •   Mentally Ill

             •   Prison Inmates

   Often essential to survey a REPRESENTATIVE
       Early Studies of Gay Men

   Sampling frame composed of men, patients of
    therapists participating in research

   But most gay men were not patients of therapists
        Representative Sample

   EXAMPLE--Success of unwed teenage mothers in
    raising children?
   To be representative, sample must contain same
    proportion of unwed teenage mothers at

        --each age level

        --each educational level

        --each socio-economic status
                                    in community
     Lists May Be Very Exclusive

       Undocumented Aliens

              --We know that many live in LA

              --But relying on Govt. lists may be useless
           Suppose You Have a “Population,”

   all registered voters in your county

   all Mercedes owners in the state

   all soccer players in your school district who drive green mopeds

     Random Sample
Random =
     Purposeful & methodical

     Not reflect biases of researcher

     Everyone has equal & independent chance
     of being selected
               Random Sample
   Once selected it cannot be chosen again (like lottery

     Example: 500 part-time students in Advancing Studies

     Sample of 20% is required

     Assign each student a number from 1 to 500

     Randomly select 100 numbers (by computer or by table of
      random numbers)
       Systematic Random Sampling

Example 1.
 2,000 in sampling frame and you want a sample of 200, then you
  might select every 10th name

Example 2.
 500 part-time students in Advancing Studies
 Sample of 20% is required

        --Randomly Select a Number from 1 to 5

        --Select Every 5th Person

        --002, 007, 012. 017, 022, and up to 497.
             Possible Problem:
   Staff in govt. agency may be listed unit by unit

   Each unit has 9 line-level workers and 1 supervisor.

   The supervisor is the 10th person on the list.

   It’s a survey of 20% -- every 5th person is selected.

   If first no. selected is 1, 2, 3, or 4 then no supervisor will be
    selected, though they comprise 10% of population.

   If first number selected is 5, then supervisors will be greatly

    Thus, possibility of bias due to periodicity or patterns.
        Stratified Sampling
Population: 2,000 (800 females; 1,200 males)

Sample required:          200

If gender is an important variable in your survey, then
both females and males should be included in
appropriate numbers, that is, in proportions that
correspond to their presence in the population.
Treat both sexes as separate populations and take 10%
sample from each.


Make sure that all females are listed first and then take
every tenth name.

Either way you will end up with 80 females and 120 males
Convenience Sampling
          Cluster Sampling
Often difficult to list all members of target population
and select the sample from among them

e.g. 1)    Population of American high school

    2)     Population of U.S. postal delivery workers

    3)     Adult population of Atlanta
                 Possible Strategies

   Population of American high school students
       choose 50 schools randomly from entire list and include all
       students in those schools in the sample.

   Population of U.S. postal delivery workers
       choose 100 post offices randomly from all 50 States and include
       all deliverers in those post offices in the sample.

   Adult population of Atlanta
       Randomly choose sample of 50 blocks from a city map and
       then poll all adults living on those blocks.
                       Potential Problems

   Confidentiality of data supplied by respondents is of prime
    concern to all reputable survey organizations.

   Strategies:
      Using only number codes to link the respondent to a questionnaire.
      Refusing to divulge names and addresses of survey respondents.
      Omitting the names and addresses of survey respondents from
       computer files used for analysis
      Presenting statistical tabulations by broad enough categories so that
       individual respondents cannot be singled out.

   Important that individual respondents are not identified in
    reporting survey findings.

   All of the survey’s results should be presented in totally
    anonymous summaries, such as statistical tables and charts.
   Volunteers usually have characteristics that differentiate
    them from the larger population.
   The fact that they volunteer makes them different from
    persons in the population who do not volunteer.
   They tend (but not in all circumstances):
     – to be better educated

     – have higher social class

     – to be more intelligent

     – have greater need for social approval

     – to be more sociable

     – more unconventional (especially when volunteering for
       studies of sex behavior)
     – less authoritarian

     – less conforming
             Volunteers (Cont.)

   Jews more likely than Protestants.

   Protestants more likely than Catholics.

   Females more likely than males.
               Volunteers -- Example

   TV programs asking viewers to vote.

         – people call who are most committed to issue.

         – “stuffing of ballots” by multiple calls.

         – Time of day is important – who’s available?
                     Margin of Error

   Error margin of 1,000 randomly chosen individuals is said
    to be 3.1%.

   Thus, if a random sample of 1,000 indicates that 59% will
    vote for Bush, the actual number could range from 55.9% to

    In the 1984 election, the Gallup Survey (using 3,456
    individual responses) missed by just +0.2 of 1% when it
    predicted that Ronald Reagan would win by 59.0%.

To top