Research Strategies and Validity

Document Sample
Research Strategies and Validity Powered By Docstoc
					Research Strategies and
        Chapter 6
      Sept. 22, 2010
• Intro to research strategies
  – Description and correlation (Ch. 12 & 13)
  – Experimental design (Ch. 7, 8, 9)
  – Quasi-experimental and developmental design (Ch.

• Validity of research findings
  – Internal validity
  – External validity
Steps in the research process
1. Find a research idea
2. Convert the idea into a hypothesis
3. Define and measure variables
4. Identify participants
5. Select a research strategy
6. Conduct the study
7. Evaluate the data
8. Report the results
9. Refine/reformulate your research idea
          Research strategy
• The scientific approach chosen to answer
  a research question

• Types of research strategies
  – Descriptive and correlational designs
  – Experimental design
  – Quasi-experimental and developmental
Descriptive and correlational designs
• Descriptive designs
  – Describe variables as they exist naturally
  – Variables are not manipulated

  Example: You could measure the keyboard skills of freshmen and
    seniors. We might identify a difference in ability, but it would not
    produce any explanation for why that difference existed.
Descriptive and correlational designs
• Correlational designs
  – Evaluate the relationship between two
  – Obtain measures of each variable from each
  – Variables are not manipulated
  – Determine predictive relationships but not
    causal relationships
Descriptive and correlational designs
• Correlational designs
   Is there a correlation between car top speed and number of dates?
   Maybe. However, one thing might not cause the other.
   Example: Being an excited thrill seeker might lead to both buying fast
   cars and being the kind of person that people love to date.
   Or: People owning fast cars are being perceived as ‘rich’ and are
   therefore more popular….

   Also, advertising: "People who have bought this item also bought the
   following items…."
         Experimental design
• Goal is to determine causal relationships
  between two variables

• Three characteristics of an experiment
  – Independent variable is manipulated
  – Dependent variable is measured
  – Random assignment
          Experimental design
• Independent variable (IV)
  – Variable that is manipulated
  – Variable we want to see the effect of
  – Divided into different levels or conditions

• Dependent variable (DV)
  – The variable used to measure the effect
          Experimental design
• Random assignment
  – Assigning participants to the levels of the IV

  – Each participant has an equal chance of
    being placed in each level/condition

  – Method of minimizing differences between
    participants across conditions
         Experimental design
• Random selection
  – Method of choosing the people who will
    participate in the study
  – Randomly select from the larger sampling
    frame or population
        Experimental design

• Random assignment is essential for an
  experimental design

• Random selection is not
 Example of an experimental study
• Dr. Anderson wants to know if the feedback she
  gives her students on their homework improves their
  scores on the final.

• By flipping a coin, she decides which students will
  get written comments on their homework and which
  students will get a total score with no comments.

• Then she compares final exam scores of the two
        Quasi-experimental and
        developmental designs

• Goal is to determine causal relationships
  between two variables
• Distinguishing characteristics
   – IV is manipulated
   – DV is measured
   – There is no random assignment

• Lack the full level of control of experiments
    Example of a quasi-experimental
•   Dr. Jones and Dr. Smith both teach a research methods
    course on Tuesday afternoon.
•   They want to determine whether one-sided or two-sided
    arguments are more persuasive.
•   Dr. Jones uses one-sided and Dr. Smith uses two-sided
    arguments with their classes to try to persuade their
    students to accept a tuition increase.

    Another example: In a “does smoking cause cancer?” study, you would
    use pre-existing groups (smokers vs. non-smokers). It would be unethical
    to make non-smokers smoke 20 cigarettes a day.
Identifying design of studies
            Variable manipulated?

       No                             Yes

 Non-experiment             Random assignment?
 (Descriptive or
                          No                   Yes

                   Quasi-experiment         Experiment
   Internal and external validity

• Describes the validity of a research study
  – Quality of the research process
     • Does it study what it intends to study?

  – Accuracy of the results
     • Can we trust the findings?
   Internal and external validity
Internal validity

  – Degree to which a study produces a single,
    unambiguous explanation for the results

  – Freedom from confounds (allowing
    alternative explanations)
   Internal and external validity
External validity
  – Extent to which the results of a study can
    generalize to different populations, settings,
    and conditions
                      Externally Valid?
•   Dr. Anderson wants to know if the feedback she gives her students on their
    homework improves their scores on the final.

•   By flipping a coin, she decides which students will get written comments on their
    homework and which students will get a total score with no comments.

•   Then she compares final exam scores of the two groups.

    The study was carried out on self-motivated, graduate students. She finds
    that there is no difference in performance between the 2 groups.

    Dr. Anderson concludes that feedback on homework is never needed for
   Internal and external validity
• Threats to validity
   – Discussed with each design

• External vs. internal validity trade-off
   – Increasing internal validity limits external validity
     and vice-versa

• Research designs and validity
   – Descriptive & correlational: better external
   – Experimental & quasi: better internal
Descriptive Research Strategies
           Chapter 13
  Descriptive research designs
• Non-experimental study
  – No manipulation of variables
  – Observe variables as they naturally exist

• Provides detailed information regarding
  naturally occurring behavior

• Strategy for preliminary data collection
• Descriptive research strategies
  – Observational research design
  – Survey research design
  – Case study design

  * These designs can be used to collect
    correlational/experimental data as well as
    descriptive data
Observational research design
• Without intervention
  – Naturalistic (non-participant) observation
  – Participant observation
  – Contrived Observation

  – Indirect measures
 Observational research design
Naturalistic Observation

• Definition
   – Behavior is observed in a natural setting
   – The observation is as unobtrusive as possible

• When are they used?
   – When manipulation would be impractical or unethical
 Observational research design
Participant Observation

   – Researcher engages in the same activities as the people being

Contrived Observation

   – Researcher sets up the observational situation
 Observational research design
Indirect Measures

• Definition
   – The participant is not present during observation
   – Non-reactive form of observation

• When are they used?
   – When there are concerns about reactivity
   – When behavior cannot be observed directly
 Observational research design
• Advantages
  – Behavior observed in the real world
  – Reduces reactivity
  – High external validity

• Disadvantages
  – Time consuming
  – Subjective interpretation possible
         Survey research design
• Self-reported information
    –   attitudes
    –   opinions
    –   personal characteristics
    –   behaviors

• Uses survey or questionnaire format
    – Open-ended or closed-ended questions, rating scales
    – Oral or written presentation
         •   In person
         •   Phone
         •   Mail
         •   Internet

• Describes a particular group of individuals
    – Often tries to be representative of a larger population
     Survey research design
• Advantages
  – Can gather large amounts of information
  – Time efficient

• Disadvantages
  – Relies on participant report
  – Response bias (reactivity)
  – Response rate (representativeness)
               Case study design
• Definition
   – The study of one individual, setting, event, or phenomenon in as
     intense a manner as possible

• Examples
   –   Multiple personality
   –   “HM” and brain injury
   –   “S” and memory
   –   Genie and language development
             Case study design
• Advantages
  – Used to study rare phenomena
  – Uses a multimethod approach
  – Rich account of an individual, event, or phenomenon

• Disadvantages
  – Time-consuming
  – Observer/experimenter bias possible
  – Difficult to generalize from a single case
          Example of case study
• Jonestown Massacre

Making Sense of the Nonsensical: An
 Analysis of Jonestown.
         --Neal Osherow
Descriptive designs and validity
• Internal validity
   – Little control
   – Little causal interpretation possible
   – Behavior vulnerable to subjective interpretation

• External validity
   – High generalizability to real world contexts from observation
   – Generalizability from surveys depends on representativeness of
   – Low generalizability beyond case study

Shared By: