Research Ideas_ Critiquing Research_ and Hypotheses

					     Chapter Two:
Research Ideas, Critiquing
Research, and Hypotheses
                 The Research Idea

   You find a research idea when you find a gap in
    the current knowledge or an unanswered
    question that interests you.
    Characteristics of Good Research Ideas

 The most important characteristic of a good
  research idea is that it is testable.
 A second characteristic of the good research
  idea is that your chances for success are
  increased when your view of nature
  approximates reality as closely as possible.
  When reality is approximated there is a good
  likelihood of success.
            Sources of Research Ideas
   Nonsystematic Sources
     Inspiration: Ideas that pop into one’s mind from
      (seemingly) nowhere. Inspiration usually
      comes more easily after one has been
      working on a particular problem for some
     Serendipity: Refers to those situations where
      we look for one phenomenon but find
     Everyday Occurrences: The people and/or
      situations one encounters daily provide some
      of the best possibilities for research.
             Sources of Research Ideas

   Systematic Sources
    Past Research
       A careful survey of the research done in a
        specific area will highlight any knowledge
        gaps or unanswered questions in that area.
         A failure to replicate a previous finding
          raises additional questions that only
          continued research will be able to
                Sources of Research Ideas

   Systematic Sources
     Theory: The guidance function of a theory
      provides an endless panorama of projects for
      researchers who take the time and trouble to
      master the theory and understand its
     Classroom Lectures: Lectures often include a
      systematic review of the relevant literature on
      a particular topic and as such are a good
      source of research ideas.
        Developing a Research Question

 Regardless of the source of your research idea,
  your first goal should be to turn it into a
 Once you have a question, you need to survey
  the literature to find out what is already known
  about the question.
       Surveying the Psychological Literature

   Selection of Index terms
    Thesaurus of Psychological Index Terms
        Is a collection of index terms taken from
         abstracts of all published psychological
         Surveying the Psychological Literature

   Computerized Searches of the Literature
      Your next step is to familiarize yourself with the
       computerized research databases (such as PsycINFO)
       available at your school (we use PsycARTICLES). Enter
       your index terms into the search box of the database.
      Internet can be useful tool, but be cautious when selecting
       information to use. Web pages present additional evaluation
       challenges. While there are no universally accepted
       guidelines, the following criteria are useful: authority,
       accuracy, objectivity, currency, coverage, and
         Surveying the Psychological Literature
   Obtaining the Relevant Publications
      You need to assemble all of your original source materials in one
      PDF Online
      Photocopy
      Interlibrary loan
          If your library does not have the relevant journals or books, you
           can request them through interlibrary loan. There is
           sometimes a small fee for this service and the amount of time
           it takes to get your materials will vary.
      Requests for reprints
          You can write or e-mail the author of a journal article directly
           and ask for a copy of the article (reprint). Many colleges and
           universities have searchable faculty e-mail databases. This
           makes it very easy to e-mail the author and ask for a reprint.
       Surveying the Psychological Literature

   Integrating the results of the Literature Search
     This is the process of making sense of the
       materials you have assembled.
     Taking good notes from articles read and
       summarizing information under APA
       headings on a single page is a helpful
       Critiquing Psychological Research

 Important to examine studies reviewed closely
  to spot flaws
 Do think critically about methods, results, and
  the conclusions.
 Do question assumptions the researchers seem
  to make.
 Do wonder if there are other ways of defining
  and measuring the variables used
Guidelines for Critiquing Psychological Research
   Does the literature review adequately describe the
    research area? Is this material consistent with the specific
    research question?
   As a research project evolves, the literature review and the
    actual experiment diverge somewhat over time.
   After you complete your project and work on the report, double-
    check to make certain that the actual project still shows a direct
    link with your research literature.
   Because most researchers carry out programmatic research,
    their new research ideas are likely to build directly on their (and
    others’) previous research.
Guidelines for Critiquing Psychological Research
   Is the research question stated clearly? Do you
    have a clear idea concerning the research to be
   The title and abstract of a research report should give
    you an indication of the research’s topic, although they
    may not contain the specific question per se.
   You will find the author’s review of relevant literature in
    the article’s introduction. As you read further into the
    introduction, the literature should apply more
    specifically to the particular research question. The
    research question will often be in the last paragraph of
    the introduction.
Guidelines for Critiquing Psychological Research
   In view of the research area and research question, are the
    hypotheses appropriate, clearly stated, and able to be
    stated in general implication form?
   Appropriate hypotheses are those that follow logically from the
    literature review.
   If you find a hypothesis that seems to come from nowhere and
    surprises you, it may be inappropriate – reread the introduction
    to make sure.
   A clearly stated hypothesis is one that you can easily
    understand without having to guess what the researcher is
   Remember that general implication form is the “if….then”
Guidelines for Critiquing Psychological Research

 Are the key terms operationally defined?
 The reader should not have to guess what a
  researcher means when he or she refers to a
  specific independent, dependent, or extraneous
 Remember that operational definitions mean
  that you should define your variables in terms of
  the operations you use to manipulate, measure,
  or control them.
Guidelines for Critiquing Psychological Research

 Are the IV’s and their levels appropriate?
 Be sure to pick a manipulation that is actually
  appropriate to the IV – don’t choose something
  merely because it is easy or convenient to use.
 Be sure to choose the levels of your IV
  appropriately. Choose levels of the IV to
  answer your experimental question, but do so
  economically (remember the principle of
Guidelines for Critiquing Psychological Research
   Is the DV appropriate for this research? Should
    the researcher have included more than one DV if
    only one was recorded?
   If a researcher wishes to study a particular outcome,
    the behavior chosen for measuring (the dependent
    variable) should be a good indicator of that outcome.
   The operational definition of the DV should be one that
    other researchers would judge to be valid.
   A researcher with broad interests should use multiple
    DV’s to get a better sense of the concept he or she is
Guidelines for Critiquing Psychological Research
   Are the controls sufficient and appropriate? Are there any
    uncontrolled variables that could affect the results of the
   Leaving variables uncontrolled can result in a confounded
    experiment which leaves the researcher unable to draw a
   As you look for possible extraneous variables, you should
    concentrate on variables that have a legitimate or reasonable
    chance to actually make a difference.
   Look for extraneous variation, but don’t go overboard and find
    variation that most researchers would consider negligible.
Guidelines for Critiquing Psychological Research
   Did the author(s) use an appropriate research design to
    test the specific hypotheses and answer the general
    research question?
   With poor planning, it is possible to gather data for which there
    is no appropriate research design and, thus, no appropriate
    statistical test.
   Make sure that research reports use designs that match the
    question(s) they sought to answer.
          For example, if the researcher asked a question involving
           multiple IV’s, the experiment should involve a factorial
Guidelines for Critiquing Psychological Research
   Assuming you had access to the appropriate equipment
    and materials, could you replicate the research after
    reading the method section?
   The method section should contain enough detail about the
    variables and procedures of the experiment to enable a reader
    to replicate the experiment.
   The reader should not have to guess about any of the
    manipulations, measurements, or controls the researcher used.
   The reader must have all the vital details of the experiment in
    order to evaluate the operational definitions, the variables, and
    the procedures used in the experiment.
Guidelines for Critiquing Psychological Research
   Did the researcher(s) use appropriate sampling procedures
    to select the participants and assign them to groups?
   Random sampling and assignment in creating independent
    groups or the appropriate matching or repeated measures
    approach for correlated groups are important for both internal
    and external validity.
   If a researcher uses sampling techniques that result in biased
    samples, the internal validity of the experiment is threatened
    because the groups are likely to be different before the
   Biased samples also threaten external validity.
Guidelines for Critiquing Psychological Research

   What procedures were used to ensure group
    equivalence prior to the experiment?
   Poor sampling techniques can result in biased
   Biased samples are usually not equivalent before the
    experiment begins, so it would be impossible to draw
    valid conclusions about the effects of the IV (internal
    validity would be compromised).
   If you have reason to doubt the equivalence of your
    groups beforehand, you would be wise to use a
Guidelines for Critiquing Psychological Research

   Did the research use a sufficient number of
   With small numbers of participants, statistical tests are
    simply less powerful to detect differences – the
    differences between groups have to be quite large for
    the difference to turn out significant.
   Don’t back yourself into a corner so that you use the
    age-old student lament after your experiment: “If I had
    run more participants, my differences might have been
Guidelines for Critiquing Psychological Research
   Were there any history, instrumentation, statistical
    regression, or mortality effects that might have influenced
    the results?
   For history, be alert to outside events that occur that could
    affect the results.
   Be sure to check the operation of your equipment before each
    session to avoid instrumentation effects.
   Choosing extreme high- or low-scoring participants can result in
    lower or higher scores, respectively, simply due to statistical
   If many participants drop out of one condition in the experiment
    (i.e., mortality), the participants who are left in that condition
    may differ in some important way(s) from the participants in
    other conditions.
Guidelines for Critiquing Psychological Research
   Were the appropriate statistical tests used, and are they
    reported correctly?
   You may need to consult a statistic text or someone who
    teaches statistics to help you answer this question.
   On the other hand, this guideline points out the importance of
    becoming statistically knowledgeable so that you can evaluate
    this guideline on your own.
   Remember that statistics are merely a tool experimenters use
    to decipher the results they obtained – you should be well
    armed with the proper tools as you evaluate and conduct
Guidelines for Critiquing Psychological Research

   Did the author(s) report means, standard deviations, and a
    measure of effect size?
   Group means may allow the reader to compare participants’
    performance against existing norms.
   Standard deviations may allow the reader to determine that
    nonsignificant findings are due to extreme variability between
    groups rather than small differences between means.
   Effect sizes give standard comparison units so that readers can
    compare significant differences from several different
Guidelines for Critiquing Psychological Research

 Are the tables and figures clearly and
  appropriately labeled and presented
 Tables and figures should present a large
  amount of data than is possible in writing.
 Just as paragraph after paragraph of statistical
  results can be confusing, a poorly constructed
  table or figure can confuse the reader.
Guidelines for Critiquing Psychological Research
   Does the author(s) correctly interpret the results?
    Does the discussion follow logically from the
   Did the researcher correctly interpret p < .05 as
    significant and p > .05 as nonsignificant?
   Did the researcher give a correct interpretation of his
    or her results in light of previous research?
   Does the discussion “make sense” given the data the
    researcher just presented.
   Authors should make it clear when conclusions follow
    from data and when they are engaging in speculation.
Guidelines for Critiquing Psychological Research

   Are the conclusions and generalizations valid and
    justified by the data? Did the author(s) consider
    other possible interpretations of the results?
   This difficulty often comes when researchers have a
    favorite theory that they espouse.
   Sometimes, this theoretical leaning is so strong that it
    seems to blind them to any alternative explanations.
   Alternative explanations for findings may provide you
    with the impetus for a new experiment.
Guidelines for Critiquing Psychological Research

   Do all references cited in the text appear in the
    reference section, and vice versa?
   It is highly unlikely you would find this problem in a
    published study.
   There should be a one-to-one correspondence of the
    citations in the text and the references at the end of
    the study.
   The reference section of an APA-format report
    consists only of material that you have read and
    included in the report.
Guidelines for Critiquing Psychological Research
   Did the experimenter follow appropriate ethical
    procedures during all phases of the experiment?
   To evaluate this guideline, you man need to refresh
    your memory of the ethical principles that
    psychologists follow in conducting research.
   Some older research involves some procedures that
    have been hotly debated as far as their ethical nature
    is concerned.
   It is doubtful that any ethically questionable study
    would receive approval from an institutional review
      Formulating the Research Hypothesis

 A research hypothesis is simply a formal
  statement of your research question, taking into
  account what you learned from searching the
 The research or experimental hypothesis is our
  prediction about the relation that exists between
  the independent variable that we are going to
  manipulate and the dependent variable that we
  will record.
     Characteristics of the Research Hypothesis

   General Implication Form
    You must be able to state (or restate) the
     research hypothesis in general implication
     (“if…then”) form.
       The “if” portion of such statements refers to
        the independent variable manipulation(s)
        that we are going to make, whereas the
        “then” portion of the statement refers to the
        dependent variable changes we expect to
     Characteristics of the Research Hypothesis

   Principle of Falsifiability
    When an experimental hypothesis is stated in
      general implication form, it is possible that a
      result is true (supported by the results of the
      study) or false (not supported by the results
      of the study).
     Characteristics of the Research Hypothesis

   Directional versus Nondirectional Research
    Directional Hypotheses
        Specify the outcome of the experiment.

    Nondirectional Hypotheses
        Do not predict the exact directional
         outcome of an experiment, but only that the
         groups we are testing will differ.

Shared By: