3.2 Analyzing Research Data and Presenting Findings

Document Sample
3.2 Analyzing Research Data and Presenting Findings Powered By Docstoc
					                Analyzing Research Data and Presenting Findings

Instrumentation—the whole process of data collection addresses:

      who will collect data?
      when will it be collected?
      how often will it be collected?

      where will it be collected?
      what data is to be collected using
      which instrument?

Instrument—documents data collected
They can provide a variety of data types:
      • Descriptions—verbal representations of participants, etc.
      • Scores—numerical values assigned to test performance
      • Measurements—numerical values resulting from instruments other than
      tests
      • Opinions—views expressed by participants and informants
      • Statements—authoritative verbal opinions

Researcher Completes                       Subject Completes
Rating scales                              Questionnaires
Interview schedule                         Self-checklists
Tally sheet                                Attitude scales
Flowcharts                                 Personality inventories
Performance checklists                     Achievement/aptitude tests
Anecdotal records                          Performance tests
Time and motion logs                       Sociometric devices
Qualitative and Quantitative Data once collected need to be analyzed before
interpretations can be made – these describe the data clearly, identify what is
typical and atypical about the data, identifies patterns and relationships in the
data and answers the research questions and hypotheses.

Mertler and Charles suggest that qualitative data are analyze logico-inductively

    Making observations of behavior, situations, interactions, objects and
     environment

    Identifies topics from the observations and reviews these to find patterns
     and categories

    Induces conclusions from what is observed

    Uses conclusions to answer research questions

Quantitative data is analyzed mathematically and results are expressed in
statistical interpretations

    Depicts what is typical and atypical among the data

    Shows degrees of difference or relationships between two or more
     variables

    Determines the likelihood that the findings are real for the population
        Qualitative Analysis Example          Quantitative Analysis Example

What is the typical January school day  Do first nations students enrolled in BC
like for a ninth-grade student attendinghigh schools retain the traditional
Victoria High School in 1935?           beliefs about natural phenomena that
                                        contradict concepts presented in the
   1.   What time did school start?
                                        science curriculum and do these
   2.   What would a student have to do influence science achievement?
        before attending school?
                                        Null hypothesis: no relationship exist
   3.   What was the weather like in    between adherence to first national
        January?                        cultural beliefs and science
                                        achievement on the Provincial exam in
   4.   How did the school transport to
                                        grade 10.
        school?
                                        This would be a correlation and
   5.   What was the morning
                                        potentially a regression problem and
        instruction like?
                                        analysis.
   6.   How was the lunch period
        managed and delivered?
        Monitored?

   7. When was school dismissed?

   8. What type and amount of
      homework was typical?

By answering the sub questions we
should be able to gather enough
evidence to answer the main research
question. Verbal data would be
analyzed logically by matching evidence
to research questions. No numerical
analysis would be required.
Type of Data Analysis Most Common to Type of Research

                                Qualitative             Quantitative

Ethnographic                        *

Historical                          *                        *

Survey                              *                        *

Correlation                                                  *

Causal-comparative                                           *

Experimental                                                 *

Mixed-method                        *                        *

Evaluation                          *                        *

Action                              *                        *
Exercise in Investigating Data Analysis and Interpretation

Activity on page 128 of the text – Video and Data Analysis Comments

Qualitative Research: Data Analysis and Interpretation

Building Research Skills

Investigating Data Analysis and Interpretation

The first step in analysis is to read and write memos about all field notes,
transcripts, and observer comments to get an initial sense of the data. First, scan
the data presented in the video to get a feel for it as a whole.

Any initial thoughts?

Once scanned, the data should be classified. The first step in classifying data is to
organize it and break it into segments. A segment is a word, group of words, or
sentence that is comprehensive by itself. That is, it contains one idea or piece of
information relevant to the study.

For example, again consider the description of after-school activities from the
second child shown in the video:

      Well, first, uh, um, when I get home usually, uh, just look if I have any
      homework usually. And then after I'm done with that I, if there's like
      enough daylight sometimes go out and play basketball if it's a nice day. Or
      maybe, uh, sometimes go to my friend's house after school.

This response should have three segments—one for each sentence (i.e., do
homework, play basketball, go to a friend's house).

To segment the data from the video and accompanying transcript, copy or retype
the data as it is provided on the site into a word processor. As you work, copy the
segments you have selected from your word processor and paste them into the
essay box provided, numbering each segment according to the video clip from
which you've taken it (e.g., Early childhood 1, Middle Childhood 1, etc.), and
separating each with a semicolon. You may highlight specific segments within the
same sentence as this example shows:
Before I started golf, I would come home and then I would usually do my
homework, or watch some TV, or get on the computer.

This sentence can be coded as containing four segments. Remember, you may
segment and code differently, based on your own intuitions.

Label each segment with a descriptive name for the subject matter (not the
meaning) of the segment. These labels are called topics.

Classification is idiosyncratic—how you classify the data will likely be different
from how a classmate classifies the same data.

Interpreting the data requires identifying any patterns you feel represent the
major issues that arise in the data. Interpretation should be focused on the
answers to the following four questions:

   1.   What is important in the data?
   2.   Why is it important?
   3.   What can be learned from it?
   4.   So what?
Exercise in Analyzing Data for a Qualitative Study

Read the article by Lenski, Crawford, Crumpler, and Stallworth (2005) see pdf
handout to answer the following questions about data analysis. Please be
prepared to discuss this with the class.

   1. Identify the section(s) of the article where the authors describe
      preparing the data for analysis.

   2. Identify the section(s) of the article where the authors describe
      exploring and coding the data.

   3. Identify the section(s) of the article where the authors describe
      building results from the coding.

   4. Identify the section(s) of the article where the authors describe
      validating their findings.
Peculiarities of Ethnographic Research (logico-inductive analysis or hypothetico-
inductive analysis)

    The questions answered by ethnographic researchers often come only after
     the data is being analyzed

    Attempts to draw conclusions from a broad rather than limited picture of
     human behavior

    Systematic process of analysis

         o   Id the topics
         o   Cluster these into categories
         o   Form the categories into patterns
         o   Make explanations from what the patterns suggest

    The reduction of large amounts of data via a coding scheme

    Description of the main features of the categories resulting from the coding

    Interpret the simplified and organized materials

Need to have the class determine just what this is - logico-inductive analysis or
hypothetico-inductive analysis

http://www.scribd.com/doc/3008642/Research-in-Education

URL’s for qualitative data analysis

    NVivo (http://www.qsrinternational.com/products_nvivo.aspx)

    ATLAS.ti (http://www.atlasti.com)

    anSWR (http://www.cdc.gov/hiv/software/answr.htm)

    EZ-Text (http://www.cdc.gov/hiv/software/ez-text.htm)

    Qualrus (http://www.qualrus.com/Qualrus.shtml)

    HyperRESEARCH (http://www.researchware)
                        Analyzing Research Data with Statistics

What Statistics are used for…

    To summarize data and reveal what is typical and atypical

    To show relative standing of individuals in a group

    To show relationships among variables

    To show similarities and difference among groups

    To identify error that is inherent in sample selection

    The test for significance of findings

    To make inferences about the population
Descriptive Statistics – these help clarify data from samples.

Central Tendency

Mean – (X bar or M) arithmetic average

Median – Mdn or Md the midpoint of scores

Mode – Mo is the most frequently made score

Variability

Range – difference between highest and lowest

Variance (s2) and Standard Deviation(s) – dispersion and standardized dispersion

Relative Standing

Percentile rank – rank assignment indicating percentage of score that fell below
that score in the norm sample

Stanine – standard 9’s ranking from 1 to 9 with 5 being median and 2 being SD

Relationships

Correlation Coefficient – (Pearson is r) measure of relationship between two or
more sets of scores made by the same group of participants
Inferential statistics – used to make estimates about the population based
upon what has been learned from the sample

Error estimates – the range within which a given measure probably lies within the
population (if you take several samples from a population these samples would
differ slightly - the standard error offers info on how well a sample represents a
population)

Confidence intervals – indicates the probability that a population value lies within
certain specific boundaries

Tests of significance –

      Correlation – if the study were repeated 100’s of times a correlation of this
      absolute value or larger would be expected to occur 95% of the time (p <
      .05)

      Difference between two means (t-test) – if this study were repeated 100’s of
      times a difference between means of this absolute value or larger would be
      expected to occur 95% of the time (p < .05)

      Difference between more than two means (ANOVA) - if this study were
      repeated 100’s of times a difference between three or more means of this
      absolute value or larger would be expected to occur 95% of the time (p <
      .05)
The Distribution of Means

To illustrate this principle, estimate the mean length of a sentence. To
begin, use the three paragraphs as your three samples. Count the number
of words in each sentence, then compute the mean length of sentence for
each sample.

Sample 1: Inferential statistics are data analysis techniques for determining
how likely it is that results obtained from a sample or samples are the same
results that would have been obtained from the entire population. Put
another way, inferential statistics are used to make inferences about
parameters, based on the statistics from a sample (see Chapter 12 to
review the distinction between statistics and parameters). In the simplest
language, whereas descriptive statistics show how often or how frequent
an event or score occurred, inferential statistics help researchers to know
whether they can generalize to a population of individuals based on
information obtained from a limited number of research participants (see
Chapter 5 to review sampling techniques and the importance of a
representative sample for making generalizations).

Sample 2: As with any normal distribution, a distribution of sample means
has not only its own mean (i.e., the mean of the means) but also its own
standard deviation (i.e., the difference between each sample mean and the
mean of the means). The standard deviation of the sample means is usually
called the standard error of the mean (SEM . The word error indicates that
the various sample means making up the distribution contain some error in
their estimate of the population mean. The standard error of the mean
reflects how far, on average, any sample mean would differ from the
population mean. According to the normal curve percentages (see Figure
12.3), we can say that approximately 68% of the sample means will fall
within one standard error on either side of the mean (remember, the
standard error of the mean is a standard deviation), 95% will fall between
±2 standard errors, and 99+% will fall between ±3 standard errors. In other
words, if the population mean is 60, and the standard error of the mean is
10, we can expect 68% of the sample means (i.e., means of the scores taken
from each sample) to be between 50 and 70 (60 ± 10), 95% of the sample
means to fall between 40 and 80 [60 ± 2(10)], and 99% of the sample
means to fall between 30 and 90 [60 ± 3(10)]. Thus, in this example it is
very likely that a sample mean would be 65, but a sample mean of 98 is
highly unlikely because 99% of sample means fall between 30 and 90. Given
a number of large, randomly selected samples, we can quite accurately
estimate population parameters (i.e., the mean and standard deviation of
the whole population) by computing the mean and standard deviation of
the sample means. The smaller the standard error, the more accurate the
sample means as estimators of the population mean.

Sample 3: Based on a test of significance, as we have discussed, the
researcher will either reject or not reject the null hypothesis. In other, the
researcher will make the decision that the difference between the means
is, or is not, likely due to chance. Because we are dealing with probability,
not certainty, we never know for sure whether we are absolutely correct.
Sometimes we'll make mistakes–we'll decide that a difference is a real difference
when it's really due to chance, or we'll decide that a difference is due to chance
when it's not. These mistakes are known as Type I and Type II errors.

Next, compute the mean of the sample means. This is the estimate of the
population mean.

The standard deviations for the three samples are 17.10 (Sample 1), 19.31
(Sample 2), and 7.65 (Sample 3). With this information, compute the
standard error of the mean.
The standard error can be calculated by summing the squared differences of the sample means from the
overall mean and dividing by the number of samples

Knowing the standard error of the mean allows you to state the confidence
limits for your estimate of the population mean. In other words, 95
percent of all sample means will fall between two values—what are those
two values?
Confidence intervals basic idea is to construct a range of values within which we think the population
lies. Mean of sample plus or minus (1.96 times Standard error)
Exercise for Understanding the Results of a Study (pdf on inferential statistics)

What statistic should you use to evaluate whether the groups are
significantly different?




Are the groups significantly different?

Did you read from the top row or the bottom row of the table? In other
words, does Levene's test show that the variances of the groups are equal?

Explain what these numbers mean, as if talking to someone who hasn't
taken statistics. Be sure, in your response to explain the purpose of the
study when describing the results (e.g., "these groups are different" is not
meaningful to someone who does not know what the groups are).

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:3
posted:12/31/2011
language:
pages:14