VIEWS: 3 PAGES: 14 POSTED ON: 12/31/2011
Analyzing Research Data and Presenting Findings Instrumentation—the whole process of data collection addresses: who will collect data? when will it be collected? how often will it be collected? where will it be collected? what data is to be collected using which instrument? Instrument—documents data collected They can provide a variety of data types: • Descriptions—verbal representations of participants, etc. • Scores—numerical values assigned to test performance • Measurements—numerical values resulting from instruments other than tests • Opinions—views expressed by participants and informants • Statements—authoritative verbal opinions Researcher Completes Subject Completes Rating scales Questionnaires Interview schedule Self-checklists Tally sheet Attitude scales Flowcharts Personality inventories Performance checklists Achievement/aptitude tests Anecdotal records Performance tests Time and motion logs Sociometric devices Qualitative and Quantitative Data once collected need to be analyzed before interpretations can be made – these describe the data clearly, identify what is typical and atypical about the data, identifies patterns and relationships in the data and answers the research questions and hypotheses. Mertler and Charles suggest that qualitative data are analyze logico-inductively Making observations of behavior, situations, interactions, objects and environment Identifies topics from the observations and reviews these to find patterns and categories Induces conclusions from what is observed Uses conclusions to answer research questions Quantitative data is analyzed mathematically and results are expressed in statistical interpretations Depicts what is typical and atypical among the data Shows degrees of difference or relationships between two or more variables Determines the likelihood that the findings are real for the population Qualitative Analysis Example Quantitative Analysis Example What is the typical January school day Do first nations students enrolled in BC like for a ninth-grade student attendinghigh schools retain the traditional Victoria High School in 1935? beliefs about natural phenomena that contradict concepts presented in the 1. What time did school start? science curriculum and do these 2. What would a student have to do influence science achievement? before attending school? Null hypothesis: no relationship exist 3. What was the weather like in between adherence to first national January? cultural beliefs and science achievement on the Provincial exam in 4. How did the school transport to grade 10. school? This would be a correlation and 5. What was the morning potentially a regression problem and instruction like? analysis. 6. How was the lunch period managed and delivered? Monitored? 7. When was school dismissed? 8. What type and amount of homework was typical? By answering the sub questions we should be able to gather enough evidence to answer the main research question. Verbal data would be analyzed logically by matching evidence to research questions. No numerical analysis would be required. Type of Data Analysis Most Common to Type of Research Qualitative Quantitative Ethnographic * Historical * * Survey * * Correlation * Causal-comparative * Experimental * Mixed-method * * Evaluation * * Action * * Exercise in Investigating Data Analysis and Interpretation Activity on page 128 of the text – Video and Data Analysis Comments Qualitative Research: Data Analysis and Interpretation Building Research Skills Investigating Data Analysis and Interpretation The first step in analysis is to read and write memos about all field notes, transcripts, and observer comments to get an initial sense of the data. First, scan the data presented in the video to get a feel for it as a whole. Any initial thoughts? Once scanned, the data should be classified. The first step in classifying data is to organize it and break it into segments. A segment is a word, group of words, or sentence that is comprehensive by itself. That is, it contains one idea or piece of information relevant to the study. For example, again consider the description of after-school activities from the second child shown in the video: Well, first, uh, um, when I get home usually, uh, just look if I have any homework usually. And then after I'm done with that I, if there's like enough daylight sometimes go out and play basketball if it's a nice day. Or maybe, uh, sometimes go to my friend's house after school. This response should have three segments—one for each sentence (i.e., do homework, play basketball, go to a friend's house). To segment the data from the video and accompanying transcript, copy or retype the data as it is provided on the site into a word processor. As you work, copy the segments you have selected from your word processor and paste them into the essay box provided, numbering each segment according to the video clip from which you've taken it (e.g., Early childhood 1, Middle Childhood 1, etc.), and separating each with a semicolon. You may highlight specific segments within the same sentence as this example shows: Before I started golf, I would come home and then I would usually do my homework, or watch some TV, or get on the computer. This sentence can be coded as containing four segments. Remember, you may segment and code differently, based on your own intuitions. Label each segment with a descriptive name for the subject matter (not the meaning) of the segment. These labels are called topics. Classification is idiosyncratic—how you classify the data will likely be different from how a classmate classifies the same data. Interpreting the data requires identifying any patterns you feel represent the major issues that arise in the data. Interpretation should be focused on the answers to the following four questions: 1. What is important in the data? 2. Why is it important? 3. What can be learned from it? 4. So what? Exercise in Analyzing Data for a Qualitative Study Read the article by Lenski, Crawford, Crumpler, and Stallworth (2005) see pdf handout to answer the following questions about data analysis. Please be prepared to discuss this with the class. 1. Identify the section(s) of the article where the authors describe preparing the data for analysis. 2. Identify the section(s) of the article where the authors describe exploring and coding the data. 3. Identify the section(s) of the article where the authors describe building results from the coding. 4. Identify the section(s) of the article where the authors describe validating their findings. Peculiarities of Ethnographic Research (logico-inductive analysis or hypothetico- inductive analysis) The questions answered by ethnographic researchers often come only after the data is being analyzed Attempts to draw conclusions from a broad rather than limited picture of human behavior Systematic process of analysis o Id the topics o Cluster these into categories o Form the categories into patterns o Make explanations from what the patterns suggest The reduction of large amounts of data via a coding scheme Description of the main features of the categories resulting from the coding Interpret the simplified and organized materials Need to have the class determine just what this is - logico-inductive analysis or hypothetico-inductive analysis http://www.scribd.com/doc/3008642/Research-in-Education URL’s for qualitative data analysis NVivo (http://www.qsrinternational.com/products_nvivo.aspx) ATLAS.ti (http://www.atlasti.com) anSWR (http://www.cdc.gov/hiv/software/answr.htm) EZ-Text (http://www.cdc.gov/hiv/software/ez-text.htm) Qualrus (http://www.qualrus.com/Qualrus.shtml) HyperRESEARCH (http://www.researchware) Analyzing Research Data with Statistics What Statistics are used for… To summarize data and reveal what is typical and atypical To show relative standing of individuals in a group To show relationships among variables To show similarities and difference among groups To identify error that is inherent in sample selection The test for significance of findings To make inferences about the population Descriptive Statistics – these help clarify data from samples. Central Tendency Mean – (X bar or M) arithmetic average Median – Mdn or Md the midpoint of scores Mode – Mo is the most frequently made score Variability Range – difference between highest and lowest Variance (s2) and Standard Deviation(s) – dispersion and standardized dispersion Relative Standing Percentile rank – rank assignment indicating percentage of score that fell below that score in the norm sample Stanine – standard 9’s ranking from 1 to 9 with 5 being median and 2 being SD Relationships Correlation Coefficient – (Pearson is r) measure of relationship between two or more sets of scores made by the same group of participants Inferential statistics – used to make estimates about the population based upon what has been learned from the sample Error estimates – the range within which a given measure probably lies within the population (if you take several samples from a population these samples would differ slightly - the standard error offers info on how well a sample represents a population) Confidence intervals – indicates the probability that a population value lies within certain specific boundaries Tests of significance – Correlation – if the study were repeated 100’s of times a correlation of this absolute value or larger would be expected to occur 95% of the time (p < .05) Difference between two means (t-test) – if this study were repeated 100’s of times a difference between means of this absolute value or larger would be expected to occur 95% of the time (p < .05) Difference between more than two means (ANOVA) - if this study were repeated 100’s of times a difference between three or more means of this absolute value or larger would be expected to occur 95% of the time (p < .05) The Distribution of Means To illustrate this principle, estimate the mean length of a sentence. To begin, use the three paragraphs as your three samples. Count the number of words in each sentence, then compute the mean length of sentence for each sample. Sample 1: Inferential statistics are data analysis techniques for determining how likely it is that results obtained from a sample or samples are the same results that would have been obtained from the entire population. Put another way, inferential statistics are used to make inferences about parameters, based on the statistics from a sample (see Chapter 12 to review the distinction between statistics and parameters). In the simplest language, whereas descriptive statistics show how often or how frequent an event or score occurred, inferential statistics help researchers to know whether they can generalize to a population of individuals based on information obtained from a limited number of research participants (see Chapter 5 to review sampling techniques and the importance of a representative sample for making generalizations). Sample 2: As with any normal distribution, a distribution of sample means has not only its own mean (i.e., the mean of the means) but also its own standard deviation (i.e., the difference between each sample mean and the mean of the means). The standard deviation of the sample means is usually called the standard error of the mean (SEM . The word error indicates that the various sample means making up the distribution contain some error in their estimate of the population mean. The standard error of the mean reflects how far, on average, any sample mean would differ from the population mean. According to the normal curve percentages (see Figure 12.3), we can say that approximately 68% of the sample means will fall within one standard error on either side of the mean (remember, the standard error of the mean is a standard deviation), 95% will fall between ±2 standard errors, and 99+% will fall between ±3 standard errors. In other words, if the population mean is 60, and the standard error of the mean is 10, we can expect 68% of the sample means (i.e., means of the scores taken from each sample) to be between 50 and 70 (60 ± 10), 95% of the sample means to fall between 40 and 80 [60 ± 2(10)], and 99% of the sample means to fall between 30 and 90 [60 ± 3(10)]. Thus, in this example it is very likely that a sample mean would be 65, but a sample mean of 98 is highly unlikely because 99% of sample means fall between 30 and 90. Given a number of large, randomly selected samples, we can quite accurately estimate population parameters (i.e., the mean and standard deviation of the whole population) by computing the mean and standard deviation of the sample means. The smaller the standard error, the more accurate the sample means as estimators of the population mean. Sample 3: Based on a test of significance, as we have discussed, the researcher will either reject or not reject the null hypothesis. In other, the researcher will make the decision that the difference between the means is, or is not, likely due to chance. Because we are dealing with probability, not certainty, we never know for sure whether we are absolutely correct. Sometimes we'll make mistakes–we'll decide that a difference is a real difference when it's really due to chance, or we'll decide that a difference is due to chance when it's not. These mistakes are known as Type I and Type II errors. Next, compute the mean of the sample means. This is the estimate of the population mean. The standard deviations for the three samples are 17.10 (Sample 1), 19.31 (Sample 2), and 7.65 (Sample 3). With this information, compute the standard error of the mean. The standard error can be calculated by summing the squared differences of the sample means from the overall mean and dividing by the number of samples Knowing the standard error of the mean allows you to state the confidence limits for your estimate of the population mean. In other words, 95 percent of all sample means will fall between two values—what are those two values? Confidence intervals basic idea is to construct a range of values within which we think the population lies. Mean of sample plus or minus (1.96 times Standard error) Exercise for Understanding the Results of a Study (pdf on inferential statistics) What statistic should you use to evaluate whether the groups are significantly different? Are the groups significantly different? Did you read from the top row or the bottom row of the table? In other words, does Levene's test show that the variances of the groups are equal? Explain what these numbers mean, as if talking to someone who hasn't taken statistics. Be sure, in your response to explain the purpose of the study when describing the results (e.g., "these groups are different" is not meaningful to someone who does not know what the groups are).