ETEC 543 Session 4 Developing research questions is a fundamental task in any research. The research questions are essential to the success of your project because the research questions drive everything that you do as they are the questions that you seek to answer through the design and implementation of the study. There are a number of ways of developing research questions. First and foremost is to examine the literature base on the topic that you are researching to understand what research questions have and have not been asked and answered. Next is to consider your own practice in your own experience in the area, what types of questions interest you and what types of questions do you believe need to be answered or would be interesting to have answered. Third is to discuss your research with knowledgeable experts in the field accept their advice and review in your development of research questions. In all of this it's important to keep in mind the practicality of the research questions. As you develop research questions, or at some point in the process of developing research questions you need to make sure that you have the ability to construct an instrument or design methodology that will actually arrive at the answers to the questions that you are asking. It's hard underestimate the importance of connecting your research questions to an established theoretical base. This is especially true in quantitative studies as a quantitative study often tests a theory or uses an established theoretical base to define approaches to apply and test within the research design. After a thorough review of the literature base related to the topic that you're researching and a discussion of the topic with experts it is time to finalize your research questions. After develop a short list of potential questions spend some time to sort them or categorizing them and began to envision the type of research study that would be required to answer the types of questions that you have developed. This will allow you to reject or revise questions based on your ability to conduct the type of research you can do. Obviously if you know from the beginning that you want to do a certain type of study this should influence your questions. After settling on a few potential questions return to the literature base to cross-reference in the list of questions that you've developed with the studies that you've read related to that topic. This will hopefully help you find some ideas for the type of research methodologies to use as well as help you locate some potential instruments to use. Finally, consult with people who are experts in the field or who are knowledgeable about the topic that you're researching to help you refine and ultimately reduce your list of research questions to a manageable set. By the time you develop the research questions you should have a clear understanding of the type of research methodology needed to answer them. Then it is time to begin developing instruments and/or a research protocol. Knowing your research questions, and the type of research required to answer them will dictate your next steps. Quantitative studies typically use survey and questionnaires, while qualitative studies more frequently use questioning protocols for interviews, focus groups, document analysis techniques, and observation. By this time in the process you should be very well-versed in the literature related to the topic you're researching and that's the place to start. As has been mentioned before, it is a good idea to look for previous work done in the area for validated instruments or factor analyses that can help you identify the types of factors related to the phenomenon that you're researching. If you are able to adapt or adopt a previously used instrument you can save some steps in the research process. But if you are unable to do so you must develop your instrument or instruments, get expert review, and then do a pilot test to ensure that your instrument is valid and reliable. Then you get to move on to the data collection and on to completion of your research. Now let’s walk through this process of research question and methodology development in an actual study. After teaching online classes for a while a college faculty member wants to know if having students communicate with each other about the course material is beneficial. Much of the literature from the early experiences with online classes include references to the loss of social contact in online classes compared to face-to-face classes and a lot of early work seemed to be moving in the direction of trying to replicate the face-to-face experience in online classes. Further, it seems that if such communication is beneficial it would make sense to plan for more of such communication in the online classes. Discussion with an expert on evaluation indicates that grades are not an effective way of evaluating the effect of class discussions. Digging deeper in the literature the faculty member finds a theoretical construct called Social Presence that has shown some promise as a way to measure interaction in online discussions and finds several studies that have looked at student satisfaction as a way to judge the usefulness of student to student interactions. Research questions begin emerging such as: 1. Are levels of Social Presence experienced by students in an online class associated with student satisfaction with the online class? 2. What impacts Social Presence in online classes? 3. Does synchronous chat result in different levels of Social Presence in online classes? 4. Are higher levels of social presence related to higher levels of interaction with the course materials? 5. Are higher levels of social presence related to higher levels of motivation? 6. What kind of faculty behaviors can influence perceptions of interaction by students in online classes? 7. Does the type of discussion or the requirements for the discussion influence the Social Presence perceived by the students? After consulting with experts familiar with online learning, and then an e-mail consultation with an expert in Social Presence the faculty member, reduced and refined the research questions to the following set: 1. Are levels of Social Presence experienced by students in an online class associated with student satisfaction with the online class? 2. What factors impact student interaction in online classes? 3. Is Social Presence related to student motivation in online classes? 4. Does establishing a minimum communication goal and giving a grade for the discussion influence the level of Social Presence perceived by the students? 5. What kinds of interactions do students prefer in online classes and what are their perceptions of the impact of different types of interactions? Then considering the types of classes the faculty member had access to it was clear that it would be impractical to try to set up a true experimental design because it would be difficult to randomly select students to participate in different treatment groups. The faculty member considered a qualitative approach to find out more about interaction, specifically related to the second and fifth potential research questions. It seemed appropriate but being unfamiliar with qualitative approaches a decision was made to use a quantitative study to look at possible relationships between interaction and some other factor in the online classes. That led to a return to the literature base and a further refinement of the research questions to address the factor of student satisfaction and Social Presence with the research question being: 1. Are levels of Social Presence experienced by students in an online class associated with student satisfaction with the online class? Using a factor analysis found in the literature review the researcher developed a pool of items for Social Presence with factors related to isolation/aloneness, mutual awareness, psychological involvement, attention, empathy, mutual understanding, behavioral engagement, mutual assistance and dependent action. After developing over 100 items the researcher used expert review to rank the items and from the rankings the top 50 items were retained. Then the process was repeated for developing a satisfaction scale. Armed with over 50 items for the Social Presence scale and 50 items for the satisfaction scale the researcher sought and obtained IRB approval and did a pilot implementation in one class. The researcher then used a coefficient alpha statistical test to establish an estimate of internal reliability. Items that didn’t contribute to internal reliability were rejected leaving 32 items in the Social Presence scale and 21 items in the satisfaction scale. Using a statistical analysis program the researcher also found that there was indeed a correlation between Social Presence and Satisfaction in the pilot test sample. The instructor then administered the instruments to all of his classes and completed the analysis of the data. He then wrote the results and published a paper! Now you might wonder what would be different if the instructor in this case had wanted to pursue a qualitative study. After the literature review and initial formulation of potential research questions this instructor might have done just that. If you will recall, one of the potential questions the researcher developed was: What kinds of interactions do students prefer in online classes and what are their perceptions of the impact of different types of interactions? Given that interaction in online classes can be viewed as a social phenomenon a qualitative methodology could help better define the important dimensions of the issue. Another research question that might be interesting is: How do faculty and student perception of the importance and usefulness of student to student interaction in online classes compare? This research question stems from the idea that in a quest for recreating face to face experiences in online classes developers of online classes may be overlooking the very real possibility that students who select online classes may have different expectations for the need for interaction with other students. Putting the two ideas together would make an interesting study. The next step would be to develop a better understanding of the literature base related to the topic. Then the instructor would develop a research protocol that would attempt to describe the types of data collection that would be undertaken. Something that differentiates qualitative studies from quantitative studies is that you sometimes don’t know how best to gather data, or even from whom until you get into the process. So a certain amount of flexibility is needed. In this case the instructor would do well to gather a group of teachers of online classes to explore their understanding of the need for student to student interaction. The questioning protocol would probably begin with rapport building. This would likely be done by having the instructors tell a bit about themselves and their experiences teaching online classes. Then some open ended questions would be used to encourage the instructors to explain their use of and expectations for student to student interactions in their classes. Some questions that might be used would include: 1. How do you structure discussions in your classes? 2. What do you expect students to get out of discussions in your classes? 3. What is the role of student to student interactions in your classes? 4. What do students get out of discussions in online classes? 5. What do students expect, or want from discussions in online classes? Another source of data would be a document recovery method where you would, with permission of course, examine course syllabi for information about the planning and requirements for activities that would foster student to student interactions. You would definitely look to the syllabus for ideas on how important discussions are by how they are graded. You might also examine transcripts from discussions or chats to explore how student to student interaction takes place. Of course you would want to also interview students with experiences in online classes. Again you would design a research protocol that would help you get your participants to explain there experiences and expectations of interaction with other students in online classes. Some questions that might be included would be: 1. What is the importance of interaction with other students in online classes? 2. Why would you choose or prefer classes with a lot of student to student interaction, or the opposite? 3. Why do instructors include student to student interactions in online classes? After the data collection the researcher would use a variety of data analysis techniques including sorting, categorizing etc. This should produce an emerging understanding of the phenomena and often leads to a second round of interview or focus group protocols to further develop an understanding of the issues. Ultimately a narrative description of the phenomena is produced and commonly the researcher will do a member check by presenting this to as many of the participants in the study to ensure that the description of the narrative is consistent with the participant’s understanding of the issues. In a study such as the one being described here it would be appropriate and interesting to compare and contrast the expectations of faculty members and students about the need for or the utility of student to student interactions in online classes as well as to explore the preferences for this type of interaction in both groups. Hopefully this presentation has shown you how to approach the development of research questions and research methodology in a structured and understandable way. Please understand that in some cases the examples given here are simplified for ease of understanding. Every research study is different and must be guided by the researcher’s best understanding of what is reasonable and appropriate given the unique circumstances of the study. As we conclude this presentation you should think about what you understand to be the general process of creating research questions and developing a research methodology. Try to come to terms with what you think would be the most difficult part of this process and what you need to do as a researcher to improve your ability to make the best choices in your own research.
Pages to are hidden for
"4a"Please download to view full document