4a by keralaguest


									ETEC 543 Session 4

Developing research questions is a fundamental task in any research. The research
questions are essential to the success of your project because the research questions drive
everything that you do as they are the questions that you seek to answer through the
design and implementation of the study.

There are a number of ways of developing research questions. First and foremost is to
examine the literature base on the topic that you are researching to understand what
research questions have and have not been asked and answered. Next is to consider your
own practice in your own experience in the area, what types of questions interest you and
what types of questions do you believe need to be answered or would be interesting to
have answered. Third is to discuss your research with knowledgeable experts in the field
accept their advice and review in your development of research questions.

In all of this it's important to keep in mind the practicality of the research questions. As
you develop research questions, or at some point in the process of developing research
questions you need to make sure that you have the ability to construct an instrument or
design methodology that will actually arrive at the answers to the questions that you are

It's hard underestimate the importance of connecting your research questions to an
established theoretical base. This is especially true in quantitative studies as a quantitative
study often tests a theory or uses an established theoretical base to define approaches to
apply and test within the research design.

After a thorough review of the literature base related to the topic that you're researching
and a discussion of the topic with experts it is time to finalize your research questions.
After develop a short list of potential questions spend some time to sort them or
categorizing them and began to envision the type of research study that would be required
to answer the types of questions that you have developed. This will allow you to reject or
revise questions based on your ability to conduct the type of research you can do.
Obviously if you know from the beginning that you want to do a certain type of study this
should influence your questions. After settling on a few potential questions return to the
literature base to cross-reference in the list of questions that you've developed with the
studies that you've read related to that topic. This will hopefully help you find some ideas
for the type of research methodologies to use as well as help you locate some potential
instruments to use. Finally, consult with people who are experts in the field or who are
knowledgeable about the topic that you're researching to help you refine and ultimately
reduce your list of research questions to a manageable set.

By the time you develop the research questions you should have a clear understanding of
the type of research methodology needed to answer them. Then it is time to begin
developing instruments and/or a research protocol.
Knowing your research questions, and the type of research required to answer them will
dictate your next steps. Quantitative studies typically use survey and questionnaires,
while qualitative studies more frequently use questioning protocols for interviews, focus
groups, document analysis techniques, and observation. By this time in the process you
should be very well-versed in the literature related to the topic you're researching and
that's the place to start. As has been mentioned before, it is a good idea to look for
previous work done in the area for validated instruments or factor analyses that can help
you identify the types of factors related to the phenomenon that you're researching.

If you are able to adapt or adopt a previously used instrument you can save some steps in
the research process. But if you are unable to do so you must develop your instrument or
instruments, get expert review, and then do a pilot test to ensure that your instrument is
valid and reliable. Then you get to move on to the data collection and on to completion of
your research.

Now let’s walk through this process of research question and methodology development
in an actual study.

After teaching online classes for a while a college faculty member wants to know if
having students communicate with each other about the course material is beneficial.
Much of the literature from the early experiences with online classes include references
to the loss of social contact in online classes compared to face-to-face classes and a lot of
early work seemed to be moving in the direction of trying to replicate the face-to-face
experience in online classes. Further, it seems that if such communication is beneficial it
would make sense to plan for more of such communication in the online classes.
Discussion with an expert on evaluation indicates that grades are not an effective way of
evaluating the effect of class discussions. Digging deeper in the literature the faculty
member finds a theoretical construct called Social Presence that has shown some promise
as a way to measure interaction in online discussions and finds several studies that have
looked at student satisfaction as a way to judge the usefulness of student to student

Research questions begin emerging such as:
1. Are levels of Social Presence experienced by students in an online class associated
with student satisfaction with the online class?
2. What impacts Social Presence in online classes?
3. Does synchronous chat result in different levels of Social Presence in online classes?
4. Are higher levels of social presence related to higher levels of interaction with the
course materials?
5. Are higher levels of social presence related to higher levels of motivation?
6. What kind of faculty behaviors can influence perceptions of interaction by students in
online classes?
7. Does the type of discussion or the requirements for the discussion influence the Social
Presence perceived by the students?
After consulting with experts familiar with online learning, and then an e-mail
consultation with an expert in Social Presence the faculty member, reduced and refined
the research questions to the following set:

1. Are levels of Social Presence experienced by students in an online class associated
with student satisfaction with the online class?
2. What factors impact student interaction in online classes?
3. Is Social Presence related to student motivation in online classes?
4. Does establishing a minimum communication goal and giving a grade for the
discussion influence the level of Social Presence perceived by the students?
5. What kinds of interactions do students prefer in online classes and what are their
perceptions of the impact of different types of interactions?

Then considering the types of classes the faculty member had access to it was clear that it
would be impractical to try to set up a true experimental design because it would be
difficult to randomly select students to participate in different treatment groups. The
faculty member considered a qualitative approach to find out more about interaction,
specifically related to the second and fifth potential research questions. It seemed
appropriate but being unfamiliar with qualitative approaches a decision was made to use
a quantitative study to look at possible relationships between interaction and some other
factor in the online classes.

That led to a return to the literature base and a further refinement of the research
questions to address the factor of student satisfaction and Social Presence with the
research question being:

1. Are levels of Social Presence experienced by students in an online class associated
with student satisfaction with the online class?

Using a factor analysis found in the literature review the researcher developed a pool of
items for Social Presence with factors related to isolation/aloneness, mutual awareness,
psychological involvement, attention, empathy, mutual understanding, behavioral
engagement, mutual assistance and dependent action. After developing over 100 items
the researcher used expert review to rank the items and from the rankings the top 50
items were retained. Then the process was repeated for developing a satisfaction scale.
Armed with over 50 items for the Social Presence scale and 50 items for the satisfaction
scale the researcher sought and obtained IRB approval and did a pilot implementation in
one class. The researcher then used a coefficient alpha statistical test to establish an
estimate of internal reliability. Items that didn’t contribute to internal reliability were
rejected leaving 32 items in the Social Presence scale and 21 items in the satisfaction
scale. Using a statistical analysis program the researcher also found that there was indeed
a correlation between Social Presence and Satisfaction in the pilot test sample.

The instructor then administered the instruments to all of his classes and completed the
analysis of the data. He then wrote the results and published a paper!
Now you might wonder what would be different if the instructor in this case had wanted
to pursue a qualitative study. After the literature review and initial formulation of
potential research questions this instructor might have done just that. If you will recall,
one of the potential questions the researcher developed was:

What kinds of interactions do students prefer in online classes and what are their
perceptions of the impact of different types of interactions?

Given that interaction in online classes can be viewed as a social phenomenon a
qualitative methodology could help better define the important dimensions of the issue.
Another research question that might be interesting is:

How do faculty and student perception of the importance and usefulness of student to
student interaction in online classes compare?

This research question stems from the idea that in a quest for recreating face to face
experiences in online classes developers of online classes may be overlooking the very
real possibility that students who select online classes may have different expectations for
the need for interaction with other students. Putting the two ideas together would make an
interesting study. The next step would be to develop a better understanding of the
literature base related to the topic.

Then the instructor would develop a research protocol that would attempt to describe the
types of data collection that would be undertaken. Something that differentiates
qualitative studies from quantitative studies is that you sometimes don’t know how best
to gather data, or even from whom until you get into the process. So a certain amount of
flexibility is needed. In this case the instructor would do well to gather a group of
teachers of online classes to explore their understanding of the need for student to student
interaction. The questioning protocol would probably begin with rapport building. This
would likely be done by having the instructors tell a bit about themselves and their
experiences teaching online classes. Then some open ended questions would be used to
encourage the instructors to explain their use of and expectations for student to student
interactions in their classes. Some questions that might be used would include:

   1.   How do you structure discussions in your classes?
   2.   What do you expect students to get out of discussions in your classes?
   3.   What is the role of student to student interactions in your classes?
   4.   What do students get out of discussions in online classes?
   5.   What do students expect, or want from discussions in online classes?

Another source of data would be a document recovery method where you would, with
permission of course, examine course syllabi for information about the planning and
requirements for activities that would foster student to student interactions. You would
definitely look to the syllabus for ideas on how important discussions are by how they are
graded. You might also examine transcripts from discussions or chats to explore how
student to student interaction takes place.
Of course you would want to also interview students with experiences in online classes.
Again you would design a research protocol that would help you get your participants to
explain there experiences and expectations of interaction with other students in online
classes. Some questions that might be included would be:

   1. What is the importance of interaction with other students in online classes?
   2. Why would you choose or prefer classes with a lot of student to student
      interaction, or the opposite?
   3. Why do instructors include student to student interactions in online classes?

After the data collection the researcher would use a variety of data analysis techniques
including sorting, categorizing etc. This should produce an emerging understanding of
the phenomena and often leads to a second round of interview or focus group protocols to
further develop an understanding of the issues. Ultimately a narrative description of the
phenomena is produced and commonly the researcher will do a member check by
presenting this to as many of the participants in the study to ensure that the description of
the narrative is consistent with the participant’s understanding of the issues.

In a study such as the one being described here it would be appropriate and interesting to
compare and contrast the expectations of faculty members and students about the need
for or the utility of student to student interactions in online classes as well as to explore
the preferences for this type of interaction in both groups.

Hopefully this presentation has shown you how to approach the development of research
questions and research methodology in a structured and understandable way. Please
understand that in some cases the examples given here are simplified for ease of
understanding. Every research study is different and must be guided by the researcher’s
best understanding of what is reasonable and appropriate given the unique circumstances
of the study.

As we conclude this presentation you should think about what you understand to be the
general process of creating research questions and developing a research methodology.
Try to come to terms with what you think would be the most difficult part of this process
and what you need to do as a researcher to improve your ability to make the best choices
in your own research.

To top