EDPSY 490 CG Tony Byungho Lee Experimenting mixed methods: Pre-service teachers’ perspective on educational technology Introduction The project started from my experience with pre-service teachers in one educational technology class. I observed the pre-service teachers showing mixed response and achievement during and after the semester. This observation was inconsistent with many of the researches emphasizing the positive aspect of educational technology. I found this inconsistency interesting, and started to study the pre-service teacher’s perspective on educational technology. My main focus was, 1) to hear how - the “real voice”- the pre-service teachers think about educational technology, and 2) study why pre-service teachers have certain perspective. I used qualitative method to study these questions in this study. As I was conducting this study, I also found one of my colleagues finished a similar study in different approach. They used quantitative method to assess the pre- service teachers’ experience and perspectives of educational technology. I found that these two independent projects could be mixed in multiple ways. This experience could provide an experience to study mixed method approach, and expand both of our independent projects. Purpose The main purpose of this project is, 1) to experiment mixing method in multiple approaches with actual data, and 2) to study the pre-service teachers’ perspectives on educational technology in multiple approaches, and develop new concepts of educational technology. Data collection 1. Selecting data This project used data from two independent studies. These two studies were used for the following reasons: 1) Both studies focus on pre-service teachers’ perspective and experience on educational technology, 2) Both studies conducted survey and interview on a similar group (undergraduate students taking C&I educational technology class), and 3) Two studies use distinct research method (either quantitative study or qualitative study). 2. Independent research description 1) “A virtual help room of teacher’s technology adoption” (by Mustafa Koc & Nesrin Bakir, http://staff.ed.uiuc.edu/nbakir/edpsy387/) This ongoing study included needs assessment survey to find out the pre-service teachers’ background and needs in technology use. The survey questionnaire was developed from the technology standards of the International Society for Technology in Education (ISTE). The survey questionnaire consists of 37 questions and one open ended question. The survey was sent to the undergraduate students in one C&I educational technology class. The response rate was 8.7% (26/300). I decided to use this data because it shows general ideas of the pre-service teachers on computer, and summarizes some explicit perspectives on educational technology. I am using this data with the permission of two graduate students Mustafa Koc and Nesrin Bakir. 2) Looking through the windows of future teachers: How do pre-service teachers view educational technology? (by Tony Lee & Mei-Li Shih, http://students.ed.uiuc.edu/byunglee/class/lis450/preservice-teacher.htm) This project focuses on searching the implicit perspectives of educational technology within pre-service teachers. We focused on qualitative research methods, interviewing 6 undergraduate students who attended an educational technology class. We also interviewed 2 Teaching Assistants of the class to understand the interview in broader view. This study reflects some of the pre-service teachers’ implicit perspectives on educational technology. The study developed themes from the interview result, and attempted find explanations of the pre-service teachers’ perspective. This data is used with the permission of the other group member, Mei-Li Shih. Assumptions This project used separate data from two independent studies. There is limitation in mixing these two independent studies, but these data is used for experiment purpose. The project makes some assumptions to mix these two data studies. 1) The following projects, using three different paradigmatic frames, are independent studies. 2) The data was collected following the research design of each project. 3) Each project has different phase of data collection, and different number of groups. Experimenting by paradigmatic frames This project used two sets of data from two different projects, and mixed these data in different ways. This mixing was based on three different paradigm frames to get experience of mixing methods in different perspectives. Project 1: Post-positivism paradigm 1. Definition Positivism has a view that social research should adopt scientific method, that this method is exemplified in the work of modern physicists, and that it consists of the rigorous testing of hypotheses by means of data that take the form of quantitative measurements. 2. Project objective Technology has an important meaning in modern society. Especially educational technology is being integrated into the curriculum, and becoming an important part of the learning experience. Therefore, it is important to find the pre-service teachers’ experience & perspective on educational technology to develop an appropriate educational technology curriculum in the future. Also it is also important to evaluate the current educational technology curriculum for pre-service teachers. From this background, project 1 focuses on two objectives: 1) measuring the pre-service teachers’ experience and perspective on educational technology, and 2) evaluating the educational technology curriculum for pre-service teachers of the University. 3. Research question What is the pre-service teachers’ educational technology experience? What is the pre-service teachers’ perspective on educational technology? How effective is the University educational technology curriculum? 4. Mixed method purpose The project purpose is complementarity (seeks elaboration, enhancement, illustration, clarification of the result from one method with the results from the other method, to increase meaningfulness-Greene 1989) and triangulation (the combinations and comparisons of multiple data sources, data collection and analysis procedures, research methods, and/or inferences that occur at the end of a study). 5. Design This project was based on the positivism paradigm. The project used sequential implementation of methods. 1) Sample: Project 1 was conducted on pre-service teachers who attended an educational technology class in University of Illinois at Urbana-Champaign. The sample is diverse, but majority are in elementary education program. 2) Phase 1 (Quantitative): The project conducted a survey to study the pre-service teachers’ experience & perspective on educational technology, and evaluation on the University educational technology class. Survey was used to find the general tendency of pre-service teachers. 3) Phase 2 (Qualitative): The project conducted an interview to clarify and elaborate issues from the survey data of phase 1. 6. Findings 1) Phase 1 (Survey data) Although the participants have used computers and have received training, they don't understand the pedagogy that lies in integrating technology into their classrooms. 17 participants (65%) find technology frustrating to use when they don't receive adequate support. More than half of the participants indicate that the college does not prepare them to use technology in their teaching. 73% of the participants need training to learn how to implement computer technologies in order to enhance their students' learning. However, our data indicates that teachers are still using technologies within objectivist model of teaching and learning. Most of the participants said that they feel comfortable using computers as drill-practice and tutorial tools in their instruction; however ,they are not able to use computers for engaging students in critical and higher order thinking. 12 participants (almost 50%) refer to and base their selections on current research regarding the effectiveness of those technologies when planning how to use computer technologies for instruction. 2) Phase 2 (Interview data) Participants had experience of using computers. Most of the participants describe themselves as “heavy computer user.” Participants had generally positive attitude on educational technology. Participants stressed that educational technology has important meaning in class - elementary level. Many of the participants had idea of using educational technology in constructive learning environment. They also emphasized the teachers’ role in implementing educational technology. 7. Analysis Two sets of data were compared for complementarity and triangulation purpose. The project used sequential mixed analysis. The interview data is used to complement and triangulate the survey data. But in the comparison process, I found more inconsistency between the two sets of data. Data from phase 1 (survey data) shows that pre-service teachers have more negative perspective on educational technology, did not develop enough understanding of educational technology, and curriculum is not preparing them. But data from phase 2 (interview data) shows that many of the teachers developed certain level of understanding of educational technology, and are generally positive on educational technology. Phase 1 (survey data) -> Phase 2 (interview data) Comparison Participants don't understand the Many of the participants had idea of pedagogy that lies in integrating using educational technology in technology into their classrooms. constructive learning environment. Inconsistent Data indicates that teachers are still using technologies within objectivist model of teaching and learning. More than half of the participants indicate that the college does not Inconsistent prepare them to use technology in their teaching. 17 participants (65%) find Participants had generally positive technology frustrating to use when attitude on educational technology Inconsistent they don't receive adequate support. These inconsistencies could be explained in few different perspectives: (1) Sampling problem or bias exists. (2) Research method was inappropriate for the research question. The survey or interview question had validity problem. (3) Phase 1 (survey) was more focused on evaluation of the course, while phase 2 (interview) more emphasized the perspectives of the pre-service teachers. (4) Pre-service teachers have inner conflict within their perspectives on educational technology. This inconsistency implies that the pre-service teacher do not have a clear idea or perspective of educational technology. (5) Educational technology is over emphasized in pre-service teacher education, but pre-service teachers did not fully internalize this value of educational technology. The data comparison shows that pre-service teachers’ external value and internal value of educational technology is inconsistent. 8. Further development This project attempted to study the pre-service teachers’ experience & perspectives on educational technology, and evaluate the educational technology curriculum. But comparison of two sequential data shows that inconsistency exists in the analysis process. This inconsistency could be explained in many ways, but it is difficult to confirm any explanations within this research design. Additional phase of data, based on data of phase 1 and 2, could help the researcher find a better explanation of this inconsistency. * Phase 3 (focus group interview) Phase 1 (survey) and phase 2 (interview) data were collected from a similar sample of University of Illinois students. This project could select a focus group from this sample, and have an open discussion or interview with the focus group. This data could be helpful because pre-service could have a chance to express their reflection on the inconsistent data, and group discussion could provide a chance to develop and share common ideas of educational technology. By adding this Phase 3 to the project, this project also mixes different paradigms: Post- positivism – Interpretivism, and possibly Pragmatism. Project 2 - Interpretivism (Constructivism) + Post-positivism 1. Definition Interpretivism has a view that observation cannot be pure in the sense of altogether excluding the interests and values of individuals. Investigations must employ empathic understanding of those being studied. The paradigm supports qualitative methods. 2. Project objective Modern society is strongly influenced by technology. The field of education is also rapidly integrating technology into the curriculum. But this integration process often lacks critical reflection, and voices of the educators are often lost. How important is educational technology to educators, and why is it important? What does this imply? It is difficult to answer these questions from a single perspective. Therefore, this project focused finding the pre-service teachers’ perspective on educational technology using multiple paradigms. 3. Research question What is the pre-service teachers’ perspective on educational technology? 4. Mixed method purpose The project purpose is initiation (the discovery of paradox and contradiction, new perspectives or frameworks, the recasting of questions or results from one method with questions or results from the other method) and comprehensiveness (developing more comprehensive, more complete, more full portraits of our social world). . 5. Design This project was based on the interpretivism and post-positivist paradigm. 1) Sample: Project 2 was conducted on pre-service teachers who attended an educational technology class in University of Illinois at Urbana-Champaign. The sample is also includes teaching Assistants from different background. 2) Phase 1 (Qualitative): Face to face interview was conducted with 5 pre-service teachers and 2 Teaching Assistants to study the pre-service teachers’ perspectives on educational technology – to hear their real voice and to find different ways to understand their voice. 3) Phase 2 (Quantitative): Survey was conducted to find the general tendency of the group (pre-service teachers) and develop implication. 6. Analysis 1) Data transformation: Quantitative survey data from phase 2 is transformed into a form of qualitative data - a profile of one pre-service teacher. Suzie is an undergraduate student in elementary education program. She considers herself as “experienced” computer user. She enjoys internet surfing and sending emails. But when it comes to her educational technology class, she feels somewhat stressed. “I feel I am quite comfortable using computer, but when I think of using computer in the classroom, it is little bit different. I think I should use it as a tool for teaching, for internet search things for projects… but I am not sure if I can teach critical level of thinking to children. I think drill-and- practice stuff will also be used ….I feel that our teachers give us great ideas but I don't really know how to implement these ideas into our instruction. I know many of the general SW, like Microsoft Office. But there are also so many educational SW. I think they will be useful, but I don’t know how to use some of them. I am quite comfortable and prepared to use computer. I think it will help teachers and students, but I will have to learn … continuously” (profile developed from survey data). 2) Data comparison and correlation: The qualitative data from phase 1 (interview) and the transformed data (student profile) was compared to find any relationship. Qualitative Quantitative -> Qualitative Computer Heavy user Experienced experience Constructive “Basically it will be more like a “When I think of using learning research…or a powerpoint computer in the classroom, it is environment thing….that the only thing I will little bit different. I think I should be using for teaching…Group use it as a tool for teaching, for work… where I come up with a internet search things for topic and I will expect them to do projects… but I am not sure if I research on computer…to find out can teach critical level of thinking things … in cooperative to children. I think drill-and- work…And then I am pretty sure practice stuff will also be used …. the libraries have computers, they can all meet at the library…” Attitude on “I think it is very important. Not “I am quite comfortable and educational because it is a right thing to say or prepared to use computer. I think technology it sounds good for teachers but it it will help teachers and students, definitely accelerates instruction, but I will have to learn … it can make them easier for you” continuously” This comparison shows that these two data share similar perspective in certain level. For example, both data explains the concept of educational technology in constructive learning environment. But qualitative data describes broader and more detailed perspectives on integrating technology into constructive learning environment – some participants have surface understanding of using computer as a learning tool, while some participant gives a clear idea of technology integration. Qualitative Qualitative Quantitative -> Qualitative Field 2 years Observation only experience Constructive I do believe that’s the “Basically it will be “When I think of using learning best way to incorporate more like a research…or computer in the environment technology into a powerpoint classroom, it is little bit elementary thing….that the only different. I think I should classroom…to integrate thing I will be using for use it as a tool for into what the classroom teaching…Group work… teaching, for internet curricula is…not just a where I come up with a search things for subject as what we were topic and I will expect projects… but I am not doing… there was them to do research on sure if I can teach critical teacher who was doing a computer…to find out level of thinking to art project, or study on things … in cooperative children. I think drill- artist, and so we did work…And then I am and-practice stuff will Powerpoint presentations pretty sure the libraries also be used …. on all of their research have computers, they can they’ve done... Another all meet at the library…” project we did was, again, it was more like an artist, it was a Picasso self portrait, so they actually drew this cutest painting of themselves – they were very young kids-, and we put that together. So there were those kind of things that I integrated with teachers. But it wasn’t… it was on my own, it was on my part and on teacher’s part, and we decided what to do with that. This data comparison indicates that there are difference in understanding and implementing educational technology in constructive learning environment between different individuals. There could be different factors influencing these differences, but field experience and educational technology learning experience seems to be some of the major factors. 7. Inferences Shulman (1987) explained that “the key to distinguishing the knowledge base of teaching lies at the intersection of content and pedagogy, in the capacity of a teacher to transform the content knowledge he or she possesses into forms that are pedagogically powerful and yet adaptive to the variations in ability and background presented by the students.” This shows how the different capacity of “pedagogical content knowledge” has important meaning for teachers and pre-service teachers in developing a constructive learning environment. It also shows that both content knowledge and pedagogical knowledge have important influence in constructive learning environment. Each teacher has different capacity of this intersection between computer knowledge and classroom management knowledge. This capacity of “content pedagogical knowledge” is not given. It requires knowledge, experience, and critical thinking about what the teachers are teaching (Fenstermacher, 1978, as cited in Shulman, 1987). Classroom management Computer Knowledge, knowledge etc Process of Knowledge of internalizing educational educational technology technology Knowledge of educational technology Classroom Computer management knowledge Knowledge, etc 8. Further development This project attempted to use integrated analysis to explain the pre-service teachers’ perspective on educational technology. The analysis intended to have equal weight on Phase 1(qualitative interview) and Phase 2 (quantitative interview -> profile). But during the analysis process, the project became to focus more on the qualitative paradigm. Adding another transformation phase could help to maintain the balance of the study. * Data transformation 2 Quantitizing the qualitative data. Measure the frequency of each interview data segment, based on the themes developed from the analysis. Measuring the level of correlation with the interview data and the themes could also be a way of data transformation. Reflections 1. Challenges of mixing methods With these two mixed method experiments, I experienced some difficulties in conducting the study. First, in the mixing process, intended and unintended mixing occurs concurrently. During this mixing method experiment, I had more possibilities and flexibilities in mixing process. But sometimes the mixing in certain level influences the whole paradigm of the project. And this sometimes brings unintended result such as change of paradigms. Second, inconsistent data within mixing methods are not always developed into initiation. For example, Project 1, which started with post-positivist paradigm found inconsistent data within the study. There were some possible explanations for this inconsistent data. But I concluded the best way is to include additional data collection phase to clarify the explanations. But during this process, I felt like losing control of the overall research procedure. 2. Considering audience Many of the researches dealing with educational technology focus on quantitative method or mixed method emphasizing quantitative component. This tendency is partly related to the issue of audience. Educational technology researches need to consider administrators and funding institutions, and show the efficacy of educational technology. But this unbalanced type of research methods generates problems. They lack the critical perspective of educational technology, and also overlook the human value in implementing educational technology. I think there needs to be more researches using diverse methods that could enable the audience to get a broad picture of educational technology.
Pages to are hidden for
"Paradigmatic frames"Please download to view full document