Making-Students Thinking Visible

Document Sample
Making-Students Thinking Visible Powered By Docstoc
					Lesson Study Supplement Gathering Evidence of Student Learning Bill Cerbin To investigate how students learn during a lesson instructors need to 1. design a lesson in which students’ thinking is open to observation and analysis. 2. determine what aspects of student work and activity count as evidence of learning. 3. determine how to collect evidence of student learning systematically. Making students’ thinking visible during the lesson. The object of study—student learning—should be open to observation and analysis during the lesson. Student thinking is visible or audible when they answer questions, participate in discussions, work out problems, write answers, and demonstrate skilled performance. In most classes it is not too difficult to design a lesson in which students externalize their thinking. But it is important to observe student thinking throughout the lesson not just at the end of the period. And, instructors need to make sure that the class activities evoke students’ thinking relevant to the learning goal(s) of the lesson. Even in highly interactive classes, instructors may want to include tasks or prompts during the lesson to insure access to key aspects of student learning. Ideally, the task would be a pedagogically relevant activity. For example, UWL Biology instructors sometimes use problem solving activities in which students draw a model of a particular process or idea. These are visible representations of their understanding of the topic. In large classes instructors can use a technique called “think-pairshare.” The instructor poses a thought-provoking question, and gives students a minute or two to write a response. Students then compare their responses to classmates sitting next to them. Finally, the instructor asks several students who have different views to report their responses to the entire class. These techniques makes students’ thinking visible and help students develop better understanding of the material. Evidence of student learning. Lesson study focuses on the interaction between teaching and learning. Ideally we want to monitor the way students interpret and construe the material, and how the instructional activities support student learning throughout the lesson. Simply put, we want evidence of what students “get” and how they “got” it. Evidence related to the lesson’s learning goal(s). First and foremost it is important to collect evidence of student learning related to the lesson’s goal(s). For example, the UWL Psychology Lesson Study Group designed a lesson to develop students’ ability to “think in terms of multiple factors that influence behavior.” The group wrote scenarios depicting certain types of social behavior and asked students to predict and explain how the protagonists would act. Students wrote individual responses to the scenarios before class, discussed their responses during class and wrote responses to a new set of scenarios after class. This produced information directly related to how students think in terms of multiple variables prior to, during and after the lesson. Evidence about how the lesson “works.” It is also important to collect evidence about how the instructional activities influence or affect student learning. In the psychology lesson, for example, students worked in groups for much of the period. Team members observed the student groups in order to learn how student interaction and discussion led to productive ideas, confusion, misconceptions or irrelevant activity.

Collecting evidence of student learning systematically. There are times during class when teachers would like to freeze the moment, peer into the minds of students and think of how best to respond to them. Sometimes we do freeze the moment by asking the class to write a “minute paper” or poll the class on their understanding of a concept under discussion. More often, teachers observe on the fly; we listen as students discuss a topic, circulate to hear what groups are doing, think about the meaning of a student’s question and how best to respond. It is difficult for a teacher to attend simultaneously to teaching the lesson and also attend to how various students are thinking about the subject. As one Japanese teacher commented, “A lesson is like a swiftly flowing river.” For the most part, teachers’ impressions of student learning are based on fragmented and scattered observations. In lesson study instructors collect more complete and representative samples of student learning and thinking. The key to collecting good evidence during a lesson is deciding in advance 1. who to observe (e.g., the entire class, small groups of students, individual students) 2. what to observe (student-student interactions, types of questions students ask in class, student responses to instructor questions, etc.) 3. how to observe (e.g., use a rubric to categorize responses, keep a written record of student activity, use focal questions to look for certain types of responses) Develop an observation protocol. The group should develop guidelines for observers. The protocol should include 1. an overview of the lesson so that observers know what to expect during the class period 2. guidelines for observing the lesson There is no single best method for observing students in class, but consider a combination of structured and unstructured observation strategies. A structured observation focuses on certain forms of behavior. Observers can use focal questions, rubrics, or charts during the lesson to record certain types of responses and their frequency. Unstructured observations involve observers in keeping a written record of student activity throughout the lesson (i.e., observers record what they see). We recommend that groups try to collect as much information as possible the first time the lesson is taught. If you focus narrowly on one aspect of student learning you may miss unexpected and interesting features of the activity. Videotape can add to the record of the lesson, but it is not a substitute for multiple observers in the classroom. Videotape can depict only limited portions of the activity. Moreover, it is difficult to record good audio in classrooms. In some cases you may have a visual record of student activity that is inaudible. An example of an observation protocol from the Psychology Lesson Study group is attached. The group videotaped their research lesson and had four observers in the room; two members of the lesson study team and two psychology instructors unfamiliar with the lesson. Their protocol included:  Information about the lesson—the topic, the rationale for the lesson, goals of the lesson, the lesson plan  Focal questions and suggestions about what to look for in the class.  An evaluation form with questions about how students interacted in small groups during the lesson  Questions about what worked well and did not work well and suggestions for improving the lesson.

2

SAMPLE OBSERVATION PROTOCOL
CONSTRUCT VALIDITY Introduction to the problem Construct validity, addresses the question “Does this test measure what it claims to measure?” Construct validity is theory dependent. Given the theory about some construct (i.e. a hypothetical trait such as depression or intelligence), we can make predictions about how people with high or low levels of the construct might behave. For example, given current theory of the biology of depression, we can predict that depressed individuals who take anti-depressant medications will experience lower levels of depressed after 2 to 4 weeks of taking the medications. The logic of construct validity follows the theory. If a test actually measures depression, depressed individuals should receive higher scores on the test before they take medication than they do after they have been taking the medication for 2 to 4 weeks. Students have had trouble understanding that logic. Another process for examining construct validity is to look at group differences. For example, if a test does a good job of measuring depression, a group of individuals who are diagnosed as depressed should score higher on the test than a group of individuals who are not diagnosed as depressed. When students are asked this question on an exam, many students become confused, and focus on demographic group differences while failing to address the logic of how different groups should score differently on the test. The actual exam question and some examples of student confusion are listed below. Test question: One of the processes used to examine construct validity is examining group differences. Explain the logic behind this process. (2 pts) Some classic problems: “The environment in which you are raised in has a huge effect on your performance. Sometime students that have lower socioeconomic status tend to not do as well in academics because it may be due to higher stress levels. Ethnicities also have to same effect sometimes. Your ethnicity may have biological as well as environmental differences. How we are perceived sometime has an effect on how we do in many aspect of society” “The logic behind this is that people are very different from one another. Going from culture to culture or even looking at gender, you can find many things that are different. We need to take this into consideration when working questions on a test because someone may take a question differently than another person.” The following lesson was designed to help students discover and better understand (hopefully!) the logic of construct validity. Goals of the Lesson Short-term goals/objectives: 1. To develop students’ understanding of construct validity 2. To develop students’ understanding of the logic behind the evaluation of construct validity 3. As a result of the lesson students should be able to explain several ways to assess whether a measure has acceptable construct validity, and demonstrate knowledge of the logic behind these methods. Long-term goals: 1. To develop students’ ability to analyze and evaluate instruments that measure psychological characteristics. 2. To develop “informed skepticism” toward popular claims about human characteristics depicted in the media, in work settings and everyday life. 3. To appreciate the differences between popular conceptions of human qualities and those based on careful measurement.

3

CONSTRUCT VALIDITY Lesson Plan (To be conducted by the lead instructor) Step 1: 10 minutes o o o Break students into groups and distribute group worksheets (direct students to introduce selves to group members) Describe basic task:  students will be creating “mini” tests of depression (as they define it) and proposing research studies to determine if their tests actually do a good job in measuring depression Additional directions  Take individual notes so that you have the information (you will be handing in the group worksheet) – accommodations for the “recorder”  Be sure everyone reads their definition out loud  Your group definition does not have to be “the perfect” definition, given time limit  You will have about 10 minutes to complete this step

Step 2: 10 minutes o o o o Instruct groups to finish definitions and begin writing items Items should related to their definitions/essential characteristics of a depressed person Review likert scale anchors Remind them to spend about 10 minutes to complete step

Step 3: 30 to 40 minutes o o Instruct groups to finish writing items and begin to propose research studies to determine if their tests actually measures depression Remind them:  Unlimited resources – funding, people from a variety of settings (e.g. clinics, university, whatever), ages, diagnoses, etc.  Think about your definitions of depression, what you know about depression as you think about what results you might expect Monitor groups progress; if groups seem to be going well, let them continue If groups struggling, off track call groups attention  Pick some examples to review

o o

Step 4: 15 minutes o o o Call for groups attention and provide instructions for final method Distribute additional measures Think about how you might use these tests to provide evidence of the ability of your test to measure depression well  Hint: think about the statistical methods we have covered in class to this point

Step 5: 10 minutes o o o After all groups have completed the task, if time, discuss some of the definitions and methods Have group members describe as worksheets are displayed on visualizer Save 5 minutes at end (if possible) for writing exercise:  What was the most difficult part of this exercise?  What was the most important thing you learned from today’s lesson?  What is still confusing?

4

CONSTRUCT VALIDITY Observation Protocol The purpose of having many instructors observe the class is to gather as much information about the process of the lesson as possible. Your primary task is to observe how the students conduct the lesson and make some conclusions about how well the LESSON worked. In other words, please note behaviors of the students and the benefits/difficulties of the lesson, NOT the behaviors of the instructor ! You will be observing one group of approximately 5 students. Given the goal of helping students understand the logic underlying construct validity, please look for evidence/examples that students are tying their understanding of depression to how people should score on their test, if, indeed their test does a good job at measuring depression. Please do take notes on your group’s behavior. In addition to noting any good and poor examples of their ability to think about the logic of construct validity, please also note such things as o How the group developed their definition of depression. Did they integrate their individual definitions? Did they simply string their individual definitions together? Something else? o Any evidence that the students seemed interested and/or engaged in the lesson o Any derailing of the process o Any problems in the group dynamics (dominating members, quiet members, etc.) o Any problems understanding the directions o Anything else you think is substantial! Please do not make comments to your group. I.e. do not correct any misconceptions, clarify instructions, etc.

5

CONSTRUCT VALIDITY Observer Reactions to the Lesson Now that you have observed the lesson, please answer the following questions. Totally Disagree 1. All members participated in the process 2. The group was able to stay on track with the lesson (i.e. did not derail, discussing irrelevant information) 3. The group seemed confused about the technical processes of the lesson 4. The group seemed confused about the concepts the lesson was addressing 5. The group seemed to understand the concept of construct validity 6. The group seemed to understand the logic of construct validity 1 1 2 2 3 3 4 4 5 5 6 6 Totally Agree 7 7

1

2

3

4

5

6

7

1

2

3

4

5

6

7

1

2

3

4

5

6

7

1

2

3

4

5

6

7

7. Given your observations, what aspects of the lesson need to be changed? How could the lesson be improved?

8. What aspects of the lesson should remain the same? What worked well?

6


				
DOCUMENT INFO
Shared By:
Stats:
views:57
posted:11/27/2009
language:English
pages:6
Description: Making-Students Thinking Visible