Assessing cognitive outcomes. Taken from: Classroom Assessment Techniques: A Handbook for College Teachers (2nd Edition) Thomas Angelo and Patricia Cross, Jossey Bass, San Francisco, 1993 CATS Classroom assessment techniques What are they? Informal or formal Ungraded or graded Paper/pencil or online ―Through close observation of students in the process of learning, the collection of frequent feedback on students’ learning, and the design of modest classroom experiments, classroom teachers can learn much about how students learn, and more specifically, how students respond to particular teaching approaches. CATS help individual faculty members obtain useful feedback on what, how much, and how well their students are learning. Faculty can then use this information to refocus their teaching to help students make their learning more efficient and effective.‖ --Cross and Angelo, CATS, 1993, p. 3 Ask yourself what you want to assess -- what examples did you bring along? Prior knowledge, recall, comprehension Background knowledge probe Focused listing Pro and con grid Empty outline Memory matrix Minute paper Content, form, and function outlines Invented dialogues Concept maps Analysis Synthesis, creativity Problem solving Application and performance Categorizing grid Defining features matrix One-sentence summary Word journal Problem recognition tasks What’s the principle? Directed paraphrasing Applications cards Studentgenerated test questions Human tableau or class modeling Documented problem solutions Muddiest point Background knowledge probe Particularly useful in sequence courses Also helps you know where to begin! Focuses students on what’s important (list of concepts or specific details) Ask at least one question everyone should be able to answer. To assess learning, do as a pre-post measure. Memory matrix (biology example) Structure Functions Enzymes Mouth Esophagus Stomach Small intestine Large intestine Pancreas Liver Gall bladder Procedure for memory matrix Always try it yourself first. Revise as needed. Make the cells large, encouraging students to write more than one word or phrase in each cell. Ask for at least 3. Group or paired work works well with this. The more cells, the greater the difficulty. To assess: see what they know well, and what they need to work on. Look for patterns. Tally correct or incorrect responses. Categorizing grid Category 1 Category 2 Category 3 Example 1 Example 2 Example 1 Example 2 Example 1 Example 2 Be sure categories don’t overlap. Look for mis-categorized items, missing items, patterns of error Use a simple grading system (0 -1 – 2, or check +/check -) Defining features matrix Works well for helping student distinguish between closely related or seemingly similar concepts or items (e.g., biology, geography, chemistry) Features Investigates symbolic features of dreams Freudian Yes Behaviorist No Select 2 or 3 most important concepts Sketch out your own matrix first Give a time limit Be sure a “yes” “no” response is reasonable. Look for patterns of missing features Explain why you’re doing this Pro and con grid 1. 2. 3. Select a decision, judgment, or dilemma. Write a prompt that will elicit thoughtful pros and cons. Identify the number of items you want in each list and whether phrases or sentences are necessary Example: ―For some time now, certain critics within and outside of the Roman Catholic Church have argued that priests should be allowed to marry and have children. In responding to this assessment exercise, consider the pros and cons of abolishing the requirement of celibacy from the perspective of the church as an organization. List about five important potential advantages and an equal number of disadvantages.‖ 1. 2. 3. Which points are most frequently made? Are any missing? Extraneous? How balanced are the sides of the grid? Content, form, and function outlines Content (What?) Form (How?) Function (Why?) ―CAT‖ title Phrase Capture the essence of the technique and reader’s attention Help readers decide whether they can use the technique Explain the information the CAT will elicit Provide vignettes that illustrate how instructors use the technique Connect the technique to faculty’s individual teaching goals Description Expository prose Purpose Expository prose Examples Narrative prose Related teaching goals List Problem recognition tasks Provide students with a few examples of common problem types. Students must recognize and identify the particular type of problem each example represents. To expand on this idea, you can ask students to 1. 2. 3. 4. Work in pairs or groups Develop parallel examples Justify responses using facts/evidence Identify the clues an expert would use in developing a solution What’s the principle? Principles for good practices in education 1. Encourage contact between students and faculty 2. Give prompt feedback on performance 3. Emphasize time on task 4. Encourage active learning a. Use think/pair/share during lecture b. Keep scheduled office hours c. Return quizzes following day d. Carefully plan classroom time l l l l l Identify basic principles you expect students to learn in your course. Find or create sample problems or short examples that illustrate each principle. (Easiest if you use one-on-one correspondence.) Create a form that includes a listing of the principles as well as the examples. Try it out on a colleague first! When it works, give to students. Documented problem solutions Great for math, music theory, logic, linguistics – fields requiring specific strategies for solving problems. The principle: When they can verbalize the strategy accurately, they understand it. 1. 2. 3. 4. Assign 2 -3 problems. Ask students to fold a piece of paper in half. On the left, they write the steps for solving the problem. On the right, they write a brief note that justifies each step. Do this yourself first, before assigning the problems. Some problems may be too long or complex to work well for students. Skim through the results. Where are students making errors? Are errors due to misunderstanding the process, or mis-ordering or skipping steps? What reinforcement/extra work is needed, and where? Directed paraphrasing examples 1. In one or two sentences, paraphrase what you have learned about hospice care to inform a dying, but still lucid, patient of its possible advantages over hospital or home care. 2. Imagine you are the city’s deputy police commissioner in charge of community relations and public affairs. For a two-minute presentation at a meeting of the police officers’ union, paraphrase the arguments in favor of creating a civilian review board. Then, for an equally short argument at the next public meeting of the city council, paraphrase the arguments against creating a civilian review board. tips Do it yourself before assigning to students Tell them who the audience is and the purpose for doing it. Provide a word/time limit. Explain how you will grade or respond to this exercise. Applications cards 1. 2. After students have heard or read about an important principle, generalization, theory, or procedure, hand out an index card Ask students to write at least one possible, real-word application for what they just learned on a card. Use separate cards if asking for more than one application. Examples: (Econ) Gresham’s law basically states that ―good money drives out bad.‖ Give at least one contemporary application of Gresham’s law to something other than money. (Physics) In his Principia, Newton set forth his Third Law, the heart of which is ―To every action there is always opposed an equal reaction.‖ Give three applications of Newton’s Third Law to everyday life around the house. (Psych) Psychologists have long noted the effects of ―primacy‖ and ―recency‖ on recall of information. These effects have some implications for classroom teaching and learning. Suggest one or two applications of the implications for teachers using the lecture method. 3. 4. Label cards G, A, M, N (Great, Acceptable, Marginal, Not acceptable) Sort cards into piles. Share 3 good examples and one or two poor examples with students Student-generated test questions To make this strategy work, you have to be willing to include (revised) student questions in your tests, or at least to let students know what kinds of questions will be included. Do this assignment 2-3 weeks before a major exam to allow time for feedback and adjustments in teaching/studying. 1. 2. Decide what types of questions on what specific topics you want on the exam. Write this information down for students. Decide how many questions they should generate. One of two questions of any type are usually enough, especially if you want students to supply answers. Explain to students how their questions will be used. Tell them when to expect feedback, and why this will help them do better on the test. Once collected, look for patterns – which topics are over-/under-represented? Well-written/poorly-written.) At what level of difficulty are questions written? (Use Bloom’s Taxonomy.) A checklist could make this task easy. Select examples to provide feedback to students. Decide which questions to share as study aids. A handout could make good test review material. Students might work in pairs or groups to develop/find answers to questions. 3. 4. 5. 6. Human tableau or class modeling -- or, apply what you just learned by performing it! 1. 2. 3. Select a process or image that has particular teaching/learning importance that can be better understood through performance, as opposed to traditional methods. Be sure your goals for student learning are clear. Develop a rubric or checklist for the project prior to assigning it. It’s easiest to select a process that can be portrayed without props. An example from an art course: The instructor wants students to demonstrate their understanding of how perspective developed in European painting. Small groups of students were given the task of ―posing‖ the same scene—the Madonna and child surrounded by the three ―wise men‖—as Human Tableaux. (They used a doll for the child.) They had to develop tableaux for the Late Roman, Late Byzantine, and Quattrocento periods. The instructor took photos, which were transferred to slides, and she and the class critiqued them together.