Writing Multiple Choice Test Items Formulate useful propositions by DnYpxz6

VIEWS: 6 PAGES: 45

									Assessment: It’s what’s for dinner!

                        Laura Henriques

           Assessment
                        California State
                        University, Long Beach

                        www.csulb.edu/~lhenriqu/
                        CSTA2004.ppt
Some Appetizers
           True or False?

1. When writing multiple choice items you
   should always use “none of the above”.
2. Multiple choice items allow you to ask more
   questions in a given amount of time.
3. Multiple choice tests are less subjective than
   essay tests.
 Multiple Choice
4. When writing multiple choice items the stem, the
  part of the questions which provides background
  information and the prompt,
  A) should include more information than students
     need.
  B) should contain multiple parts, enabling you to
     ask more than one question at a time, thereby
     resulting in more data about students’
     understanding of the content.
  C) should minimize reading demands.
  D) should include negatives so that you can catch
     students who don’t read carefully.
Short Answer

5. _____________ assessment is the type of
  assessment given at the end of a unit.
6. You should have more ____________ items
  on a true/false test than ______ items.
Multiple Choice

7. When grading student work it is best to . . .
  A) grade the best student’s paper first.
  B) grade the worst student’s paper first.
  C) grade the same item for every student at the
    same time before moving on to the next
    question.
  D) all of the above.
         Overview of Today’s Talk
   Understand what makes for “good” test items
    & how to write them (This is important to
    understand this because the people who
    make high stakes tests follow these rules.)
   Use assessment results to inform instruction
   Understand the relationship between your
    objectives/goals/standards, instruction, and
    assessment
          How were you assessed?
       How do you assess your students?
   Multiple choice        Labs
   True/false             Journals
   Matching               Projects
   Fill in the blank      Reports
   Short/long essay       Presentations
    questions              Posters
   Papers                 Observations (check
                            lists)
       For what purpose do you assess?
            (or were you assessed?)
   Grades
   Rank order
   To move on to the next level

To inform instruction & better help students learn!
As we begin to think about assessment,
consider the following two forms:
   Formative Assessment- assessments
    occurring throughout the unit - sometimes
    referred to as Checking for Understanding
   Summative Assessment- the assessment
    done at the end of a unit or year
Multiple Choice Items

   Multiple choice items have a stem or
    proposition followed by several potential
    answers. Students must select the most
    appropriate/best/ideal response.
   We call the answer choices foils or
    distracters.
    Writing Multiple Choice Test Items

   Formulate useful propositions
   Translate the propositions into questions.
   Develop a concise answer.
   Develop plausible distracters.
   Introduce novelty.
   When all else fails, consider a true/false
    alternative.
What does a correctly answered multiple
choice question tell you about what
students know?


   Not necessarily a whole lot!
            Types of Multiple Choice
                     (Wiggins & McTighe)

• TRADITIONAL- used for testing content standards
  worth being familiar with and/or important to know
  and do.
  • Stem with 4 or 5 choices
• ENHANCED- also used for testing content
  standards worth being familiar with and/or important
  to know and do.
  • Stem supported by graph, chart, table or diagram with 4 or
    5 choices
       Enhanced Multiple Choice
       Taken from CA Dept. of Education (1994)
          A Sample of Science Assessment (p.100)

Leon and Lupe helped to          Leon decided to sprout his
plant a garden for their fifth   bean seeds before planting
grade science project on         them in the school garden.
                                 The picture show several
the grounds of their school.
                                 bean seeds starting to grow.
                                 The direction of the growth of
                                 the root is a response of the
                                 root to . . .

                                 a) light  b)      heat
                                 c) oxygen d)      gravity
            Types of Multiple Choice
                      (Wiggins & McTighe)

• TRADITIONAL- used for testing content standards
  worth being familiar with and/or important to know
  and do.
  • Stem with 4 or 5 choices
• ENHANCED- also used for testing content
  standards worth being familiar with and/or important
  to know and do.
  • Stem supported by graph, chart, table or diagram with 4 or
    5 choices
• JUSTIFIED- used for a deeper understanding of
  content knowledge.
  • A traditional or enhanced question that requires a written
    justification of the student’s choice.
            Justified Multiple Choice
            Taken from CA Dept. of Education (1994)
               A Sample of Science Assessment (p.104)
Directions: Please circle the best answer. Explain why you
chose your answer.

Judy wanted to find out whether certain materials were good
conductors of electricity. She replaced one of the wires in her
circuit with a piece of string.

What will happen to the bulb now?
     A. It will blink.
     B. It will not light.
     C. It will shine more brightly.
     D. It will shine less brightly.
Developing the stem:
   single idea or question
   use of negatives (underlined or capitalized)
   minimize reading (if you must have lots of reading
    consider having several questions linked to the
    reading)
   don't use lots of window dressing/excess
    baggage that is not needed to answer the
    question
Developing the options/foils/distracters:
   one correct answer is ideal, a "best“ answer is okay
   exclusive 3-5 choices is the best -- 2 is not enough,
    >6 too many (takes too long to read/sort)
   use of none of above is okay but it shouldn't be in
    each item. You don't want it to be seen as filler so be
    sure that it is the right answer some of the time.
   all of the above should be avoided because it
    doesn’t distinguish between students who know it
    really is ALL of the above from students who know
    it’s 2 of the above
           Distracters … continued
   use sensibly
   common misconceptions/misunderstandings make
    attractive wrong answers
   avoid paired opposites (mutually exclusive answers)
   common phrases and textbook language are attractive
   consistent units and language; watch that grammar
    'matches‘
   be sure answers are approximately the same length
   be careful about convergence
   Use distracters that provide you with meaningful
    information
              Paired Opposites

   In a matriarch which group rules?
       A) Kings
       B) Men
       C) Priests
       D) Women

This item was a $8,000 question on the TV
  show Who Wants to Be a Millionaire?
      Consistent Units And Language

   A cart with a mass of 10 kg is initially at rest.
    It accelerates at 4 m/s2. How far does it travel
    if 5 seconds?
    A) 100 m      The only answer that has distance units
    B) 20 m/s
    C) 10 m/s
    D) 5 s


d = ½ a t2 + vo t
           Meaningful Distracters
   A cart of mass 10 kg is traveling at a speed of
    4 m/s. How much kinetic energy does it
    have?
   A) 160 joules Forgot to multiply by 1/2
   B) 80 joules    Correct answer KE = ½ m v2

   C) 40 joules    Used formula for momentum instead of KE

                    Forgot to square velocity
   D) 20 joules
Let’s Review…..
1. When writing multiple choice items you
  should always use “none of the above”.
4. When writing multiple choice items the stem,
  the part of the questions which provides
  background information and the prompt,

A) should include more information than
  students need.
B) should contain multiple parts, enabling you
  to ask more than one question at a time,
  thereby resulting in more data about
  students’ understanding of the content.
C) should minimize reading demands.
D) should include negatives so that you can
  catch students who don’t read carefully.
              True / False Items

   A true/false item is a sentence about which
    students must decide the veracity.

   Often times students are instructed to correct
    (or make true) a false statement.
        Developing True/False Items
   Write T-F items in pairs. Be sure the content is
    parallel with one T and one F statement.
   Use specific determiners wisely and sparingly
    * absolutes: never, only, always, every, all
    * qualifiers: generally, sometimes, usually, some,
    most, few, often
   Internal comparisons help reduce ambiguity.
   Use more False than True.
   You can ask more T/F items in a given amount of
    time than you can multiple choice (less reading
    involved).
Let’s Review…..
2. Multiple choice items allow you to ask more
  questions in a given amount of time.
               Matching Items

   Matching items consist of a group of
    questions, propositions, definitions or
    problems in one column and a set of answers
    in another column. Students must match the
    question with its appropriate answer.
         Creating Matching Items
   Be sure to group the items around a common
    theme or topic
   Have the longer text be in the left column with
    shorter texted ‘answers’ on the right column
    (reduces the reading demands)
   Be sure to have the answer column in some
    logical order (alphabetical, numerical, etc.)
   Have more answers than questions
   Give instructions that answers can be used more
    than once
      Short Answer (Fill in the Blank)

   Short answer items are statements,
    definitions or propositions which are missing
    a word or phrase. Students insert the missing
    word(s) to complete the statement.
       Creating Short Answer Items
   Put the blank(s) towards the end of the
    statement.
   Be sure that there’s only one answer that
    works.
   Use as many blanks as there are words in
    the answer.
   Don’t give clues by length of blank, make all
    blanks the same length.
   Don’t give grammatical clues i.e., a ___ or an
    ____ make it a(n) _____
              Let’s Review

5. _____________ assessment is the type of
  assessment given at the end of a unit.
6. You should have more ____________ items
  on a true/false test than ______ items.
                 Grading Issues
                    (so far)
   Quick and easy! Poor handwriting is less of a
    concern.
   Students view these types of items as being
    less subjective than essay or short answer
    tests.
   Grading multiple choice, true/false, matching
    and short answer items is fast and easy.
    Creating good items, however, is time
    consuming and difficult.
       Essay Questions & Problems

   Essay questions are prompts or questions to
    which students must respond in written
    format.
   Given information, students are required to
    solve scientific problems.
   Both essay questions and problems require
    students to show their work and often result
    in the awarding of partial credit.
        Essay Questions & Problems
   Try to use a larger number of essays that require
    brief responses as opposed to fewer long essays
    (able to sample from a larger content area --> more
    reliable results)
   Do not let students select which essay or problems
    to answer.
   It is important to write ideal response before using
    the item (you will need to do it anyway, it's better to do it as
    part of item development!)
   Specify number of points or the amount of time you
    expect students to spend on essay.
    Essay Questions & Problems (cont.)
   Make physical separation between parts of
    the essay question - prompt/remind students
    about them (bullets, listed by #, space, etc.)
   Pay attention to the amount of space
    allocated for the answers.
   Word choice is fairly important - don't want
    value questions or philosophical questions,
    "discuss" is often too vague - focus students
    on what is wanted.
                       An Example
   Surprised to see a number of students
    suddenly leave the room only halfway
    through an exam, the professor followed
    them into the corridor. He discovered the
    students completing the balance of the
    questions while leaning against the walls.
    When he asked for an explanation, one
    student pointed to the third question, which
    asked "Describe Hemingway's The Old Man
    and the Sea. You may supply a drawing but
    please leave room to answer."
        http://www.educationworld.com/a_lesson/lesson275.shtml
Scoring Issues for Essays & Problems
   Grade questions in a group (all students’ #1,
    all students’ #2, etc.)
       Consistent use of partial credit
       Halo effect
   Try not to look at student names when
    grading
   Try not to let sloppy/neat papers influence
    your assessment of their content
    Scoring Rubrics or Grading Criteria

   Determine the criteria that will be evaluated
   Develop levels of performance/quality for
    each criteria
   Decide if certain areas are to be weighted
    more heavily than others

   Visit http://rubistar.4teachers.org/index.php
    for help in creating rubrics
Teacher Name: Dr. Henriques



        Science Fair Experiment : Science Fair 2004


Student Name:     ________________________________________

         CATEGORY           4                          3                         2                        1
      Scientific Question   Independently identified   Identified, with adult    Identified, with adult   Identified a question
                            a question which was       help, a question which    help, a question         that could not be
                            interesting to the         was interesting to the    which could be           tested/investigated or
                            student and which          student and which         investigated.            one that did not merit
                            could be investigated.     could be investigated.                             investigation.

      Description of        Procedures were            Procedures were           Procedures were          Procedures that were
      Procedure             outlined in a step-by-     outlined in a step-by-    outlined in a step-      outlined were seriously
                            step fashion that could    step fashion that could   by-step fashion, but     incomplete or not
                            be followed by anyone      be followed by anyone     had 1 or 2 gaps that     sequential, even after
                            without additional         without additional        require explanation      adult feedback had
                            explanations. No adult     explanations. Some        even after adult         been given.
                            help was needed to         adult help was needed     feedback had been
                            accomplish this.           to accomplish this.       given.

      Variables             Independently identified   Independently             With adult help,         Adult help needed to
                            and clearly defined        identified which          identified and clearly   identify and define
                            which variables were       variables were going to   defined which            almost all the variables.
                            going to be changed        be changed                variables were going
                            (independent variables)    (independent variables)   to be changed
                            and which were going       and which were going      (independent
                            to be measured             to be measured            variables) and which
                            (dependent variables).     (dependent variables).    were going to be
                                                       Some feedback was         measured
                                                       needed to clearly         (dependent
                                                       define the variables.     variables).
        Other Points to Consider
   The allocation of points should mirror
    instructional time (content)
   Questions ought to span Bloom’s Taxonomy
    and be of varying difficulty levels
   Types of items/tasks should mirror
    instructional strategies (process)
   Notice the amount of reading - you want to
    test science knowledge, not reading skills
   Develop scoring criteria BEFORE giving test
This should never happen!
Let’s Review…..
3. Multiple choice tests are less subjective than
  essay tests.
                    Last one!

7. When grading student work it is best to . . .
  A) grade the best student’s paper first.
  B) grade the worst student’s paper first.
  C) grade the same item for every student at the
    same time before moving on to the next question.
  D) all of the above.
                    Laura Henriques
                    California State University, Long Beach




              THANK YOU!




   This presentation is available @
www.csulb.edu/~lhenriqu/CSTA2004.ppt
w.csulb.edu/~lhenriqu/CSTA2004.ppt

								
To top