Docstoc

Slide 1 - American Academy of Ophthalmology

Document Sample
Slide 1 - American Academy of Ophthalmology Powered By Docstoc
					Developing Multiple Choice
                 Questions
             David K. Coats, MD
           OKAP Committee Chair
     Texas Children’s Hospital in Houston
   The Ophthalmic Knowledge Assessment Program
    (OKAP) is an in-training ophthalmology examination
    offered annually by the American Academy of
    Ophthalmology for ophthalmology residency
    programs in the United States and Canada.
   Dr. Coats has disclosed the following financial
    interests:
David J. Wilson, MD
 WQE Chairman
              Stacey Samuels
Program Manager, Assessment and Evaluation
   Clinical Education Division of the AAO
 Theintention of this tutorial is to
 educate item writers on acceptable
 formats, good technique, and what to
 avoid when writing questions.
              Which of the following phenomena increases as a
    STEM
              pinhole aperture gets smaller?



   Correct    A.   Diffraction
 Response

              B.   Chromic aberration

Distractors
              C.   Polarization


              D.   Spherical aberration
 Category   1: Memory or Knowledge Recall

 Category   2: Interpretive

        3: Decision Making and Clinical
 Category
 Management



 Examples of each type of question are provided in the
          training copy of this presentation
 Memory or Knowledge Recall
A Hudson-Stahli line represents iron
 deposition in which of the following
 corneal layers?
    A. Bowman’s membrane
    B. Descemet’s membrane
    C. Epithelium
    D. Stroma
INTERPRETIVE
The lesion shown in Figure 1 is most consistent
with a diagnosis of:
                       A. chronic viral infection.
                       B. hemangioma

                       C. Kaposi sarcoma.

                       D. recurrent conjunctival
                          dysplasia.
   INTERPRETIVE
    Three months after sustaining blunt ocular trauma and
      associated hypotony, a patient experiences sudden
      onset of pain with corneal edema. Which of the
      following is the most likely cause of this condition?
    A. Corneal decompensation from acute endothelial
      dysfunction
    B. Late bleeding and hyphema from injured iris
      vessels
    C. Neovascular glaucoma
    D. Spontaneous closure of an occult cyclodialysis
      cleft
Decision Making and Clinical Management

Three weeks ago, the parents of a 2½-year-old girl
noticed intermittent esotropia in the child's right
eye. Examination reveals 20 prism diopters of
esotropia at 6 ⅓ m. Which of the following is the
most appropriate initial therapy?
      A. Alternate occlusion
      B. Base-out, 10 PD Fresnel prisms on plano
      spectacles
      C. Recession of the medial rectus muscles
      D. Spectacles for correction of hyperopia
 Decision Making and Clinical Management (can
  include visuals)
A 24-year-old woman experiences sudden loss of vision in her
  right eye with a visual acuity of 20/400. She has a right
  afferent pupillary defect, optic disc swelling, and a central
  scotoma. Her left eye is normal. A complete neurological
  history and physical examination reveal no abnormalities.
  Which of the following is the most appropriate next step?
      A. Administer intravenous methylprednisolone alone
      B. Administer intravenous methylprednisolone, followed by
        oral prednisone
      C. Perform CT scan of the orbit
      D. Perform MRI of the brain
 The ABO’s secure website contains a
  visual library of images available to its
  item writers
 You may upload images that will become
  part of the question directly onto the
  website
  • Be sure to include a diagnosis and a description
    for uploaded visuals
  • Once submitted, visuals are copyrighted and
    should not be used or have been previously
    used in a publication or lecture materials.
 You  will be asked to complete a copyright. This
  allows the AAO unlimited use of the images in
  its educational programs.
 Ifyou have concerns about unlimited
  copyright, work with your staff liaison to
  explore alternatives.
 You  can post your images directly to a secure
  area of the AAO’s ftp site. Staff will then catalog
  the image as appropriate.
  Due to the new “patient privacy” and HIPAA
  regulations, it is essential that you have a
  signed release form (see Release Form) if you
  are submitting published material or
  photographs in which patients may be
  identifiable. This applies to photographs taken
  after April 15, 2003.
 Release forms from your particular institution
  also are acceptable.
 Images  such as photographs, diagrams, or
 visual field results can enhance the quality
 of your questions
 The ABO and AAO encourage use of images
 when they contribute required clinical
 information necessary to answer the
 question
 Images should clearly illustrate only the
 condition or finding described
Two examples with images are provided in the training
            copy of this presentation.
The 60-year-old woman with the findings shown has pericentral
scotoma in the right eye 2 months after combined cataract and
glaucoma surgery. Which of the following is the most likely
diagnosis?
A. Age-related macular degeneration
B. Cytomegalovirus retinitis
C. Histoplasmosis
D. Photic maculopathy




 The visual provided is an accurate representation of the
  entity helps to improve the validity of the examination.
 The 60-year-old woman with the findings shown has
 pericentral scotoma in the right eye 2 months after
 combined cataract and glaucoma surgery. Which of
 the following is the most likely diagnosis?
 A.   Age-related macular degeneration
 B.   Cytomegalovirus retinitis
 C.   Histoplasmosis
 D.   Photic maculopathy


While the stem is well written and the distractors are plausible,
    the photograph is not a good representation of photic
                         maculopathy.
   The stem or question must provide a sufficiently
    clear and focused description of the problem/case
    scenario for a candidate to make a decision.
   The stem should include all clinical information
    necessary for the candidate to make an
    unequivocal choice about the correct response.
   The information should assure that there is only
    one, clear correct response.
   No excess verbiage irrelevant to the question.
   a knowledgeable candidate should be able to
    answer the question before reading the responses
   Category I questions should be clearly stated and
    have clinical significance
   Category II and III questions should consist of:
       A description of the clinical findings, patient
        symptoms, or other specific data that provides
        background for the question (clinical scenario),
        and
       A question that details specifically what is
        being asked
   As you develop the question, you have a clear idea
    of exactly what the correct response will be.
A  clear question should be presented at
  the end of a clinical scenario
 Good lead-in questions include:

     • Which of the following is the most likely
       diagnosis, cause, complication?
     • Which of the following procedures, tests,
       treatments is most appropriate?

 Good Category 1, 2, and 3 stems are provided in the training
                 copy of this presentation
   Which of the following is an early feature
    of choroideremia (progressive
    tapetochoroidal dystrophy)?

   Which of the following treatments is most
    likely to minimize the risk of epithelial
    keratitis in a patient with herpes simplex
    infection?
   The histopathology report on a specimen of
    posterior retina indicates thinning of the nerve fiber
    layer and loss of ganglion cells, but the inner
    nuclear layer, outer nuclear layer, and
    photoreceptors are intact. What is the most likely
    diagnosis?
   Eosinophils are found in conjunctival scrapings from
    a patient who has symblepharon. What is the most
    likely diagnosis?
   After 1% atropine drops are instilled into the
    eyes of a blue-eyed, 3-year-old boy, he becomes
    hyperpyretic, disoriented, and hyperactive.
    Which treatment options are most appropriate?
   A patient who undergoes complicated
    intraocular surgery with an open lens capsule
    develops secondary open-angle glaucoma
    that requires a combination of a beta blocking
    agents, topical carbonic anhydrase inhibitors,
    and latanoprost to control. Two months after
    surgery, vision has decreased to 20/200.
    Which of the following is the most likely
    cause of the decrease in vision?
A  70-year old with open angle glaucoma in
 both eyes presents with pain, redness, and
 decreased vision in his right eye for 2 days. He
 is using no medication in that eye. Two years
 ago a trabeculectomy was performend on the
 right eye, with mitomycin C. His best corrected
 visual acuity is 20/200, and slit lamp exam
 shows 3+ injection with a seidel positive bleb
 and yellowish discharge. There is a 1+ cell and
 flare in the interior chamber. What is the most
 important condition to rule out?
 Trivia, esoteric information
 clinically irrelevant information

 Poor gauge of knowledge or clinical
  competence
 The stem is an open-ended statement

 Unfocused    (True/False or Global)
 Negatively   worded
   Examples of Poor Question Stems are
     provided in the training copy of this
                presentation
Which of the following medications can cause
 palinopsia?
  A. Amitriptyline
  B. Doxycycline
  C. Gabapentin
  D. Trazodone


               Problem: TOO ESOTERIC
 Extremely difficult for candidates to answer correctly.
A 5-year-old boy has acute and chronic polyarthritis,
recurrent fever, pericarditis, rash, and hepatosplenomegaly.
What is the probability that iridocyclitis will develop?
A. Negligible
B. 25%
C. 50%
D. 90%



          Problem: NOT CLINICALLY RELEVANT
The chances that the child will develop iridocyclitis are not
  clinically relevant. Would knowing the % change the
           candidate’s treatment of this patient?
Which of the following best describes the Landolt ring?
  A. A device for stabilizing the lens capsule during cataract
   surgery
  B. A measuring tool for determining the corneal diameter
  C. An optical phenomenon that results from diffraction
  D. An optotype for measuring visual acuity
       Problem: POOR GAUGE OF KNOWLEDGE OR CLINICAL
                            COMPETENCE
 The question would be better if it focused on the use of the Landolt
  ring Instead of asking for the description of the Landolt ring. This
               question could be improved by asking:
  The Landolt ring is best used in the measurement of which of the
                      following visual functions?
1.   The interference property of light:
2.   Statin use:
3.   Viruses:
4.   Lung cancer is:
5.   Which of the following statements is likely regarding XXX?
6.   Which best describes a common occurrence in a patient
     who XXX?
7.   What would be a common practice regarding XXX?

            Problem: THESE STEMS ARE OPEN-ENDED
Whether open ended or in question format , unfocused stems do not
     provide candidates with enough information to formulate a
   response before reading the choices. A well-constructed stem
 allows knowledgeable candidates to look for the correct response.
 Which of the following statements regarding lattice
 degeneration is true?
   A. It is equally common in emmetropes and myopes.
   B. It is found in 20-30 % of all eyes that present
   with a rhegmatogenous retinal detachment.
   C. It frequently is associated with peripheral
   retinoschisis in children.
   D. It is nearly always bilateral.

                      Problem: True/False
The answers are a list of statements following an unfocussed
stem. A, C, and D may be related to frequency, so rewriting the
            stem to ask about frequency might help.
 1.   Which of the statements about the reflex of the
      retinoscope is true?
 2.   Which of the statements about a patient with
      mixed astigmatism is correct?
 3.   Identify which statement is accurate with regard to
      closed-angel glaucoma:
 4.   Compared to essential blepharospasm, hemifacial
      spasm usually:

Problem: UNFOCUSED (TRUE/FALSE OR GLOBAL)
If the correct response and distractors are a series of
true/false statements, the stem most likely is unfocused.
Epithelial keratitis is less likely to occur after a herpes
simplex infection if the active infection is treated with:
  A. minimal wiping debridement and topical
  trifluridine drops.
  B. topical trifluridine drops.
  C. topical trifluridine drops and vidarabine ointment.
  D. none of the above.

                Problem: Negatively-worded
“Less likely” is an example of negative question writing.
Additionally, there is NO correct response. In reality, A, B,
and C all are correct treatments. This type of question is
                 confusing for candidates.
 Eachanswer choice should rarely exceed more
 than 1 line
       in logical, alphabetical, or numerical
 Listed
 order unless doing so gives away the correct
 answer
 Have both the correct response and the
 distractors balanced in their plausibility, length
 and amount of detail
 Allresponses are parallel, e.g., all tests, all
 treatments
 Wrong  answers should reflect faulty reasoning
 or misconceptions. There should only be 1
 defensible answer. Content that is controversial
 or that varies regionally should be avoided in
 order to eliminate the possibility of more than
 one correct response.
          answers should be attractive to at
 Incorrect
 least some candidates
Question Stem: What is the most
 appropriate initial management of
 intraoperative bleeding at the site of a
 neodymium: yttrium aluminum garnet
 (Nd:YAG) laser iridotomy?
A. compression of the eye with the laser
    iridotomy lens
B. pharmacologic pupillary dilation and
    cycloplegia
C. cauterization of the vessel with the
    Nd:YAG laser
D. bed rest and topical corticosteroid
    eyedrops
     of the Above” /“None of the Above” are
 “All
 unacceptable to the ABO and the AAO
 attemptto test too many areas at once, are
 confusing, and when a candidates gets the
 question wrong, it’s difficult to determine
 which aspect of the question they did not
 know.


 Examples of poor distractors are provided in training
              copy of this presentation
Ocular conditions associated with
hypertension included which of the following?
A. Retinal vein occlusion
B. Ischemic optic neuropathy
C. Glaucoma
D. All of the above

Problem: In addition to having an unfocused stem, “All of
 the above” and “None of the Above” are unacceptable
answer choices. Similarly, “Part I only” and “Part II and III
       only” types of answers are unacceptable.
Which of the following characteristics distinguishes
congenital fibrosis
syndrome from double elevator palsy?
A. Convergence on attempted elevation
B. Hypotropia in primary gaze increasing on upgaze
C. Positive forced ductions to elevation
D. Ptosis or pseudoptosis

Problem: There is more than 1 defensible answer. Although
 choice A is marked as the preferred response, answer D is
     arguable (ptosis or pseudoptosis is also correct.)
 What  should treatment of acute
  dacryocystitis ALWAYS involve?
  A. Systemic antibiotics
  B. Orbital imaging
  C. Immediate nasolacrimal duct probing
  D. Lacrimal sac massage
Problem: Answers choices are not homogenous or parallel.
   All the answer choices should be tests, symptoms, or
  differential diagnoses, for example. Unparallel answer
   choices are a clue that the stem needs to be focused.
   Threshold retinopathy of prematurity (ROP) disease is defined
    as which one of the following?
    A. Five contiguous clock hours of Stage 3 or 8 total clock
    hours of Stage 3 in Zone I or II with plus disease
    B. Five contiguous clock hours of Stage 2 or 8 total clock hours
    of Stage 2 in Zone I or II with no plus disease
    C. Five contiguous clock hours of Stage 3 or 8 total clock hours
    of Stage 3 in Zone II or III with plus disease
    D. Five contiguous clock hours of Stage 3 or 8 contiguous clock
    hours of Stage 3 in Zone I or II
                Problem: Answer choices are too long.
 Answer choices should rarely exceed 1 line. If the answers are too
long, it takes a disproportionate amount of time for the candidate to
  answer a question even in areas which they are knowledgeable.
Which of the following pairs best describes the outflow of aqueous
humor?




         Problem: Attempts to test too many areas
 The question is complex in format and difficult to answer.
Moreover, if candidates get the question wrong, it’s difficult to
 determine which aspect of the question they did not know.
 Implausible    or silly distractors can be easily
  dismissed, increasing ability to guess the
  correct response
 the same words appear in both the stem and
  one answer choice.
 Incorrect answers can be too obvious (e.g.
  ‘smoke more cigarettes’)
 One answer is longer than the others
 One answer is grammatically different
 Mutually exclusive responses help the
  candidate, because if you know one response
  is false, the other must be true.
 Ambiguous    or indefinite terms like might, may,
  sometimes and absolute terms like never, always
    Scenarios almost always “might” or “may” be
    true. Especially in the absence of qualifiers in
    other answers, vague terms are a clue to the
    correct response. The reverse is true of absolute
    terms, where scenarios can be ruled out as
    incorrect since they almost always may be true.


Examples of questions that have clues to the correct response are
       provided in the training copy of this presentation.
Which of the following lesions is most likely to
cause a retinal hole or tear leading to retinal
detachment?
A. Cyst in the pars plana
B. Cystic retinal tuft
C. Enclosed oral bay
D. Peripheral cystoid degeneration


Problem: the same words appear in both the stem and
                one answer choice.
Which of the following is an appropriate alternative to the
use of cyanoacrylate glues in the management of
corneal perforations?
A. Aminocaproic acid
B. Autologous plasmin
C. Erythropoietin
D. Fibrinogen and thrombin glue

 Problem: The incorrect answer choices are too obvious.
While D is not a proven “appropriate alternative”, the other
 distractors are so obviously incorrect that they lead the
            candidate to the correct response.
Pupillary block and iris bombé:
 A. occur in reverse in pseudoexfoliation syndrome.
 B. are absolute in primary angle closure glaucoma.
 C. are common in neovascular glaucoma.
 D. may be compartmentalized in pseudophakic eyes
 due to localized adhesions between the intraocular
 lens surface and iris pigment epithelium, not all of
 which will respond to a single iridectomy.

 Problems: longer, grammatically different, vague terms.
In addition to an unfocussed stem, the correct answer is
 much longer and grammatically different than the other
 answers. Further, it contains the vague term “may be”.
 Which one of the following is true about acquired
 limbal stem cell deficiency?
 A. It can be caused by preservatives in topical
    medications.
 B. It is always diffuse and not focal.
 C. It can never be treated by limbal stem cell
    transplantation.
 D. It is never seen in graft-versus-host disease.

        Problem: absolute terms “always” and “never”
In addition to a true/false stem, the incorrect answer choices
   all contain absolute terms. This allows the candidate to
                eliminate answers B, C, and D.
  In the treatment of herpes simplex stromal keratitis
   without epithelial keratitis, oral acyclovir:
   A. alone is more effective than topical
        corticosteroids alone.
   B. provides no beneficial effect when added to
        topical corticosteroids.
   C. speeds resolution of keratitis when added to
        topical corticosteroids.
   D. is less effective than topical trifluridine.

    Problems: Mutually excusive answer choices B and C.
In addition to an unfocused stem, B and C are mutually
exclusive. If a candidate knows one is false, the other must be true.
The globe shown was enucleated 6 weeks after cataract
extraction, which was followed by rapid growth of a
subconjunctival mass at one end of the cataract incision.
Higher magnification of the cataract incision is shown. Which
of the following is the most likely diagnosis?
A. Endophthalmitis
B. Metastatic tumor from an extraocular site
C. A primary conjunctival tumor invading the eye
D. Previously unrecognized
  iris and ciliary body
  melanoma extending
  through the cataract
  wound
 The globe shown was enucleated 6 weeks after
 cataract ….
 A. Endophthalmitis
 B. Metastatic tumor from an extraocular site
 C. A primary conjunctival tumor invading the eye
 D. Previously unrecognized iris and ciliary body
 melanoma extending through the cataract wound

Problem: Correct response and distractors should be
  approximately the same length and contain the same
  detail
 have   repeated phrases in the distractors

 contain   unnecessary information

    unnecessary medical jargon, slang,
 use
 eponyms, and/or abbreviations


 Examples and exercises to help you improve questions
  are provided in the training copy of this presentation
Which of the following is most likely to be associated
with crystalline lens dislocation and/or subluxation?
A. Aniridia
B. Essential iris atrophy
C. Chandler’s syndrome
D. Cogan-Reese syndrome


               Problem: C and D are eponyms.
     In order to include common eponyms, reformat to :
• iridocorneal endothelial syndrome (Chandler’s)
• iris nevus syndrome (Cogan-Reese)
A patient undergoes a difficult phacoemulsification. An
opening is made in the posterior capsule, necessitating a
limited anterior vitrectomy and placement of the intraocular
lens implant in the sulcus. The postoperative vision is 20/40,
but a secondary open-angle glaucoma develops that requires
a combination of a beta blocking agent, topical carbonic
anhydrase inhibitors, and latanoprost to control the
intraocular pressure. Two months after surgery, vision has
decreased to 20/200. What is the most appropriate next step?

   Problem: This stem contains unnecessary information.
  Rewrite this stem and compare your effort to the up-side
         down re-write provided on the next slide.
Which of the following therapies is most likely to be
associated with pseudopemphigoid?

A.   Topical administration of cromolyn sodium
B.   Topical administration of gentamicin
C.   Topical administration of timolol
D.   Topical administration of trifluridine


       Problem: This question contains repeated phrases.
     Rewrite this question. Compare your effort to the up-side
                   down re-write provided there.
Negative       question stems
    ask what is false, not, least, or the exception
     to the rule.
    do not perform well. Statistics show that
     knowledgeable test takers perform poorly
     on negatively worded questions
    are not useful for assessment and act much
     like “trick questions”
Negative      question stems
  The   ABO does not use negative questions
  TheAAO is slowly eliminating negative
  questions from its programs and will not
  accept new, negative items
                      an acceptable use of a
  “Contraindicated” is
  negative only when testing clearly
  contraindicated medications or procedures.
 Questions   should not be intentionally
    misleading
    “Trick” questions are not good. They
    tend to test savvy or reading skills
    instead of clinical competence



Examples of unacceptable questions follow this slide in the
            training copy of this presentation.
Which of the following agents is LEAST
effective in the treatment of ocular disease?
  A. Thiabendazole
  B. Clindamycin
  C. Ganciclovir
  D. Acyclovir
When planning injection of an eyelid hemangioma in an
infant, which of the following systemic consequences
must be considered?
        A. Adrenal insufficiency
        B. Central retinal artery occlusion
        C. Fat necrosis
        D. Skin necrosis


    This is a trick question because B also is correct,
     although not “systemic” as stated in the stem.
Inthe following examples,
 determine whether the format
 shown is acceptable or
 unacceptable for ABO
 examinations
Which of the following is true regarding the
 oculomotor nucleus?


□ Acceptable
□ Unacceptable

  This question does not give candidates any idea what the correct
  response might be until all of the choices are read. Unacceptable.
A persistent, clear fluid encountered when
removing bone superior to the medial canthal
tendon during a dacryocystorhinostomy most
likely would result from inadvertent penetration
of which of the following structures?
  □ Acceptable
  □ Unacceptable
This question provides sufficient information for a candidate to formulate a
response and quickly find it among the choices. Acceptable.
 Which of the following histocompatability
 leukocyte antigens is least likely to be
 associated with retinal vasculitis of noninfectious
 origin?

    □ Acceptable
    □ Unacceptable


This question asks candidates to identify what is NOT correct. Unacceptable.
Which of the following findings is most likely
to be associated with meibomian gland
dysfunction?

 □ Acceptable
 □ Unacceptable


  This question directs candidates to look for the MOST likely
                      finding. Acceptable.
As compared to a normal rigid contact
lens, a translating bifocal rigid contact lens
does not have:

 □ Acceptable
 □ Unacceptable

    This question is both unfocused AND
     negatively worded. Unacceptable.
 Examples  of items that are well-written.
 e.g., have good question stems and
 answer choices, are provided in the
 hard-copy of this hand-out after this slide.
  Prophylactic  laser photocoagulation of
   asymptomatic lattice degeneration is most
   indicated for a patient with which of the following
   findings?
    A. High myopia

    B. Preoperative cataract
    C. Pseudophakia
    D. Retinal detachment in fellow eye
Focused question that asks candidates to recognize the
significance of retinal detachment in lattice degeneration
 Which of the following is the most likely
  diagnosis for the condition shown?
A. Chloroquine retinopathy
B. Cone dystrophy
C. Dominant drusen
D. Fundus flavimaculatus

  Interpretive question that tests candidate’s ability to
           recognize and diagnose conditions
  Which of the following is the most appropriate
   management of a corneal perforation that
   remains unsealed after 3 days of aqueous
   suppression therapy?
    A. Aminocaproic acid
    B. Autologous plasmin
    C. Cyanoacrylate glue
    D. Topical antibiotic
This question tests whether candidates know the most
appropriate accepted management for the problem.
   References are required for every American Board of
    Ophthalmology (ABO) exam item and for every OKAP
    exam item.

   References must be complete in order to claim CME
    credit for high-stakes examination item-writing
    √ BCSC section 9, 2006-07, p. 52
    √ Mamalis n et al, journal of cataract and refractive surgery 2006;
      32: 324-333.
      Unacceptable: BCSC
      Unacceptable: Shields, The Glaucomas
 Pleasespeak to your staff representative
 from the ABO or AAO as appropriate if
 you write items for a high-stakes
 examination. They will provide you
 appropriate information .

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:2
posted:7/31/2012
language:English
pages:89