Organ retention

Document Sample
Organ retention Powered By Docstoc
					 Question Banking –
the UMAP experience

   Emyr Wyn Benbow
 University of Manchester
              So what was UMAP?
UMAP consortium
 - five original partners
   - Manchester
   - Leeds
   - Liverpool
   - Newcastle
   - Sheffield
             So what is UMAP now?

UMAP consortium now
Birmingham   Cork
Hull York    Keele
Leeds        Leicester
Liverpool    Manchester
Newcastle    Peninsula
Sheffield    Southampton
UEA          Warwick
                             The competition

 IDEAL                     International
                             Database for
                                  Assessments &

An International Collaboration of Medical Schools
                                             The competition
-   Australia (Sydney, Flinders, ANU, Monash, Queensland, Newcastle,
    Tasmania, Adelaide)
-   Canada (U Alberta, U British Columbia)
-   China (Hong Kong U, Chinese U Hong Kong)
-   Indonesia (U Padjadjaran)
-   Islamic Republic of Pakistan (Shifa College of Medicine, U Health Science)
-   Kingdom of Bahrain (Arabian Gulf U)
-   Kingdom of Saudia Arabia (King Saud Bin Abdulaziz U Health Sc)
-   Kuwait (Kuwait U)
-   Republic of South Africa (U Witwatersrand)
-   Sultanate of Oman (Sult.Qaboos U)
-   United Arab Emirates (United Arab Emirates U, U of Sharjah)
-   United Kingdom (Pennisula Medical Sch)
                                     The competition
- summative assessment bank > 13,000 items
- formative assessment bank > 6000 items
- six types of question
   -   best one of “n” choices (A)
   -   extended matching (R)
   -   multiple Response (Pick n)
   -   short Answer (SAQ)
   -   modified Essay (MEQ)
   -   OSCE/OSPE
                        The competition
ITEM FORMAT                     NO.
A type: 3-5 options           15887
A type: > 5 options             103
A type: Multiple Response        30
R type                         1646
R type: Multiple Response        62
Short Answer                    339
Modified Essay                  339
OSCE                            229
OSPE                              3
                                           UMAP process
            UMAP Admin

                                                 UMAP Admin



    Question               Question            Results
    Release                  Use               Return
                             UMAP funding
- originally HEFCE grant
   - Fund for the Development of Teaching
     and Learning Phase 4 (FDTL-4)
- self-funding from January 2006
   - each partner school pays an equitable
     proportion of costs (£7,500 in 2996-7)
                             UMAP workshops
- two workshops at each partner each year
- each led by UMAP’s manager
- one for MCQs, one for EMQs
- use best practice
- authors use question templates
- authors review each others questions
- clinical and academic participation, all levels
-   workshops
                              Typical UMAP MCQ
A 52 year old man presents to his General Practitioner with pains
  in both hands. The symptoms began as a tingling sensation
  which have now developed into a pain which keeps him awake
  at night. The man describes the overall sensation as a feeling
  of ‘uselessness' in his fingers and he says though there is no
  swelling they feel swollen.

  What nerve has been compressed?
A. A          Axillary nerve
  B Median nerve
  C Musculcutaneous nerve
  D   Radial nerve
  E Ulnar nerve
        UMAP quality assurance teams
- items are pre-reviewed for adherence to “house
-   question review teams concentrate on content
-   each partner site runs six sessions a year
-   specially designed database
-   QA begins with cover up test
-   each item to be accepted or rejected
-   no pressure to accept items
   Responsible: UMAP Manager,
     Best practice instruction,
        writing and review
                                        UMAP administration
                                         Responsible: UMAP Administrator
                                      Enter and check questions for common
                                       errors etc e.g. abbreviations, spelling,
                                                   normal ranges
        Pre reviews
   Responsible: UMAP Manager
   Questions checked for overall
concordance with UMAP ‘house style’

                                       Review team meetings
                                             Responsible: team leads
                                           Network of teams working on
                                            UMAP reviewer databases
          Follow up
 Responsible: UMAP Administrator
 Reviewer decisions and comments
             acted on
Quality assurance
Quality assurance
Progress with banking
Progress with banking
                             UMAP “rules”
- all partners must
   - write items
   - provide quality assurance
   - only use items in secure summative
        How it works: setting a paper
Stage 1 - the planning form
Stage 2 - question selection
Stage 3 - question receipt
Stage 4 - reporting problems
Stage 5 - reporting your outcomes
How it works: the planning form
How it works: question selection
How it works: question selection
        How it works: question receipt
Two documents
 - Word document with all selected questions
     - UMAP ID number
     - theme
     - focus
     - question text
     - feedback
       How it works: question receipt
Two documents
 - Excel spreadsheet
    - to report which questions you’ve
How it works: reporting problems and
- any difficulties found at local question
  selection and quality control processes
  need to be reported back
- detailed examination results are fed
  back, and performance data will be
How it works: reporting outcomes
       The End

- but this is just the end of
       the beginning!

Shared By: