Presentation 5

Document Sample
Presentation 5 Powered By Docstoc
					Specialty Certificate Examinations (SCEs):
           Where Are We Now?

              Oliver J Corrado
         Leeds General Infirmary,
     Former Lead SCE Geriatric Medicine
        Chair SAC Geriatric Medicine
Specialty Certificate Examinations (SCEs)

• Background to SCEs and Pilot Examination

• The Development of SCEs

• Where we are now?
Why Do We Need Another Exam?

• As part of new curriculum PMETB require all
  StRs registering on or after 1 August 2007 to
  have formal assessment of competence
  including work placed based assessments eg
  mini-CEX and SCE

• To ensure that specialists dealing with patients
  have the appropriate knowledge of their
Pilot Examination

2006 JCHMT in partnership with specialist societies ran KBA
pilots in 4 specialties
(cardiology, neurology, geriatric med and dermatology)

Question writers limited training, small question bank, for
geriatric medicine pilot standard setting not robust

100 best-of-five questions in 3 hours, paper based exam,
sat on 2 dates in May 2006 a week apart

448 sat exam (411 SpRs (77% total SpRs), 37 consultants)
Best-of-five Question

An 82-year-old woman had a 3 year history of urinary
frequency, with a sudden desire to urinate at times
associated with incontinence, but no dysuria. She was on no
medication and there were no abnormal findings on examination. Mid
stream urine specimen was negative.

The most likely cause for her symptoms is:

A    atonic bladder
B    autonomic neuropathy
C    detrusor instability
D    pelvic floor weakness
E    overflow incontinence
Question: Courtesy of Adam Harper !
A 92- year-old woman was admitted to hospital with pneumonia. Her
daughter was concerned her mother may develop Clostridium difficile
What is the greatest risk factor for her developing Clostridium difficile
   A a matching tie and cufflink set

   B cramming 6 beds into an area designed for 4

   C having insufficient nursing staff so discharge is delayed by 1

   D sharing a commode with 6 other patients

   E sharing a sphygmomanometer cuff with 10 other patients
    Answer: A
Pilot Evaluation and Comments

• Questions? : About right 64%

• Right amount of time? : About right 70%

• Familiar with question format? : Yes 80%


Poor questions: ambiguous, too long, negative (eg “which is
least likely”) and double negative questions
Too many questions
Too many on falls, inappropriate orthopaedic questions
Some questions not relevant to Scottish law
Pilot Evaluation

Evaluation of pilot by JCHMT positive

However for pilot KBA in Geriatric Medicine Cronbach’s coefficient
alpha (measure of reliability) 0.67
(for high stakes exam should be 0.80 +, preferably 0.9+)

(Spearman-Brown formula if 200 questions used alpha 0.81)

But no standard setting was undertaken in Geriatric
Development of SCEs

Federation of Royal Colleges proposed introducing this format as KBA for 13
major medical subspecialties in partnership with specialist societies split
75% v 25%. KBA called SCE in November 2008

Initial Proposals

2 diets a year 200 b-o-f questions, on-line (using Pearson Vue (DVLA theory)

Use infrastructure and expertise of MRCP (UK) to administer/organise

Candidates with MRCP who pass SCE awarded MRCP (specialty) those without
Diploma. No differentiation in post-nominal Award for UK trainees from others.

Cost £800 if sat in UK, £1000 if taken in an overseas centre
BGS Newsletter March 2007
Development of SCE


Question Writing Group - 25 BGS members (in response to advert and
email to Regional and Council reps) broad interests, non-academics
and some academics, representation from all 4 nations, all trained and
not paid!

Examination Board - 10 members including Chair and
secretary of SAC

Standard Setting Group – several members with expertise in standard
setting undergraduate examinations.
Relationship of 3 Groups

  Question Writing Group
                           Questions              Reject



                                 Select 200+ questions for paperfor

                                       Standard Setting Group
                                           “Anghoff Method”

Where Are We Now?

1.    Successfully negotiated award title change from Diploma to
Certificate (avoids confusion with DGM). For UK trainees award will
convert to MRCP (specialty) at time CCT.

2. Not been able to reduce cost of exam (£800 if taken in UK, £1000 if
taken overseas) but negotiated free resits 2008-10

3. One “diet” (exam) to be held per year (not 2)

4. 200 b-o-f questions (2 papers), same day, computer-based test,
Pearson Vue 12 centres (centres in all 4 UK nations)
Where Are We Now?

5. Gastroenterology held exam 24 June 2008. Geriatric Medicine 4
   March 2009, followed by Nephrology, Respiratory Med and

6. Dermatology and Medical Oncology September 2009. Med Onc,
   Rheumatology and Acute Medicine 2010

7. Next exam in Ger Med March 2010 date to be finalised

8. 5 sample questions on MRCP website – need log in
Eligibility Criteria and Curriculum

• Non UK candidates must have MRCP
   UK candidates ST3+, ideally pass it before PYA

• SCE based on JRCPTB Curriculum in Geriatric

• SCE regulations include exam “blueprint”
Area                                                                                           %

Health Status; Health Promotion; Disease Prevention                                             3

Gerontology and Illness of Old Age                                                              6

Rehabilitation; Poor Mobility; Intermediate Care/Community Practice; Assessment Scales;         5
Discharge Planning; Continuing Care; Health Care Organisation

Ethical Issues; Medico-legal Aspects; End of Life Planning                                      4

Acute and Chronic Disease (Cardiovascular 5; Respiratory Disease 4; GI Disease 4; Endocrine    37
Disease 3; Renal Disease 3; Neurology 5; Sensory Impairment 2; Dermatology 1; Musculo-
Skeletal 3; Anaemia/Haematology 2; Infection 5)

Falls                                                                                           7

Delirium and Dementia                                                                           6

Continence                                                                                      5

Palliative Care                                                                                 4

Orthogeriatrics and Osteoporosis                                                                4

Old Age Psychiatry (other than Delirium and Dementia)                                           2

Stroke and TIA                                                                                  7

Tissue Viability; Thermoregulation and Nutrition                                                3

Clinical Pharmacology and Therapeutics and Safe Prescribing                                     7

Specialty                   Overall   pass UK pass rate
                            rate           %

Geriatric Medicine (n=15)   100            100

Nephrology                  57.58           81
Respiratory Medicine        60.00           89

Neurology                   80.00           87.5
Endocrinology & Diabetes    38.46           64

1. What happens to trainees who cant pass the exam?

2. Why cant UK based trainees who pass the exam but
    are ineligible for a CCT (ie pursuing CESR route to
    specialist certification) have MRCP (Ger Med)
Useful Websites


• (curriculum, training)



Thanks to
• Question writers
• Examination Board in particular Mike Vassallo (Secretary)
  (now SCE Lead)

• Standard Setting Group
• SAC Geriatric Medicine and BGS ETC

• Trainee reps and trainees

Shared By: