A global leader in serving libraries of all types, ProQuest LLC (“ProQuest”) supports the breadth of the information community with innovative discovery solutions that power the business of books and the best in research experience. More than a content provider or aggregator, ProQuest is an information partner, creating indispensable research solutions that connect people and information. Through innovative, user-centered discovery technology, ProQuest offers billions of pages of global content that includes historical newspapers, dissertations, and uniquely relevant resources for researchers of any age and sophistication—including content not likely to be digitized by others.
0058_Patterson_paper.qxd 9/18/09 4:55 PM Page 417 ■ PROFESSIONAL ISSUES Clinical Medicine 2009, Vol 9, No 5: 417–20 New machine-marked tests for selection into core medical training: evidence from two validation studies Fiona Patterson, Victoria Carr, Lara Zibarras, Bill Burr, Liz Berkin, Simon Plint, Bill Irish and Simon Gregory ABSTRACT – This study examined whether two machine- they may encounter at work and are asked to identify an marked tests (MMTs; a clinical problem-solving test and sit- appropriate response from a list of alternatives. uational judgement test), previously validated for selection This paper describes an evaluation study to examine whether into UK general practice (GP) training, could provide a valid these MMTs can provide a valid selection methodology for methodology for shortlisting into core medical training shortlisting into core medical training (CMT). (CMT). A longitudinal design was used to examine the Any new selection methodology must satisfy exacting psycho- MMTs’ psychometric properties in CMT samples, and corre- metric criteria such as reliability, validity and fairness.2–4 This lations between MMT scores and CMT interview outcomes. paper therefore reports the psychometric properties of the CPS Independent samples from two years were used: in 2008, a and SJT for independent samples of CMT applicants in two con- retrospective analysis was conducted (n 1,711), while in secutive years (2008 and 2009) and examines whether scores in 2009, CMT applicants completed the MMTs for evaluation these MMTs (time 1) predict performance in the CMT selection purposes (n 2,265). Both MMTs showed good reliability in interview (time 2) approximately one month later. For the 2009 CMT samples, similar to GP samples. Both MMTs were good sample, MMT marks were also compared with the current predictors of CMT interview performance (r 0.56, shortlisting process, based on scores awarded to sections of a p 0.001 in 2008; r 0.61, p 0.001 in 2009) and offered structured application form. The objectives of this paper were to incremental validity over the current shortlisting process. explore the reliability and predictive validity of the CPS and SJT The GP MMTs offer an appropriate measurement method- for applicants to CMT. In particular, the aim was to explore any ology for selection into CMT, representing a significant inno- potential gains in effectiveness by using these MMTs compared vation for selection methodology. to current application form scoring procedures. Specifically, we KEY WORDS: interviews, machine-marked tests, selection, addressed the following research questions: shortlisting, validity 1 Are the psychometric properties of the CPS and SJT robust Introduction in the CMT applicant samples? 2 Are the CPS and SJT valid methods of selection for CMT Selection into postgraduate training has been a widely debated applicants? Do they predict subsequent performance in the topic, especially in the UK. Recently, the selection methodology CMT interview? used for entry into UK general practice (GP) has demonstrated 3 Compared to the current application form-based short- good reliability and validity evidence.1 The methodology listing procedure, do the MMTs add incremental validity in includes two invigilated, machine-marked tests (MMTs): predicting outcomes? • a clinical problem solving test (CPS) comprising questions Method that require applicants to apply clinical knowledge to solve problems reflecting diagnostic processes or develop man- Recruitment into general practice and core medical training agement strategies for patients in the UK
Pages to are hidden for
"New machine-marked tests for selection into core medical training: evidence from two validation studies"Please download to view full document