Diagnostic tests

Evidence based medicine Diagnostic tests Ross Lawrenson Diagnostic tests • When looking at a paper about a diagnostic test we ask ourselves three questions. Diagnostic tests • Is this test useful? Diagnostic tests • Is this test useful? • Is it reliable? Diagnostic tests • Is this test useful? • Is it reliable? • Is it valid? Is this test useful? • The test should have been researched in a study population relevant to the individual or population in whom it is to be used. Reliability • Reliability refers to the repeatability or reproducibility of a test. • It can be assessed by repeating the test using the same or different observers. Validity • Relates to whether the test measures what it purports to measure. Is the result true? Validity • For example if you measure blood pressure in an obese patient and use a cuff that is too small you are likely to get a falsely high reading. The reading maybe reliable (you get the same blood pressure if you do it again) but it lacks validity. Sensitivity and specificity Sensitivity and specificity Test + Test Total Disease Healthy Total a b a+b c d c+d a+c b+d Sensitivity and specificity Disease Test +ve Test -ve Total a c a+c Healthy b d b+d Total a+b c+d a+b+c+d Sensitivity • The probability that the test will be positive if the disease is present • = a/a+c Sensitivity • The probability that the test will be positive if the disease is present • = a/a+c • A sensitive test is likely to also record a number of false positive tests Sensitivity If the cut off point of this test is set low then it will be sensitive (all patients with disease will test positive) but there will also be a number of false positives Healthy Diseased Specificity • Theprobability that the test will be negative if the disease is truly absent. • d/b+d Specificity • Theprobability that the test will be negative if the disease is truly absent. • d/b+d • In this situation there is a high likelihood of false negatives. Hpitl s sv i s c i,oeti g e c wi t h i y f n y i Nl oa r m Am b a n l o r Siva s ic eti np ii n t de t s y i c y f Teu l a df e e h i sl t e b e e su r o t n r a a f w y s iva s ic Tm eti np ii . h o n t de t e r s y i c y f e s iv t t e e le etee h w s n a t f r s i s e f a n te t T i i p nr ei t s i s o t a g e. h m f a s s v r o t a red iuie scs a ne s a s a ra s r ds u o se h p yt u Sa t m h le r. i i r h o ek n n oi m e r a ll y e s ice h w le pia t e e s e c t t f r f s e f a pte a eeoc ovt trl l to r s s ai y c i i h k u w cb p n c m h ae o t o o ih c ni r i m m t n a n d ss aibs ie scs e. s eu d t a h ae P pi v o rt a s ee ieiv e t i vd l c u Tb o h a fl e b t pi r l ot r i y y u h td w a ha e ve sn i n i e a gs h e s i t i o. cn st r g pe e e s e s i n t i v a /b a + N pi v eeiv e g rt a a ee t i vd l c u T b og h a fi e b b pi r l ot e i y n d fw ia e t sr h ee n s e e h e s i t i ee cn s t. r g n e e g e s a n t i v d /d c + Lo i il o s k d e r i ht a o Fo t rl os e u r ie e= at s t p i v s t (+ + a/b /c d a/ ) ) ( b Feerl ogt e r ae u at s t n i v s t = (+ + c/b /c d a/ ) ) ( d Accuracy of the test (a+d)/(a+b+c+d) Example 5000 women underwent a test for blood glucose at 24 weeks following a glucose load. 243 women were found to have a blood glucose greater than 6.8 mmol/L and were referred for an OGTT. 186 were found to have gestational diabetes. Four women who initially had tested negative were diagnosed as having diabetes later in their pregnancy. Example Prevalence Sensitivity Specificity Positive predictive value Negative predictive value Likelihood ratio + test Likelihood ratio - test Accuracy Diabetes Positive Negative Total 186 4 190 No diabetes 57 4753 4810 Total 243 4757 5000 Example Prevalence 190/5000 Sensitivity Specificity Positive predictive value Negative predictive value 186/190 4753/4810 186/243 4753/4757 Likelihood ratio + test Likelihood ratio - test Accuracy (186/190)/(57/4810) (4/190)/(4753/4810) 186+4753/5000 E p x l a e m Pa c r ln e ee v S iv e ti n t s y i S ic pf i ei t c y Pt ee t e le ov r i i v s p c a i i dv u Nt ee t e le e i pi i v g a r c a v dv u Ll o ri + t ie oa t kh i d oe t s Ll o ri -e ie oa t t kh i dos t Ar y ca cc u 3 .% 8 9% 7 . 9 9% 8 . 8 7% 6 . 5 9% 9 . 9 8 2 . 6 .2 0 9% 8 . 8 Gt d o ar ls a d n d Garnb i o n idt s l sd d a i e t d as e t O hG eT T . O em e t rs a v h t y a e s h t a gs ah t o ar a o l t dt t o d n d i s en o a f xs rv e pv i s o e e nv r i i rte el rc o u. fo e u sg on i n e.u i s a g y dt n rhrac ga f i e i po b o i rnt . ep to y i a h Gold standard . . Gold standard • The gold standard is the test or battery of tests that will most accurately diagnose a particular disease or condition. • Thus traditionally the OGTT has been seen as the gold standard when testing for diabetes. Other diagnostic tests may have a gold standard that is too expensive or invasive for routine use e.g. fluoroscein angiography for diabetic retinopathy. • Sometimes the gold standard is a battery of tests or symptoms e.g. the Jones criteria for rheumatic fever Reer r e r r cs c ot u e p i v a v o e B t t stt y i h sy p g ev l n o e ii t n i ap i o s ne t fttr di y e sf c i c af o d n ois i r cfoa f e tf n f t e up t R n ru Ob d Ce c c p e a od w els t h l i te i h l r h c p ae hs u t ou tf i t pm p o t i m on cfo u t u s e . Riept c e evoa us c r eo r e r r v T ptet r ova u s re e i i 1 0 . 8 0 . 6 > 4 0 > 8 0 2 8 0 0 . 4 > 0 . 2 0 0 . 000000001 0 . . . . . . . . . 1234567890 Fe s et a pt re l ova s i i Rf c t ie af dnn I Orr i ni s rig ig C e o a n k e ao M n o s Ror e e pc c eu e r r i v av e t r os Dcm i evy f ns e f t eb er a ru f df t or r ud ni e f fn o e p ooe o nr r ps dt u of n l ar i t i f f e pc re ee v. a l n Or ba s in ea r vt eo r i v I ov nra tsr r e in ae o b a vt r i I ov nra tsr evt r eo b in ea r i Eo a s br t n eo ii s in m rt ae a t g v r vi Prn e e rg ce em ne t a t . K A P P A Percent agreement Abnormal Abnormal Suspect Normal A D G Suspect B E H Normal C F I Percent agreement = (A+E+I) / Total X100 P lg dns a ot ig i o t h i' a s f os s o mm eo lna a 3a oeo s ie 7s f lnabt c sm m m e a u td b p l f eo e r o y a o lnap f an m m e e a xt s c s yne d i a t c i r et s h oddf e ee se i n cs a. s e Rwyot lg e e t poit v d wh s i e b a os Oodd c s n ne 2a e sr 1s ci e e ma n6n,h a n d bg e l n a1en i gt i t orodd ma t c ie 1 a n h n r 0 l n, e se i gt oidria n6n n em a2en e t n d bg ne t e i Percent agreement Melanoma Melanoma Indeterminate Benign 10 0 0 Indeterminate 1 0 0 Benign 10 0 16 Percent agreement = (10+0+16)/37 X100 = 70 % KAPPA Second Normal First Exam Normal Retinopathy Total 46 12 58 Exam Retinopathy 10 32 42 Total 56 44 100 Observed agreement = 46 + 32/100 = 78% KAPPA Second Normal First Normal 58%x56 Exam Retinopathy 42%x56 Total 56 Exam Retinopathy Total 58%x44 58 42%x44 42 44 100 KAPPA Second Normal First Exam Normal Retinopathy 32.5 25.5 Exam Retinopathy Total 23.5 18.5 Total Agreement expected by chance =32.5+18.5/100 =51% Estimating observer variation KAPPA = % obseved agreement - % expected by chance 100% - (percent agreement expected by chance) E t ov r o s i be ii ta s r a i n ev n m g r a t K =5 =5 A 71 P 8/ P - 4 0 A 9 . 5 Kc e e a a a be n p n e nd p bt a w 01 U arb0d s s s a .i i t u c o4 c a o v ne l l y e e a a oe lf r e r nl e g n e ae a m a b oe t sl v e a b0 g no 6o dv i o a .sd e . KA A P P O rn g a A m Ka g e r e n a e t p p 4 7 % 0 . 2 0. .-5 1 0 0 . 4 0 . 4 L no iee s v ci r r s R l a r 59 e ce c n t a c 0% 6 gig r n a d Hk o is dn g 5 6 % csa lsc n af t i i i o B ta r rs n ece a c 7 3 % csa lsc n af t i i i o Bo 3 a l r7 ni d e