The State of Eating Disorders Research Publications 1980-2000. An

The State of Eating Disorders Research Publications 1980-2000. An Empirical Mary J. Markland, MA, AHIP (Southeast Clinical Campus Librarian); Analysis Stephen A. Wonderlich, PhD (Associate Chair/Professor Department of Neuroscience) James E. Mitchell, MD (Chair/Professor Department of Neuroscience); Ross D. Crosby, PhD (Director of Biomedical Statistics & Methodology, Department of Neuroscience/Neuropsychiatric Research Institute) Martina de Zwaan, MD (Research Scientist, Department of Neuroscience/Neuropsychiatric Research Institute) Research Questions • What is the quality of eating disorders publications? • Has the quality of eating disorders publications changed over time? • How does the quality of eating disorders literature compare to publications in anxiety? PsycInfo Search Strategies Eating Disorders (anorexia nervosa OR bulimia OR binge eating OR eating attitudes OR body image disturbances OR eating disorder*) AND (treatment* OR therap*) Limits: journal article, English (anorexia nervosa OR bulimia OR binge eating OR eating attitudes OR body image disturbances OR eating disorder*) NOT (treatment* OR therap*) Limits: journal article, English Population/Sample The interpretation of the results of any study depends on the characteristics of the population included for analysis. Define the population clearly. If control groups are part of the design, present how they are defined… Describe the sampling procedures and emphasize any inclusion or exclusion criteria. Clear Inclusion/Exclusion Criteria Eating Disorders Year Anxiety Disorders 80 Interval Estimates Percent Percent 77.9 85.5 84.2 70.9 85.7 88. 2 60 40 20 0 Methods Raters • 8 raters with master’s or doctoral level training in psychology/psychiatry Interval estimates should be 16 given for any effect sizes involving principal outcomes. 12 Provide intervals for correlations and other 8 coefficients of association or 4 variation whenever possible. APA Task Force on Statistical Inference 0 Confidence Intervals 20 Presented Eating Disorders Anxiety Disorders 13.2 10.4 6.6 1.2% 0.0% 3.9 1980 1990 2000 1980 1990 2000 APA Task Force on Statistical Inference Anxiety (anxiety disorder* OR phobia* OR obsessive-compulsive disorder OR panic disorder OR panic OR post-traumatic stress disorder OR traumatic neurosis OR stress reactions OR ocd OR ptsd OR post-traumatic stress disorder) AND (anxiety management OR treatment* OR therap*) Limits: journal article, English (anxiety disorder* OR phobia* OR obsessive-compulsive disorder OR panic Methods disorder OR panic OR Rating Instrument posttraumatic stress disorder OR traumatic neurosis 75-item rating based on recommendations from APA Task Force on OR stress reactions OR ocd OR ptsd OR 1999 54(8): 594-604. Statistical Inference American Psychologist post-traumatic stress disorder) NOT • Funding source • Description of measures •(anxiety management OR treatment* OR therap*) Research question • Psychometrics of measures •Limits: journal article, English Theory-based hypothesis • Blind rating of outcome • • • • • • • • • • • Sample size Completion rates Follow-up rates Clear inclusion criteria Method of determining inclusion Recruitment strategies Demographic information IRB approval Informed consent Description of procedures Random assignment • • • • • • • • • • • Research Design Control/Comparison group Effect size Clinical significance Constraint of alpha Multivariate tests Post hoc tests Reporting of statistics Confidence intervals A priori power analysis Methodological limits Interpretation Random Assignment of Subjects 30 Percent 20 25.6 Clinical Significance Data Presented 20 Percent 16 12 8 6.5 9.2 5.8 3. 9 1.3% 7.9 Training • 21 practice articles were reviewed jointly Assignment Random assignment allows for the strongest possible causal inferences free of extraneous assumptions…provide enough information to show that the process for making the actual assignments is random. (If) random assignment is not feasible, describe methods used to attenuate sources of bias, including plans for minimizing dropouts, noncompliance, and missing data. APA Task Force on Statistical Inference Procedure • Citations were stored in EndNote libraries by subject, treatment vs. non-treatment and year • Citations were randomly selected from EndNote and articles obtained • Article assignments were balanced by field, year, content, and rater pairs • All articles rated independently by 2 raters • Raters were blind to journal, authors, and author affiliation • Disagreements resolved via consensus Eating Disorders Anxiety Disorders When you interpret effects, think of credibility, generalizability, and robustness. APA Task Force on Statistical Inference Eating Disorders Anxiety Disorders 4 0 10 7.8 0 15.8 15. 8 9.1 1980 1990 2000 9.2 1980 1990 2000 Conclusions Note the shortcomings of your study…acknowledging limitations if for the pupose of qualifying results and avoiding pitfalls in future research. Methodological Limits Discussed 80 Percent 60 40 34.9 51.3 Statistical Analysis • Power Analysis: n=456 articles to detect small effect size for ED vs. Anxiety • Interrater reliability assessed using kappa coefficients • Statistical analysis performed using hierarchical log linear analysis Eating Disorders Anxiety Disorders 63.2 73.7 67.5 PubMed Search Strategies Eating Disorders (eating disorder* OR anorexia nervosa OR bulimia) AND (therapy OR therapies OR treatment) Limits: human, journal article, English (eating disorder* OR anorexia nervosa OR bulimia) NOT (therapy OR therapies OR treatment) Limits: human, journal article, English Procedures Clearly describe the conditions under which the measurements are taken (e.g. format, time, place, personnel who collected data). Describe the specific methods used to deal with experimental bias… APA Task Force on Statistical Inference Assessor Blind to Condition Eating Disorders Anxiety Disorders 60 Percent 40 30.8 62.5 54.5 45.0 62.5 20 0 APA Task Force on Statistical Inference 22.1 1980 1990 2000 20 0 25.0 Conclusions • Eating Disorder publications tend to be less methodologically rigorous than Anxiety Disorder publications in many important areas. Structured Interviews Prospective Longitudinal Designs Random Assignment Blind Outcome Assessment • Both Eating Disorder and Anxiety Disorder publications demonstrate improvement in methodological rigor over the last 20 years. Sample Size Alpha constraint Effect Size Psychometrics • The majority of Eating Disorder and Anxiety publications do not include many of the APA Task Force recommendations. Confidence Intervals Clinical Significance A priori power analysis Alpha constraint Anxiety Disorders (anxiety disorder* OR obsessive compulsive disorder OR panic disorder OR phobia* OR agoraphobia OR stress, disorders, post-traumatic OR posttraumatic stress disorder* OR ptsd) AND (therapy OR therapies OR treatment) Limits: human, journal article, English (anxiety disorder* OR obsessive compulsive disorder OR panic disorder OR phobia* OR agoraphobia OR stress, disorders, post-traumatic OR posttraumatic stress disorder* OR ptsd) NOT (therapy OR therapies OR treatment) Results Limits: human, journal article, English 1980 1990 2000 Study Design Eating Disorders Anxiety Disorders Total 1980 76 (38/38) 76 (38/38) 152 (76/76) 1990 76 (38/38) 76 (38/38) 152 (76/76) 2000 76 (38/38) 76 (38/38) 152 (76/76) Overall 228 (114/114) 228 (114/114) 456 (228/228) Power and Sample Size Provide information on sample size and the process that led to sample size decisions. Document the effect sizes, sampling and measuring assumptions, as well as analytic procedures used in power analysis. APA Task Force on Statistical Inference A Priori Power Analysis Mentioned 100 80 Percent 60 40 20 0 0.0% 0.0% 0.0% 0.0% 3.9% 0.0% Eating Disorders Anxiety Disorders 1980 1990 2000 Prospective Longitudinal Design Eating Disorders Anxiety Disorders 60 57.1 57.0 56.6 43.4 35.5 28.6 Effect Sizes Always present effect sizes for primary outcomes. Effect Size Data Presented 30 Percent Eating Disorders Anxiety Disorders 20.8 16.4 Design Make clear at the outset what type of study you are doing. Do not cloak a study in one guise to try to give it the assumed reputation of another…be sure to define and prioritize (the) goals. APA Task Force on Statistical Inference Research Question/Hypothesis StatedEating Disorders Percent 80 60 40 20 66.2 89.6 92.1 Limitations 25.7 Percent 100 Anxiety Disorders 76.7 78.9 77.6 40 20 20 APA Task Force on Statistical Inference 10 3.1 10.0 8.3 • • • • • Literature searches may not have found all the articles Excludes animal/basic science studies Excludes non-English articles Low interrater agreement on some ratings Raters could deduce field & year from articles 0 Year: 1980 1990 2000 0 0 1980 1990 2000 1980 1990 2000