Systematic Review of Research Methodology in Telemedicine Studies

Document Sample
Systematic Review of Research Methodology in Telemedicine Studies Powered By Docstoc
					 Systematic Review of
Research Methodology in
 Telemedicine Studies
          Pamela Whitten
      Liv Karen Johannessen
          Tove Soerensen
          Deede Gammon
          Michael Mackert
• Article in press, Journal of Telemedicine
  and Telecare
                Rationale
• Review articles struggling to document
  large scale evidence of the positive effect
  of telemedicine on health and cost
  outcomes
• Previous review articles claim
  methodology often poor in published
  papers
         Research Questions
• What is the incidence of theory testing in
  telemedicine literature? What theories are
  actually tested?
• What % of studies clearly report specific study
  variables (e.g., objective, hypotheses/RQ, # and
  type of subjects, randomization, units of
  analysis, location, time frame)
• What data collection strategies and analyses
  were employed?
• Were results presented in qualitative and/or
  quantitative fashion?
               Methodology
• Keyword search strategy employed on 15
  scientific databases
• Initial search yielded more than 10,000 articles
• Articles then assessed against inclusion criteria
  (e.g., published post 1990, included data,
  included some form of a methods section, etc.)
• N=1615 articles
• Inter-rater reliability (Cohen’s Kappa) exceeded
  .80 for three pairs of coder
                 Results
• 5% mention theory or paradigmatic
  approach
• 96% report overall study aim
  – Seek some form of outcome (80%)
  – Test technical feasibility (13%)
  – Address hypothetical issues (7%)
• 11% provided explicit hypotheses or
  research questions
              Results (cont.)
• 26% did not provide # of subjects
• For those reporting patients as subjects:
  – Median = 47
  – 65% had sample size of 100 or less
• For those reporting providers as subjects:
  – Median = 29
  – 86% had sample size of 100 or less
• 11% reported randomization for patient studies
  and 4% for provider studies (most papers did not
  even mention subject selection strategies)
                        Results
• 83% reported location of study
  – 34% in US
  – 24% Europe
• Data collection
  – Patients
     • 31% multiple data collection strategies
     • Image Transfer (35%)
  – Providers
     • 18% multiple data collection strategies
     • Surveys (60%)
                 Results
• Few significant relationships
  – Use of theory and presence of formal
    research questions (r=0.25, P<0.01)
  – Research questions and study aim (r=0.067,
    P<0.01)
  – Inclusion of theory and study based on an
    actual intervention (r=-0.069, P<0.01)
                Discussion
• These methodological challenges not
  owned by telemedicine
• Implications of missing methodological
  details should not be underestimated
  – Time impacts technologies
  – Subject selection and sample size matter
  – Lack of explicitly stated RQs prevent true
    critique of study design
          Kuhn Basic Tenet:
• The existence of intertwined theoretical
  beliefs and methodological strategies
  make it possible for a field to evaluate its
  own body of research