sim by xiaohuicaicai

VIEWS: 13 PAGES: 43

									         RCT Schema
     The Trial Bank Project

                     Ida Sim, MD, PhD
            Associate Professor of Medicine
Director, Center for Clinical and Translational Informatics
       University of California, San Francisco, CA

                          Supported by
             The Trial Bank Project R01-LM-06780
   National Center for Biomedical Ontology U54 HG004028-01



                                                              Copyright 2007, Ida Sim
Outline
• Background
• RCT Schema
  »   modeling approach
  »   the class structure
  »   evaluation
• Relationship to CTO
• Summary



                            Copyright 2007, Ida Sim
Goals of the CTO
• (1) fully and faithfully capture the types of entities and
  relationships involved in clinical trials
• (2) comprehend terms like: cohort, randomization, placebo, etc.,
  including ... statistical terms and terms for ... meta-analysis;
• (3) organize these terms in a structured way, providing
  definitions and logical relations designed to enhance retrieval of,
  reasoning with, and integration of the data annotated in its terms
• ...
• (6) draw on and seek maximal alignment with existing clinical
  trial ontologies, including:
   »   RCT Schema ontology used by theTrial Bank Project




                                                                Copyright 2007, Ida Sim
Major Axes for Aligning Models
• Domain of clinical trials
   »   design: (non-)randomized, crossover, cluster randomized,
       factorial...
   »   objective: interventional, diagnostic, preventive...
   »   clinical domain: drugs, procedures, organizational change...
• Task
   »   trial design, execution, reporting, analysis, application
   »   for individual trials, sets of clinically related trials
• Purpose (application vs. domain ontology)
   »   to support accomplishment of domain task(s)
   »   to define shared meaning for “integration of the data annotated
       in its terms”

                                                                   Copyright 2007, Ida Sim
Trial Tasks
                                                                  Trial Application


                                                                                Ele ctronic
                                                            Guidelines
                                                                              Patient Record

                                                        Decision Models
               Trial Interpretation
                                          Systematic Review

                            Trial Bank
       Trial Registration                Scientific R eporting

                                                                            Fe edback to
Trial Design      Trial Condu ct                                            Trial De sign
                                         Reg ulatory Rep orting


                    Trial Execution

                                                                                      Copyright 2007, Ida Sim
Trial Bank Definition
• Computable repository of RCT information sufficiently
  detailed to support scientific analysis for
   »   designing future clinical trials
   »   evidence-based practice and policy making
• Detailed information on
   »   study design
   »   study execution
   »   summary and individual participant-level results
• Trial Bank is NOT for running a trial




                                                          Copyright 2007, Ida Sim
Trial Bank Target Uses
                                                                  Trial Application


                                                                                Ele ctronic
                                                            Guidelines
                                                                              Patient Record

                                                        Decision Models
               Trial Interpretation
                                          Systematic Review

                            Trial Bank
       Trial Registration                Scientific R eporting

                                                                            Fe edback to
Trial Design      Trial Condu ct                                            Trial De sign
                                         Reg ulatory Rep orting


                    Trial Execution

                                                                                      Copyright 2007, Ida Sim
Trial Bank Software
• RCT Bank built on RCT Schema
   »   Ocelot frame-based ontology
• Bank-a-Trial
   »   web-based program for trialists to enter trial instances into RCT Bank
   »   clinical descriptions of trial features (slot values) are in UMLS
• RCT Presenter
   »   web-based browser of individual trials


         Bank-a-Trial                                  RCT Presenter


                                     RCT Bank
                                     RCT Schema



                                                                           Copyright 2007, Ida Sim
Major Axes for RCT Schema
• Domain of clinical trials
   »   design: (non-)randomized, crossover, cluster randomized,
       factorial...
   »   objective: interventional, diagnostic, preventive...
   »   clinical domain: drugs, procedures, organizational change...
• Task
   »   trial design, execution, reporting, analysis, application
   »   for individual trials, sets of clinically related trials
• Purpose
   »   to support accomplishment of domain task(s) [application ontology]
   »   to define shared meaning for “integration of the data annotated in
       its terms”


                                                                   Copyright 2007, Ida Sim
Outline
• Background
• RCT Schema
  »   modeling approach
  »   the class structure
  »   evaluation
• Relationship to CTO
• Summary



                            Copyright 2007, Ida Sim
Entity Specification Problem
• What RCT aspects to model in RCT Schema? What
  not to model?
  »   multiple users (e.g., trialists, systematic reviewers)
  »   multiple tasks (e.g., analysis, interpretation)
      – multiple methods
  »   no one correct RCT ontology
• Need principled, systematic approach
  »   to specifying, documenting, and evaluating




                                                          Copyright 2007, Ida Sim
Competency Decomposition
Method                         (Sim, et al, JBI 2004: 37(2):108-119)


• To define the entities that must be in a conceptual
  model
• General approach
  »   specify a task hierarchy of target tasks and subtasks
  »   specify methods for each task
  »   specify entities required for completing each task
      using each method
• Generates a specification of required entities
  »   the information requirements for the competencies
      (tasks and subtasks) that the model/knowledge base
      is to support

                                                                       Copyright 2007, Ida Sim
Target Task for RCT Bank
                                                                Trial Application
  • Using RCTs for trial design or
    clinical application requires
                                                                              Ele ctronic
    synthesizing evidence across                          Guidelines
                                                                            Patient Record
    all trials on a topic
                                                      Decision Models
               Trial Interpretation
                                        Systematic Review

                            RCT Bank
       Trial Registration              Scientific R eporting

                                                                          Feedback to
Trial Design      Trial Condu ct                                          Trial Design
                                       Reg ulatory Rep orting


                    Trial Execution

                                                                                    Copyright 2007, Ida Sim
Systematic Reviewing
• Canonical method for synthesizing evidence across
  trials
• Major steps are
  »   retrieve related RCTs (e.g., 32 trials of metformin for
      diabetes)
  »   analyze how comparable the trials are
  »   statistically combine data if appropriate
      – combining smaller trials increases statistical power to detect
        effects



                                                                Copyright 2007, Ida Sim
 Target Task = Systematic Review
• RCT Bank entities = sys. review information needs
  »   identified all systematic review tasks
      – review of literature and personal experience conducting 3
        systematic reviews
  »   identified methods for completing these tasks
  »   organized tasks and methods into a task hierarchy
  »   derived RCT entities necessary and sufficient for each
      (sub)task




                                                               Copyright 2007, Ida Sim
Top-Level Sys Review Tasks
• Trial retrieval
• Trial critiquing
   »   judging internal validity
   »   judging generalizability
• Meta-analysis of quantitative results
   »   analysis of clinical and statistical heterogeneity
• Contextual interpretation
   »   scientific, socio-economic, and ethical
                                     http://rctbank.ucsf.edu/tasks/tasks.html


                                                                     Copyright 2007, Ida Sim
 Judgment of Generalizability
• Were the people enrolled in the trial representative and
  unbiased?
  »   were eligible patients randomly selected from the source
      population?
  »   were enrolled subjects a random subset of those
      eligible?
• Are the trial subjects similar to mine?
• Do I have the tested intervention available here?



                                                       Copyright 2007, Ida Sim
Method-(In)dependent Entity
Specification
• Were the people enrolled in the trial representative
  and unbiased?
  »   were eligible patients randomly selected from the
      source population?
      – method: no computable algorithm available
         recruitment method

  »   were enrolled patients a random subset of those
      eligible?
      – method: using standard statistics
         number and clinical characteristics of enrolled subjects

         number and clinical characteristics of eligible but non-
          enrolled subjects

                                                             Copyright 2007, Ida Sim
    Results: Entity Requirements
High-level     Retrieval           Critiquing        Quantitative     Contextual
Tasks                                                 Synthesis     Interpretation
SubTask I                      Internal   External                       .....
                               Validity   Validity                            3
Methods                 2
                                                     ...
               1                                      4        1


SubTask II    1
                                 ...       ...       ...      ...           ...
                        2
(n=35)                            11        4         7        2              9

SubTask III
                                 ...       ...                             ...
(n=74)                           39         13                              22
     171 Unique Entities Required 112       29                              30

                                                                      Copyright 2007, Ida Sim
 Entity Specification Evaluation
• Evaluated match between
  »   the 171 information items
  »   388 requirements in 18 published trial-critiquing
      instruments
• Results
  »   entity specification is comprehensive
  »   entity specification is reasonable

                           Sim, et al, KR-MED 2004; JBI 2004: 37(2):108-119



                                                                    Copyright 2007, Ida Sim
Benefits of Approach
• Task hierarchy understandable by domain experts
  »   identifies which information items are required for which
      tasks
• Provides an evaluation “yardstick”
  »   if an ontology contains all the information requirements
      for a task
       – then is it “competent” for that task
  »   can evaluate and compare application ontologies
• Documents an (application/domain) ontology
  »   states which tasks an ontology is competent for, and why
  »   cross-indexes tasks and entities in the ontology

                                                            Copyright 2007, Ida Sim
Outline
• Background
• RCT Schema
  »   modeling approach
  »   the class structure
  »   evaluation
• Relationship to CTO
• Summary



                            Copyright 2007, Ida Sim
Implementation of 171 Items
• Purpose of RCT Bank KB is to support scientific
  analysis of trial evidence
  »   needed a “data-schema” or “instance-style” ontology

       Bank-a-Trial                   RCT Presenter


                         RCT Bank
                         RCT Schema



• RCT Schema implemented the 171 information
  items in a frame-based ontology

                                                      Copyright 2007, Ida Sim
RCT Schema Ontology
• Ocelot frame-based model
  »   7 levels deep
• 192 frames, 607 unique slots
  »   avg. 9.8 slots/frame
  »   3 frames (1.6%) have multiple parents
  »   193/607 slots (32%) take other frames as values
• Available at http://rctbank.ucsf.edu/




                                                        Copyright 2007, Ida Sim
• RCT Schema displayed in
  GKB Editor
   »   classes are red boxes
   »   instances are blue boxes
• Class hierarchy organizes
  entities as
   »   trial concepts
   »   trial descriptions (details)
• Not fully compliant with
  “Werner’s Rules”
Includes IS-A Hierarchies




                            Copyright 2007, Ida Sim
Instantiating RCT Schema
• Clinical content described by terms from a clinical
  vocabulary, e.g.,
   »   for a breast cancer trial, instance of BASELINE-
       CHARACTERISTIC is described by
       – term “menopause” from UMLS preferred term
       – the UMLS CUI
   »   Trial Bank software supports any vocabulary in
       UMLS (e.g., SNOMED)
• Each trial is a collection of instances of classes


                                                          Copyright 2007, Ida Sim
• 518-TRIAL
  »   518-
      BACKGROUND-
      DETAILS
  »   518-ADMIN-
      DETAILS
  »   518-EXECUTED-
      PROTOCOL
      – 518-ALL-SUBJECTS
      – 518-PRIMARY-
        OUTCOME-1
      – etc.
  »   518-ERRATUM
  »   518-CONCLUSION-
      DETAILS
• 518-PRIMARY-OUTCOME-1 (e.g., all-cause mortality)
  »   518-STAT-ANALYSIS-AND-RESULTS-1 (e.g. t-test)
       – 518-ALL-COMPARISONS-AT-TIME-X-1 (e.g., at 6
        months)
         518-SINGLE-TIME-X-COMPARISON-1 (e.g., between
          PCI and thrombolysis groups)
• 518-SINGLE-TIME-X-COMPARISON-1
  »   datapoint for PCI group
      – numerator (all-cause deaths at 6 months)
      – denominator (had all-cause death outcome assessed at 6 months)
      – 518-STUDY-ARM-POPULATION-1 (the PCI group)
  »   datapoint for thrombolysis group
  »   summary odds ratio under intention-to-treat analysis
  »   summary odds ratio under efficacy analysis
Outline
• Background
• RCT Schema
  »   modeling approach
  »   the class structure
  »   evaluation
• Relationship to CTO
• Summary



                            Copyright 2007, Ida Sim
 Expressivity Evaluation
Characteristic Examples
               Cardiology, Radiology, Geriatrics, etc.
Clinical domains
Inter vention  Procedures (th rombolysis), Single and Multi-
Types          step Drugs (as pirin, warfarin), Counseling,
               Multiple interventions in one arm
Outco me Types Dichotomous, continuous, uni variate,
               multivariate, survival, regression, scored
               instru ments (e.g., Wechse l Memory Scale)
Result Types   Inten tion-to-treat, efficacy analysis, subgroup
               analyses

   • Captured 17 full and 20+ partial trials

                                                          Copyright 2007, Ida Sim
Modeling Challenges Met
• Multi-armed, crossover, cluster randomized studies
• Many variations of patient drop-out, loss to followup
  (e.g., excluded after randomization)
• For each outcome, the # of subjects assessed at each
  timepoint in each subgroup
• Blinding efficacy: did subjects know which arm they
  were assigned to?
• etc.




                                                    Copyright 2007, Ida Sim
Modular, Extensible
• Extensions possible with only minimal changes to 192
  existing classes
                                 New      New     Old Classes
       Modeling Extension       Classes   Slots    Changed
Cluster randomized trials          2       10         3
Complex intervention regimens      4       8          3
Computable eligibilty rules        ?       ?          3

• Extensible to new clinical domains (e.g., genomics) via
  clinical vocabularies
   »   no clinical terms in RCT Schema


                                                          Copyright 2007, Ida Sim
Limitations of Representation
• Modeled, not yet tested
  »   participant-level data
  »   factorial designs
  »   designs with run-in and washout periods
• In development
  »   computable eligibility rules
• Not yet modeled
  »   genomic data
  »   nested subgroups
  »   secondary studies (e.g,. followup studies)

                                                   Copyright 2007, Ida Sim
Trial Bank Publishing
• How to get trials into RCT Bank?
• Collaborated with JAMA and Ann Int Med to explore co-
  publishing trials as articles and RCT Bank entries
   »   authors submit manuscripts for peer review as usual
   »   trial-bank staff enter accepted trials into trial bank
   »   co-published 14 trials in RCT Presenter
• Evaluation
   »   83 respondents evaluated a trial using both RCT
       Presenter and the Journal Article
       – mostly trialists and meta-analysts



                                                                Copyright 2007, Ida Sim
                RCT Presenter Evaluation
                                 Presenter Better Same as Article

                100                                                             •   70% of respondents
                 90   12   12        12
% Respondents




                 80                               23                                rated RCT Presenter
                                27
                 70                                            39   50   39         as good as or better
                 60                                     31
                 50                                                                 than the Journal
                                           65
                 40   85   81        85                                             Article for all
                                65                73
                 30                                                                 attributes
                                                        42     46   42   50
                 20
                 10                        15
                  0
                                                     er


                                                      r




                                                     er
                                                      e
                                                      d
                                                     er




                                                      y




                                                     le

                                                     re
                                                      l
                                                    ia
                                                   re




                                                  or
                                                  th
                                                  ze




                                                 ab
                                                  si




                                                 ef
                                                 st




                                                ca
                                                 tr
                                               ea




                                                m
                                               or
                                               ni
                                              Ea
                                              Fa




                                              pr
                                             nd
                                             ng




                                             al
                                             Cl

                                           ga

                                            tw




                                             e

                                          ld
                                          ta

                                          ic

                                         us
                                          si
                                        us
                                         or




                                        in




                                      ou
                                       rs
                                       ai
                                     tr




                                     ld
                                     cl
                                     e




                                    pr

                                   de




                                   W
                                  or




                                 ou
                                  e




                                   r
                                ap

                                un

                                fo
                                or
                                M




                              W
                             M

                              r




                            ul
                            e
                           fo

                          or

                         ef
                       er




                                                                         N=30
                        M

                      us
                     si




                    e
                  Ea




                  or
                M




                                                                                                 Copyright 2007, Ida Sim
Outline
• Background
• RCT Schema
  »   modeling approach
  »   the class structure
  »   evaluation
• Relationship to CTO
• Summary



                            Copyright 2007, Ida Sim
Alignment to CTO
• Have participated in CTO working group from
  inception (Simona Carini)
• Contributed RCT Schema classes and definitions to
  list of terms for consideration
• Contributed to draft of high-level concept hierarchy
     – http://www.bioontology.org/wiki/index.php/High-
       level_Concepts_v0.2
• Contributed to creation of the draft CTO presented
  this morning


                                                         Copyright 2007, Ida Sim
How Would Trial Bank Use CTO?
• Use CTO as common index into RCT Bank and into
  other clinical trial data and information systems
• Map RCT Schema class and slot names to terms in
  CTO
  »   so that RCT Bank instances can be made available to
      machines and humans who wish to
      – query
      – reason, or
      – integrate
  »   clinical trial information using CTO
  »   e.g., trials co-published with PLoS, etc. into RCT
      Bank
                                                           Copyright 2007, Ida Sim
 Example Use (NCBO)
• Map class names from any trial bank to terms in CTO
   »   e.g., in RCT Bank or “European Trial Bank”
       – PRIMARY-OUTCOME to CTO term for this
       – BASELINE-CHARACTERISTIC to CTO term for this
• CTeXplorer an ontology-driven tool for visualizing
  complex design differences across trials [MA Storey, et al
  UVic, Canada]
   »   given variable HgbA1C from any trial bank
   »   would know how to handle and display if annotated as
       – PRIMARY-OUTCOME [CTO] or
       – BASELINE-CHARACTERISTIC [CTO]

                                                        Copyright 2007, Ida Sim
CTO for Integration with
Trial Bank Collaborators...
• National Center for Biomedical Ontology
• Electronic Primary Care Research Network
   »   Primary Care Research Object Model
• Global Trial Bank, with AMIA
   »   trial-bank publishing
   »   “dedicated to assuring the implementation and maintenance
       of an open global infrastructure for computable clinical trial
       results information”
• European Clinical Trial Data Repository
   »   submitted to Framework Programme 7
• with BRIDG, clinical trial management systems, etc?


                                                                 Copyright 2007, Ida Sim
Summary
• Trial banks are computable repositories of trial information
  for analysis, interpretation, and application of trial evidence
  to research and care
• We specified the entities for RCT Bank using competency
  decomposition method
• RCT Schema implemented entities specification as an
  “instance-style” frame-based ontology
   »   expressive, extensible, useful for reporting and interpretation
• Mapping of RCT Schema terms to CTO terms makes RCT
  Bank instances available for CTO-driven integration and
  reasoning



                                                                  Copyright 2007, Ida Sim

								
To top