Expert Committees by lonyoo


									Expert Committees

        R. F. Shangraw, Jr., Ph.D.
        Chief Executive Officer
        Project Performance Corporation

        Methods and Approaches in Screening
        Proprietary Drug Names Public Meeting
        Renaissance Washington DC Hotel
        June 26, 2003

   Framing the Problem
   Identifying Relevant Research
   Addressing the Key Questions
   Raising Two Other Concerns
Framing the Problem

Bound the        Identify       Evaluate      Consensus
 Decision      Alternatives    Alternatives    Decision

   Using expert committees to review proposed
    proprietary drug names for potential sound alike and
    look alike confusion with existing drug names
   Using expert committees to review multifactoral studies
    on potential sound alike and look alike confusions
   Using expert committees to provide input to a single
    factor study
The Key Questions

   Is an expert committee necessary to review
    information from studies?
   How many people should staff an expert
   What credentials are important for expert
    committee members?
   Should expert committees meet in person, via
    videoconference, teleconference, email?
Identifying Relevant Research
   Psychology                       Medicine
   Sociology                         –   Appropriate and
                                          Necessary Care
   Law
    –   Jury Deliberations
                                     Group Decision-Making
    –   Expert Witness
                                      –   Delphi Method
    –   Science Court
                                      –   Nominal Group Technique
   Policy Sciences                   –   NIH Consensus
    –   Forecasting                       Development Program
    –   Game Theory                   –   RAND/UCLA
    –   Organizational Behavior           Appropriateness Method
Necessity of Expert Committees:
   Empirical research suggests group decisions are more
    consistent – individual judgments are prone to personal
    bias (McDonnell et al, 1996)
   There is some empirical evidence that groups make
    quicker and better decisions (Blinder & Morgan, 2000; Hogarth,
   However, some empirical research on group decision
    making suggests a systematic bias in the outcomes
    (Janis & Mann, 1977)
   But, a recent study found that groups are better than
    individuals in interpreting letters and number
    substitutions (Laughlin, et al., 2002)
Necessity of Expert Committees:

Use an Expert Panel When:
 Historical data does not exist or is limited
 Future events are likely to invalidate the results
  of historical analysis
 Ethical and moral factors are sufficiently
  important to the decision
Optimal Size of Expert Committees:

   Assuming specific voting rules, committee size
    is negatively correlated with decision accuracy
    (Gabel & Shipan, 2000)

   Communication quality and complexity is
    greater in small groups (n=5) than in larger
    groups (n=10) (Fay, et al., 2000)
   Limit to 12 – 15 members for effective
    functioning of the group (Shekelle, et al., 1999)
Optimal Size of Expert Committees:

  Group Decision-Making     Recommended Expert
         Method               Committee Size
RAND/UCLA                 7 – 15 Panelists
Appropriateness Method
Nominal Group Technique   8 – 12 Participants

NIH Consensus             9 – 16 Panelists
Development Program
Credentials for Committee
Participation: Research

   Need a baseline level of expertise but
    additional expertise does not improve accuracy
    (Armstrong, 1980)

   Committees should include participation from a
    variety of relevant disciplines (Fitch et al., 2001)
   Participants’ status affects group dynamics
    (Murphy, et al., 1998)
   Participants must be justifiable as "expert" on
    the matter under discussion (Jones & Hunter, 1995)
Credentials for Committee
Participation: Practice

   Establish baseline qualifications
   Review conflicts of interest
    –   No advocacy or promotional position
    –   No financial interests
   Minimize domineering personalities
   Increase diversity
Optimal Media for the Expert
Committee: Research
   Recent research focuses on decision quality,
    satisfaction, and media richness
   Computer-mediated group decision systems decreases
    group effectiveness, increases time to complete task,
    and decreases member satisfaction in the process
    (Baltes, et al., 2002)
   However, groups with a history can be just as effective
    with a computer-mediated system (Alge, et al., 2003)
   Group decision support systems lead to more complex
    communications than simple chat systems (Brandy & Young,
Optimal Media for the Expert
Committee: Research

   Adding video to an audio-only system improves
    decision quality (Baker, 2002)
   Participants are less likely to be committed to
    computer-mediated processes (Shangraw & Bozeman,

   After reviewing 200 studies, “no difference”
    between face-to-face meetings and specific
    collaborative technologies (Fjermestad & Holtz, 1998)
Optimal Format for the Expert
Committee: Practice

   Combine computer-mediated and face-to-face
    discussions if economically feasible
   Match media to the type and structure of the
Addressing the Concerns with
   Facilitator should be impartial
   Facilitator should assign the role of “critical evaluator”
    to all committee participants
   One or more participants should be rotated through the
    role of “devil’s advocate”
   If feasible, sub-divide the group to work under different
   Immediately following preliminary consensus, hold a
    “second chance” meeting to express any residual
Using a Structured Approach: The
Nominal Group Technique

   Briefing provided on topic and method
   Ideas are silently generated on paper
   Participants share one idea from his/her list
   Review and consolidate ideas
   Voting is done privately
   Votes are tabulated
Direction of Future Research

   Intense focus on the value of computer-
    mediated group decision-making system
   Specific interest in web-based, distributed
    computer-mediated group decision support
   Increasing interest in the value of combining
    expert panels with empirical, data-driven

To top