The Delphi Technique Brian Garrod Institute of Rural Sciences University of Wales Aberystwyth What is the Delphi Technique • Qualitative research technique … • … but by sdfwerte

VIEWS: 302 PAGES: 17

									The Delphi Technique
                     Brian Garrod
      Institute of Rural Sciences
 University of Wales Aberystwyth
What is the Delphi Technique?
• Qualitative research technique …
• … but with quantitative elements
• Relies on the judgement of a panel of experts
• Iterative process, taking place over a number of
• Much admired:
    ¤   Very flexible
    ¤   Good at getting beneath the surface of issues
    ¤   Good at addressing difficult or „expert‟ questions
    ¤   More structured than conventional interviewing
• But also much criticised:
    ¤   Often dismissed as being „discredited‟
    ¤   Accused of being anti-democratic/anti-participatory
    ¤   Reputation tarnished by sloppy execution in early days
The Delphi Technique
• Name eludes to the Oracle of Ancient Greece
• First used in 1950s by RAND Corporation in the USA
• Cold war application
• Panel of experts formed
• Question related to likely
  number of USSR nuclear
• Mean and standard deviation
• Feed back between rounds
• Repeated until „consensus‟ approached
The Delphi Technique
• What is the Delphi Technique?
  „a systematic method of collecting opinions from a group
  of experts through a series of questionnaires, in which
  feedback on the group’s opinion distribution is provided
  between question rounds while preserving the anonymity
  of the respondent’s responses.‟ (Helmer, 1972, cited in
  Masser and Foley, 1987: 217-218)
• Key features
   ¤ Systematic
   ¤ Questionnaire
   ¤ Expert opinions
   ¤ Iterative process, i.e. „rounds‟
   ¤ Feedback - individual opinions mediated by group
   ¤ Yet anonymity of individuals
The Delphi Technique
Example applications
• Technological forecasting, e.g. medicines in the 1990s
• Demand forecasting, e.g. Hawaii tourism in the Year 2000
• Forecasting changes in hotel management in Hong Kong
• Marketing of the island of Gozo separately from Malta
• Environmental Impact Assessment of development of
  Salt‟s Mill
• The setting of priorities in managing heritage attractions
• Definitions of „ecotourism‟
The Delphi Technique
Example: “Managing Heritage Tourism” (Garrod and
Fyall, 2000)
• Follow-up study to a mail-based survey of 300 managers
  of heritage attractions
• Delphi used to explore some intriguing issues in greater
• Survey asked respondents if they
  were willing to be involved in a
  follow-up study
• Panel members recruited from
  among those who said yes
The Delphi Technique
• 17 panel members
   Heritage management consultant      2
   Local authority officer             2
   Heritage organisation officer       2
   Historic property manager           6
   Academic                            3
   Museum officer                      2
• Three topic areas
   ? What should be the major priorities in the mission of heritage
   ? What factors are most likely to influence your admission pricing
     policy over the next decade
   ? What should be the funding priorities of major heritage
     organisations such as the National Trust and English Heritage?
The Delphi Technique
• The rounds
   > Round 1 – establish themes
   > Round 2 – assess themes and place in rank order
   > Round 3 – assess ranking and re-rank
• Example – Heritage attraction mission
                      Second Round Third Round         Change
  Conservation               1          1                -
  Accessibility              2          2                -
  Finance                    3          4                ↓
  Education                  4          3                ↑
  Quality                    5          5                -
  Relevance                  6          6                -
  Recreation                 7          7                -
  Local Community            8          8                -

                                                   Rs = 0.9462
The Delphi Technique
   Panel membership

                          Round:      1    2    3
    Heritage management consultant    2    2    1
    Local authority officer           2    2    2
    Heritage organisation officer     2    2    2
    Historic property manager         6    5    5
    Academic                          3    2    2
    Museum officer                    2    2    2
                         Total:      17   15   14
Strengths of the Delphi Technique
 The technique is flexible enough to be applied in a
  variety of situations and to a wide range of complex
  problems, for which there is often no other suitable
  means of analysis
 The iterative approach allows experts to reconsider their
  judgements in the light of feedback from peers
 The process also gives participants more time to think
  through their ideas before committing themselves to
  them, leading to a better quality of response
 The anonymity of the approach enables experts to
  express their opinions freely, without institutional
  loyalties or peer group pressures getting in the way
Strenghts of the Delphi Technique
 The potential influence of personality is also removed in
  this way
 Redundant „noise‟ (issues that tend to side-track the
  debate) can be controlled by the project manager
 The process generates a record of the group‟s thoughts,
  which can be reviewed as required
 The method can be used to evaluate the spread of
  opinion as well as consensus points.
Weaknesses of the Delphi Technique
 Delphi can be extremely sensitive to:
    the level of panellists‟ expertise
    the composition of the panel
    the clarity of the questions
    the way in which the project manager reports reasons for outliers
    the administration of the questionnaire
 It assumes that experts are willing to allow their
  judgements to be re-formed by the opinions of others
 Expert panel is vulnerable to high rates of attrition due to:
    boredom with the subject matter
    disillusionment with the process, and/or
    lack of time to complete the questionnaire before the following
     round commences
Weaknesses of the Delphi Technique
 Some Delphi practitioners use monetary payments or
  moral persuasion to encourage panellists to stay the
  course; however, this may bias the results of the study
 There is also a risk of „specious consensus‟ being
  formed, whereby panellists acquiesce and conform to
  the median judgement („group think‟)
 Where consensus is being sought there is a problem in
  determining what actually constitutes consensus
 The technique often requires a substantial period of time
  to complete and can be costly in terms of the
  researcher‟s time
Proposed Best-Practice Guidelines
1. The Delphi technique should not be seen as a main tool
   of investigation but a means of supporting/extending
   studies which better established and more reliable
   methods of investigation.
2. The topic must be appropriate, for example there must
   be no widely-perceived „correct answers‟ to the
   questions posed.
3. Questions must be pilot-tested to avoid ambiguity
4. Panellists should be recognised experts in their field (a
   self-assessment selection procedure may be useful in
   this respect).
5. The panel should comprise a good balance of different
   disciplines and areas of expertise.
6. Adequate time must be given to experts to think deeply
   about the questions at hand.
Proposed Best-Practice Guidelines
7. Once a subsequent round has commenced, those
    completing the previous round late should nevertheless
    be excluded from continuing.
8. Criteria for panel balance should be set in advance.
    Should these no longer be met the study should be
9. Attrition of the panel may be minimised by selecting
    experts who already have a strong interest in the
    outcome of the project.
10. This is preferable both to using monetary payment and
    moral persuasion as a means of ensuring that experts
    remain committed to the project.
11. Experts must also believe that the Delphi technique is a
    valid way of going about the task at hand.
Proposed Best-Practice Guidelines
12.Full anonymity must be preserved at all times between
  the panellists (but not necessarily between the panellists
  and the coordinating researchers).
13.The coordination group should make themselves
  available as a resource for locating further information
  on specific subjects or clarifying the questions.
14.The coordination group should intervene in the process
  as little as possible.
15.The panellists must do the initial scoping themselves,
  the coordination group should not set the agenda for
  discussion (although they will have to determine the
  research questions that will need to be answered
  through this process).
16.Where consensus is being sought, the coordination
  group should determine the criteria for bringing the
  consensus rounds to a close before the project begins.
• „Quick and dirty‟?
• Delphi – a warning from history!
• Need to establish best practice

To top