Docstoc

NATIONAL INSTITUTE FOR CLINICAL EXCELLENCE

Document Sample
NATIONAL INSTITUTE FOR CLINICAL EXCELLENCE Powered By Docstoc
					Guidance producer subject to accreditation:

British Thoracic Society




Date:          11 November 2009

Version:       1.9

Draft Accreditation Report – for consultation
Contents
1.    Accreditation recommendation ................................................................................. 4

2.    Implementation......................................................................................................... 7

Appendix A: Overview Summary Table ........................................................................... 8

Appendix B: NHS Evidence accreditation analysis........................................................ 11

Appendix C: Additional information analysed ................................................................ 15

Appendix D: Advisory Committee members, external advisers and NHS Evidence
accreditation team ......................................................................................................... 16




BTS: draft accreditation decision report

                                                                                                               Page 2 of 18
The NHS Evidence Accreditation Scheme recognises organisations that demonstrate
high standards in producing health or social care guidance. Organisations will display a
seal of approval called an Accreditation Mark for those processes that have been
granted accreditation.


The benefit of accreditation is that users of evidence will know that the information they
have accessed is of a high standard. Accredited organisations can display the
Accreditation Mark in publicity as a sign of high standard information development.
Accreditation lasts for three years.




BTS: draft accreditation decision report

                                                                                Page 3 of 18
1. Accreditation recommendation
It is proposed that the process used by the British Thoracic Society (BTS) to produce
its guidance is not recommended for NHS Evidence accreditation. This draft
decision is subject to public consultation before a final decision is made.




Background to the guidance producer

The BTS was formed in 1982 by the amalgamation of the British Thoracic Association
and the Thoracic Society. It is a registered charity and a company limited by
guarantee. The BTS is managed by its Trustees, who are the officers and members of
the Executive Committee. BTS members include doctors, nurses, respiratory
physiotherapists, scientists and other professionals with an interest in respiratory
disease.

The BTS produces guidelines to meet its objective which is to ‘improve the standards of
care of people who have respiratory diseases’. The production of guidelines is the
responsibility of the BTS Standards of Care Committee (SOCC). BTS guidance is
produced by guideline groups selected and approved by the BTS SOCC with advice
from the BTS network of Specialist Advisory Groups (SAG).



Summary

The processes for the development of BTS guidance are described in the BTS
Guideline Policy, BTS Standards of care constitution and ‘BTS, Biomedical Industries
and Commercial Sponsorship’ documents. The guidance producer stated that its
guideline policy document is not intended to be an instruction manual or step-by-step
guide. BTS processes are assessed as meeting 18 accreditation criteria, and not
meeting one. There is uncertainty around six criteria, four of which are in the rigour of
development domain, which are critical for evidence based guideline production. This is


BTS: draft accreditation decision report

                                                                                 Page 4 of 18
the reason for the recommendation not to accredit. This guidance recommendation only
applies to guidance produced solely by the BTS.

The objective and the recommendations of each piece of BTS guidance is explicitly
stated and well documented. BTS guidelines are developed by representative
multidisciplinary groups, with input from relevant professional stakeholder organisations,
potential users and patient groups. Guideline group membership is selected to ensure
that guidance is produced through a committee process. Potential bias is accounted for
by the combination of declaration of interests, guideline group composition, stakeholder
consultation and transparency in the funding mechanisms.

The guidance producer’s policy document is supplemented by the Standards of Care
Committee (SOCC) providing support and advice as needed to each guideline group to
assist it in its work and to ensure that the guideline group adheres to the requirements
of the policy document. However, because the policy manual is not a step-by-step guide
and therefore lacks detail, particularly describing the how clinical questions are
formulated and the process used to gather and synthesise information, it is
recommended that the guidance producer should not be accredited at this time. It
should be noted that this recommendation not to accredit BTS guidance is applied to
guidance that has been solely produced by the BTS; guidance produced in collaboration
with other groups is not the subject of this decision.

The methods of development and the processes used in the production of BTS
guidance should be explicitly documented. The lack of a documented process
specifically for domains 1 and 3 is the reason for the guidance producer not attaining
accreditation at this time. The guidance producer has made progress in improving the
processes underlying its guidance production, notably by appointing a Deputy Chief
Executive to support the Standards of Care Committee and updating its guideline policy
document in March 2009. This progress is recognised, but because of the lack of detail
in its current guideline policy document, the process cannot yet be recommended for
accreditation. The guidance producer’s lack of resource capacity was noted as a reason
for adopting its current approach. However the processes in areas of crucial importance
for evidence based guideline production, specifically those addressed by domain 3

BTS: draft accreditation decision report

                                                                                Page 5 of 18
(rigour of development), need to be documented fully. The guidance producer indicated
that it intends to include more detail when it updates its guideline policy document.

Areas that did not meet the required standard and where improvement is needed are:

        a detailed documented policy for guidance production

        more description on how clinical questions are formulated and patients and/or
         target audience for guidance are identified

        detailed policies that describe the identification, gathering and synthesising of
         evidence used to produce the guidance




This draft decision is now going out for consultation, and the decision will be reviewed
by the committee in the light of any feedback received before making a final
recommendation.




David Haslam

Chair, Advisory Committee

November 2009




BTS: draft accreditation decision report

                                                                                 Page 6 of 18
2. Implementation
Following public consultation a final accreditation decision will be made. If accreditation
is granted, guidance from the accredited producer will be identified on NHS Evidence by
a graphic – the accreditation mark. The accredited guidance producer is also granted a
royalty-free, worldwide licence to use the NHS Evidence accreditation mark in
accordance with the Conditions and Terms of Use
(http://www.nice.org.uk/nhsevidence/?domedia=1&mid=5AE1D938-19B9-E0B5-
D471CA81220F57DA) for the duration of their accreditation for a category of evidence.
Provided these conditions are complied with, a guidance producer's accreditation will
last for three years from when NHS Evidence's decision to award accreditation is
published on the NHS Evidence website.

Accredited guidance producers must take reasonable steps to ensure that processes
approved by NHS Evidence are followed when generating the type of evidence for
which they are accredited. Accredited guidance producers should have quality
assurance mechanisms in place to ensure compliance with accredited procedures.
Accredited guidance producers shall inform NHS Evidence of any change to a process
which may impact on the fulfilment of the relevant accreditation criteria within 30 days of
that change occurring.

Guidance producers that are not accredited following the accreditation process have the
opportunity to reapply from one year after the previous assessment. It is assumed that
the organisation will have addressed any concerns highlighted in the original
assessment before reapplying.




BTS: draft accreditation decision report

                                                                                 Page 7 of 18
Appendix A: Overview Summary Table
Domain     1     Scope and purpose is concerned with the overall aim of the guidance,      Accreditation
                 the specific clinical questions and the target population.                decision
Criteria   These criteria appraise whether the guidance producer has a policy in place
           that requires them to explicitly detail:
           1.1     The overall objective of the guidance                                    Criterion met
                                                                                               Some
           1.2     The clinical questions covered by the guidance                           uncertainties

                                                                                               Some
           1.3     The patients and/or target audience to whom the guidance applies         uncertainties

                                                                                            Criterion met
           1.4     That the producer ensures guidance includes clear recommendations
                   in reference to specific clinical circumstances.
Domain     2     Stakeholder involvement focuses on the extent to which the guidance       Accreditation
                 represents the views of its intended users.                               decision
Criteria   These criteria consider whether the guidance producer has a policy in place
           that means it includes:
                                                                                            Criterion met
           2.1     Individuals from all relevant professional groups

                                                                                            Criterion met
           2.2     Patient representatives and seeks patients views and preferences

           2.3     Representative intended users in developing guidance.                    Criterion met

Domain     3     Rigour of development relates to the process used to gather and           Accreditation
                 synthesise information and the methods used to formulate                  decision
                 recommendations and update them.
Criteria   These criteria consider whether the guidance producer has a clear policy in
           place that:
                                                                                               Some
           3.1     Requires the technical team to use systematic methods to search for
                   evidence and provide details of the search strategy                      uncertainties

                                                                                               Some
           3.2     Requires the guidance producers to state the criteria and reasons for
                   inclusion or exclusion of evidence identified by the evidence review     uncertainties

                                                                                               Some
           3.3     Describes the strengths and limitations of the body of evidence and
                   acknowledges any areas of uncertainty                                    uncertainties




BTS: draft accreditation decision report

                                                                                              Page 8 of 18
                                                                                            Criterion met
           3.4     Clarifies the method used to arrive at recommendations (for
                   example, a voting system or formal consensus techniques like Delphi
                   consensus)
                                                                                               Some
           3.5     Requires the guidance producers to balance the health benefits
                   against the side effects and risks                                       uncertainties

                                                                                            Criterion met
           3.6     Details the processes of external peer review

           3.7     Mentions the process of updating guidance and maintaining and            Criterion met
                   improving guidance quality
Domain     4     Clarity and presentation deals with the language and format of the        Accreditation
                 guidance.                                                                 decision
Criteria   These criteria appraise whether the guidance producer ensures that:

                                                                                            Criterion met
           4.1     Their recommendations are specific, unambiguous and clearly
                   identifiable

           4.2     Different options for the management of the condition are clearly        Criterion met
                   presented
                                                                                            Criterion met
           4.3     The date of search, the date of publication or last update and the
                   proposed date for review are clearly stated
                                                                                            Criterion met
           4.4     The content of the guidance is suitable for the specified target
                   audience. If patients or service users are part of this audience, the
                   language should be appropriate.
Domain     5     Applicability deals with the likely organisational, behavioural and       Accreditation
                 cost implications of applying the guidance.                               decision
Criteria   These criteria measure whether the guidance producer routinely considers:

                                                                                            Criterion met
           5.1     Publishing support tools to aid implementation of guidance

                                                                                            Criterion not
           5.2     Discussion of potential organisational and financial barriers in
                   applying its recommendations                                                 met

                                                                                            Criterion met
           5.3     That their guidance is current, with review criteria for monitoring
                   and/or audit purposes within each product.
Domain     6     Editorial Independence is concerned with the independence of the          Accreditation
                 recommendations, acknowledgement of possible conflicts of interest,       decision
                 the credibility of the guidance in general and their recommendations in
                 particular.
Criteria   These criteria measure whether the guidance producer:
                                                                                            Criterion met
               6.1 Ensures independence from the funding body

                                                                                            Criterion met
               6.2 Is transparent about the funding mechanisms for its guidance


BTS: draft accreditation decision report

                                                                                              Page 9 of 18
                                                                                          Criterion met
              6.3 Records and states any potential conflicts of interest of individuals
                   involved in developing the recommendations
                                                                                          Criterion met
              6.4 Takes account of any potential for bias in the conclusions or
                   recommendations of the guidance




BTS: draft accreditation decision report

                                                                                          Page 10 of 18
Appendix B: NHS Evidence accreditation analysis
The Advisory Committee considered the following analysis of the guidance producer’s
compliance with NHS Evidence accreditation criteria, summarised below. Appendix C
lists the additional information taken into account in the analysis and considered by the
Committee. The process for accrediting producers of guidance and recommendations
for practice is described in the process manual which can be found here
http://www.nice.org.uk/nhsevidence/aboutaccreditation/aboutaccreditation.jsp?domedia
=1&mid=27C232A0-19B9-E0B5-D4A11FA899F4C219.

    Scope and purpose
Does the guidance producer have a policy in place that requires them to specifically
detail the domain criteria?

The guidance producer meets two of the four criteria in this domain, described in the
BTS Guideline Policy, BTS Standards of care constitution document and ‘BTS,
Biomedical Industries and Commercial Sponsorship’. Evidence for the application of
these criteria was found in two examples of guidance [Recommendations for assessing
risk and for managing Mycobacterium tuberculosis infection and disease in patients due
to start anti-TNF-α treatment, 2005 and Radiologically guided lung biopsy, 2003].

The objective and the recommendations of each piece of BTS guidance is well
documented within each piece of guidance. However the lack of a detailed documented
policy, particularly in describing the how clinical questions are formulated, means that
there are uncertainties within this domain.




BTS: draft accreditation decision report

                                                                              Page 11 of 18
    Stakeholder involvement
Does the guidance producer have a policy in place that means it includes information
detailed in the domain criteria?

The guidance producer meets the criteria in this domain, described in the BTS
Guideline Policy, BTS Standards of care constitution document and ‘BTS, Biomedical
Industries and Commercial Sponsorship’. Evidence for the application of these criteria
was found in two examples of guidance [Recommendations for assessing risk and for
managing Mycobacterium tuberculosis infection and disease in patients due to start
anti-TNF-α treatment, 2005 and Radiologically guided lung biopsy, 2003].

All BTS guidelines are developed by multidisciplinary groups representing relevant
professions, relevant professional stakeholder organisations, potential users and
patient groups with an interest in the topic of the guideline.


Rigour of development
Does the guidance producer have a clear policy in place that means it includes
information detailed in the domain criteria?

Clear documented processes describe how the BTS arrives at its recommendations via
the voting systems and the peer review process. The BTS Guideline Policy document
shows that a procedure for updating the guidelines is provided and existing guidelines
are updated every 3-4 years.

Because of its available resources, the guidance producer’s process relies on
discussions between the Standards of Care Committee and the chairs of the guideline
groups to ensure that appropriate processes are followed. However, despite
acknowledging that guidance should comply with AGREE criteria, the processes used
by the BTS in searching, including and prioritising the evidence base used to produce

BTS: draft accreditation decision report

                                                                             Page 12 of 18
its guidance are unclear. The lack of a detailed documented policy, particularly around
the processes used to gather and synthesise information, means that there are
uncertainties within this domain. The processes used for searching, stratifying and
categorising of evidence on which the BTS make their recommendations are not
documented. This is of utmost importance in the production of evidence based guidance
to allow the assessment of the rigour of the development process. This lack of
documented process is the primary reason for the recommendation not to accredit.


    Clarity and presentation
Has the guidance producer submitted sufficient detailed information to ensure the
domain criteria are met?

The guidance producer meets the criteria in this domain, described in the BTS
Guideline Policy, BTS Standards of care constitution document and ‘BTS, Biomedical
Industries and Commercial Sponsorship’ document. Evidence for the application of
these criteria was found in two examples of guidance [Recommendations for assessing
risk and for managing Mycobacterium tuberculosis infection and disease in patients due
to start anti-TNF-α treatment, 2005 and Radiologically guided lung biopsy, 2003].

Recommendations are written and presented to be specific and unambiguous. The date
of the search, date of publication/last update and proposed review date are stated in
guidelines. Where there are different treatment options for different patient groups,
these are clearly identified in the recommendations.


    Applicability
Has the guidance producer submitted sufficient detailed information to evidence routine
consideration of the domain criteria?

The guidance producer publishes support tools including educational documentation,
audit tools and patient information, and makes its guidance available electronically.
Guidelines are revised or updated at least every 3-4 years to ensure they are always
current Evidence for the application of these criteria was found in two examples of

BTS: draft accreditation decision report

                                                                               Page 13 of 18
guidance where applicable [Recommendations for assessing risk and for managing
Mycobacterium tuberculosis infection and disease in patients due to start anti-TNF-α
treatment, 2005 and Radiologically guided lung biopsy, 2003].

However the guidance producer does not have a documented policy in place with
regards to barriers to implementing its recommendations.


    Reliability and trustworthiness
Has the guidance producer submitted sufficient detailed information to evidence
achievement of the domain criteria?

The guidance producer meets the criteria in this domain, described in the BTS
Guideline Policy, BTS Standards of care constitution document and ‘BTS, Biomedical
Industries and Commercial Sponsorship’. Evidence for the application of these criteria
was found in two examples of guidance [Recommendations for assessing risk and for
managing Mycobacterium tuberculosis infection and disease in patients due to start
anti-TNF-α treatment, 2005 and Radiologically guided lung biopsy, 2003].

Guideline group membership is selected to ensure that guidance is produced through a
committee process. The SOCC, and in particular the role of the Chair of the SOCC, is
responsible for ensuring that recommendations included in BTS guidelines are balanced
and fair. Potential bias is accounted for by the combination of declaration of interests,
stakeholder consultation and transparency in the funding mechanisms.




BTS: draft accreditation decision report

                                                                               Page 14 of 18
Appendix C: Additional information analysed
List of information taken into account in the accreditation analysis and considered by the
Advisory Committee.

Document name                     Description                Location
BTS Standards of Care             BTS Guideline Policy       BTS Standards of Care Committee
                                                             Guideline Policy Mar 09 update
Committee Guideline               document (Mar 09 update)
Policy
BTS Standards of Care             BTS SOCC document (Mar     StandardsofCareConsitutionMAR09


Constitution                      09)
British Thoracic Society          Full list of guidance      British Thoracic Society Guidelines List


Guidelines List                   produced (supplied)
Emergency oxygen use              BTS example guideline      http://www.brit-
                                                             thoracic.org.uk/Portals/0/Clinical%20Informa
in adult patients guideline (supplied by guidance            tion/Emergency%20Oxygen/Emergency%2

                                  producer)                  0oxygen%20guideline/THX-63-Suppl_6.pdf


Asthma guideline                  BTS example guideline      http://www.brit-
                                                             thoracic.org.uk/LinkClick.aspx?link=83&tabi
                                  (supplied by guidance      d=81

                                  producer)
Recommendations for               BTS example guideline      http://www.brit-
                                                             thoracic.org.uk/Portals/0/Clinical%20Informa
assessing risk and for                                       tion/Anti%20TNF%20Treatment/Guidelines/

managing                                                     antitnf_treatment.pdf


Mycobacterium
tuberculosis infection and
disease in patients due to
start anti-TNF-α
treatment
Radiologically guided             BTS example guideline      http://www.brit-
                                                             thoracic.org.uk/Portals/0/Clinical%20Informa
lung biopsy Guideline                                        tion/Radiologically%20Guided%20Lung%20
                                                             Biopsy/Guidelines/lungbiopsy.pdf

Guidelines and                    Full list of guidance      http://www.brit-
                                                             thoracic.org.uk/LinkClick.aspx?link=157&tab
statements list                   produced (website)         id=69




BTS: draft accreditation decision report

                                                                                        Page 15 of 18
Appendix D: Advisory Committee members, external
advisers and NHS Evidence accreditation team

NHS Evidence Advisory Committee Members

The NHS Evidence Advisory Committee operates as a standing advisory committee of
the Board of the National Institute for Health and Clinical Excellence (NICE). The
Committee provides advice to the Institute on a framework for accrediting sources of
evidence that should be recognised as trusted sources of information for the NHS. The
Chair of the Committee is appointed by the Institute’s Board and the meetings are
conducted by the Chair or in his/her absence the vice chair. The current Chair is David
Haslam. A full list of the Advisory Committee membership is available on the NICE
website (http://www.nice.org.uk/nhsevidence/nhseac.jsp). The members have been
appointed for a period of 18 months. This may be extended by mutual agreement to a
further term of 3 years and up to a maximum term of office of 10 years.

The decisions of the Committee are arrived at by a consensus of those members
present. The quorum is set at 50% of committee membership. The Committee submits
its recommendations to the Institute’s Guidance executive which acts under delegated
powers of the Institute’s Board in considering and approving its recommendations.

Committee members are asked to declare any interests in the guidance producer to be
accredited. If it is considered that there is a conflict of interest, the member(s) is
excluded from participating further in the discussions. A list of the committee members
who took part in the discussions for this accreditation decision appears below.


 Title           Name             Surname     Role                        Organisation

                                                                          Calderdale & Huddersfield
                                              Clinical Development &
 Mr              Richard          Brownhill                               NHS Trust & Kirklees
                                              Nurse Practitioner
                                                                          PCT

                                              The William Leech           Institute of Health and
 Professor       Martin           Eccles
                                              Professor of Primary Care   Society


BTS: draft accreditation decision report

                                                                                         Page 16 of 18
                                                 Research and Professor
                                                 of Clinical Effectiveness

                                                                               NHS Alliance (GP and
 Dr              Brian            Fisher         General Practitioner          national patient/public
                                                                               lead)

                                                 Director of National          NHS National Knowledge
 Sir             Muir             Gray
                                                 Knowledge Service             Service

                                                 Head of Knowledge             Social Care Institute for
 Ms              Diane            Gwynne Smith
                                                 Management                    Excellence (SCIE)

                                                 President, Royal College
                                                 of General Practitioners
 Professor       David            Haslam                                       Care Quality Commission
                                                 and National Clinical
                                                 Advisor

                                                 Director of London Health
                                                 Observatory, Vice Chair       London Health
 Dr              Bobbie           Jacobson
                                                 of Association of PH          Observatory
                                                 Observatories

                                                 Professor of Paediatric       The Peninsula College of
 Professor       Stuart           Logan
                                                 Epidemiology                  Medicine

                                                                               Royal Shrewsbury
 Ms              Catherine        Mercer         Midwife
                                                                               Hospital

                                                                               Cochrane Consumers and
                                                 Professor of Public Policy,
                                                                               Communication Review
 Professor       Sandy            Oliver         Deputy Director     Social
                                                                               Group, University of
                                                 Science Research Unit
                                                                               London

                                                 National Clinical Director,
 Dr              Sheila           Shribman       Children, Young People        Department of Health
                                                 and Maternity Services

                                                 National Director for
 Mrs             Julie            Wood           Practice Based                NHS Alliance
                                                 Commissioning




BTS: draft accreditation decision report

                                                                                           Page 17 of 18
Advisory Committee Deputies

 Title   Name       Surname    Role                              Organisation           Deputising for

                               Reader in Evidence Based
                               Information Practice & Director   School of Health and
 Mr      Andrew     Booth      of Information School of Health   Related Research
                                                                                        Jon Nichol
                               and Related Research              (ScHARR)
                               (ScHARR)

                               Head of Knowledge Sharing         North East Strategic   Stephen
 Ms      Lynda      Cox
                               and Learning, NHS North East      Health Authority       Singleton

                                                                 Moorfields Eye
                               Consultant in Opthalmology
 Ms      Parul      Desai                                        Hospital NHS
                               and Public Health                                        Mark Davies
                                                                 Foundation Trust


External Advisers for British Thoracic Society accreditation
application
Holger J Schünemann, MD, PhD, Research Associate Professor, Department of Social
and Preventive Medicine, University at Buffalo, Associate Professor, Department of
Medicine, University at Buffalo, Professor and Chair, Department of Clinical
Epidemiology and Biostatistics, Michael Gent Chair in Healthcare Research, McMaster
University, Hamilton, Ontario, Canada

António Vaz Carneiro, MD, PhD, FACP, Faculty of Medicine, University of Lisbon,
Portugal, Head, Center for Evidence Based Medicine, Executive-Director, Institute for
Advanced Education, Head, Main Library - Center for Information and Documentation

NHS Evidence accreditation team for British Thoracic Society
accreditation application
Stephanie Birtles, Accreditation Technical Analyst, NHS Evidence, National Institute for
Health and Clinical Excellence, Manchester, UK

Dr Paul Chrisp, Associate Director Accreditation, NHS Evidence, National Institute for
Health and Clinical Excellence, Manchester, UK


BTS: draft accreditation decision report

                                                                                          Page 18 of 18