Docstoc

271 - PowerPoint

Document Sample
271 - PowerPoint Powered By Docstoc
					         Consumer Reports WebWatch
              Trust or Consequence
                Berkeley, California
                   June 9, 2005



Rating Health Information Websites


        Peter G. Goldschmidt, President
         Health Improvement Institute
              Bethesda, Maryland
Presentation
1. Introduction
    – HII-CRW partnership
    – Independent ratings
    – Project/ratings philosophy
    – Types of health websites

2. Methods
3. Results
4. Conclusions




                                   2
1A HII-CRW Partnership
Health Improvement Institute
   1990, founded; non-profit 501(c)3 education/research
    charitable organization
   1991, established Aesculapius Awards for excellence in health
    communication
   1997, held workshop on quality of health information on the
    Internet
   2003, entered into partnership with CU/CRW


Purpose/objectives regarding health websites
   To enable consumers to be more effective users
   To provide consumer with independent ratings

Medical Library Association is actively supporting project


                                                                3
1B Independent Ratings
Many past/present activities
   Criteria
    – American Medical Association
    – Mitretek Systems
   Codes of conduct
    – eHealth Code of Ethics
    – HI-ETHICS
   Trustmarks/seals
    – Health on the Net Foundation (HONcode)
    – URAC accreditation


Growing literature on criteria, quality of health websites




                                                             4
1C Project/Ratings Philosophy
Health information websites must be
 Transparent regarding                 Editorially adequate, for
   – Identity, contact information       example, disclose & be
                                         appropriate regarding
   – Ownership
                                          – Providing authors' credentials
   – Privacy (or its invasion)
                                          – Selecting/grading & editing
 Easy to use regarding                     contents
   – Finding/navigating contents on       – Resulting content
     website
                                              ° Accurate, complete,
   – Comprehending contents (clarity            objective, balanced
     of writing, reading level, etc)
                                              ° Well-organized
   – Printing contents of interest
                                              ° coherent/clear; not muddled
   – Serving consumers with special
     needs                                    ° Referenced

 Meticulous in distinguishing          Current
  advertising/selling from              Useful to consumers
  contents



                                                                             5
1D Types of Health Websites

   Health communication                 On-line product
    –   Health information                sales/marketing
    –   Decision support tool             –   On-line pharmacy
    –   Health ratings                    –   On-line store
    –   Health information resource       –   Health product marketing
    –   Health website search
        engine                           Health care organization
    –   Health advice on-line             –   Health plan
                                              enrollment/transaction
   Behavior modification                 –   Health care provider
    –   Behavior self-help                –   Producer of health resources
    –   Disease management                –   Public health program
                                          –   Other health care organization




    To start, focus is "health information websites"


                                                                               6
2 Health Website Rating Methods

                                                           Select
 Enter into      Select
                                                          additional
partnership    websites to
                                                        websites to be
 HII-CRW        be rated
                                                            rated


                  Develop
              health website   Rate health    Refine    Rate additional
                   rating       websites     methods      websites
                instrument
  Develop
 concepts/
 approach
                Credential/                   Analyze    Rerate rated
               select raters                 feedback     websites



                                                          Add/update
              Design ratings      Display
                                                        ratings website
                website           ratings
                                                            displays




                                                                          7
2A Ratings Concepts/Strategy
3 levels for health information websites
   Transparency/accountability
   Editorial adequacy
   Information reliability — completeness/accuracy of
    what is stated in any medium for any audience —
    excluded from present project because
    – Website may contain information on very many subjects
    – To rate validity of health information for given subject requires
      panel of qualified medical/research experts
    – Ultimately, assessments of reliability/validity of health
      information reflect state of medical science




                                                                          8
2B Select Websites

To start
   Define health information websites
   Identify 100 most-visited "health" websites (A. C. Nielson)
   Select top-20 "health information websites"


To continue
   Rate additional top-100 websites
   Rate health websites suggested by consumers


           Periodically, rerate rated websites




                                                                  9
2C Develop Ratings Instrument

Consumer Reports WebWatch          Health Improvement Institute

 Developed general instrument      Analyzed/assessed criteria sets
  applicable to rating all types     intended to evaluate health
  of websites (CRW principles)       websites
 Adapted CRW instrument to         Compiled list of (generic &
  meet project purposes              specific) criteria
 Created "Part I" after            Asked HII volunteers to
  evaluation by HII volunteers       evaluate criteria
  of adapted CRW instrument
  (& compiled generic/specific      Created "Part II" to focus on
  criteria)                          specific criteria

 Pretested/revised Part I          Piloted/refined Part II




                                                                     10
2C Contents of Ratings Instrument

I     Website transparency/accountability
II    Health information editorial policies
III   HONcode compliance
IV    Raters' feedback




                                              11
2C HWRI, Part I, CRW Principles

Developed/applied by CRW staff
   Identity
   Advertising & sponsorships
   Ease of use
   Corrections & currency
   Privacy




                                  12
2C HWRI, Part II, HII principles
Developed by HII; applied by volunteers
A. Criteria/descriptors              B. Evaluative narratives
     – Characteristics of                 – Purpose/scope & intended
        website contents                    audience
     – Accessibility of contents          – Characteristics of website
        to consumers
                                          – Accessibility
     – Editorial
        policies/procedures               – Editorial policies/procedures
     – Authors of                         – Contents
        articles/contents                 – Website's greatest strengths
     – Articles                           – Website's greatest
     – Summary scores                       weaknesses
 Optional                                – Utility of website to
     – Raters could also                    consumers
        complete Part-I (to
        facilitate construction of
        evaluative narratives)


                                                                        13
2D Credential/Select Raters

Process                              Credentialing criteria
   Call for raters                     Health professional
   Request applicants present
                                        At least 5 years continuous
    their credentials
                                         relevant experience
   Submit applicants'
    credentials to raters               Currently active in health field
    credentialing committee             Sufficient
   Credential health website            qualifications/experience to
    raters                               evaluate health websites
   Select credentialed raters for      No apparent disqualifying event
    panel
                                        After initial credentialing,
     – Type of organization in
        which employed                   satisfactory performance as rater
        currently
     – Role
     – Professional background
     – Region of country


                                                                            14
2E Rate Websites

 CRW staff applied HWRI, Part I
 Panel of HII volunteers; each applied HWRI
   – Raters signed HII "Policy on Conflict of Interest“
   – If conflict of interest, website reassigned
 HII summarized individual panel members'
  scores/assessments to produce coordinated
  ratings
 HII/CRW integrated ratings for website
  display
 CRW created ratings website displays




                                                          15
2F Create Ratings Website
 Purpose/objectives
  – To display independent health website ratings in consumer-
    friendly way
  – To solicit feedback

 Scope —website                 Ratings include
  contains                           – Website's stated purpose
   – Introductory/explanatory        – HII-CRW description of
     material                          website
   – Description of methods          – Global/attribute scores
   – Disclosures/disclaimers         – Greatest
   – Ratings page for each             strengths/weaknesses
     rated website                   – Noteworthy items




                                                                  16
3 Results

 Rating health websites
 Rated websites
 Median attribute ratings
 Distribution of ratings
 Excellent top-20 health websites
 Example of ratings webpage




 Ratings are accessible on www.healthratings.org



                                                   17
3A Rating Health Websites

Process                               Attributes
 20 most-visited health websites        Identity
 Rated for
                                         Advertising & sponsorship
   – Transparency, 2 CRW raters
                                         Ease of use
   – Editorial policies, HII raters
 14 HII raters (3-member                Corrections & currency
  panels)                                Privacy
   – 2 health practitioners              Design
   – 5 health information experts
                                         Coverage
   – 2 health education specialists
   – 5 media, production & related       Accessibility
                                          (navigation/reading level)
 Resultant ratings
   –   Excellent                         Contents
   –   Very good                         Overall rating
   –   Good
   –   Fair
   – Poor


                                                                       18
3B Rated Websites (20 most-visited)

 webmd.com             realage.com
   nih.gov               kidshealth.org
   health.yahoo.com      rxlist.com
   about.com/health      qualityhealth.com
   mayoclinic.com        healthology.com
   medicinenet.com       health.ivillage.com
   emedicine.com         medscape.com
   drugs.com             heartcenteronline.com
   intelihealth.com      healthboards.com
   pfizer.com            healthsquare.com



                                              19
3C Median Attribute Ratings

 Identity - Excellent
 Advertising & sponsorship - Excellent
   Ease of use - Good
   Corrections & currency - Fair
   Privacy – Excellent
   Design – Good
   Coverage – Very good
   Accessibility – Very good
   Contents – Very good




                                          20
3D Distribution of Ratings

                Top 20 Health Websites



    Excellent                       6    (30%)

   Very Good                    5   (25%)

       Good                                 8 (40%)

         Fair        1   (6%)

        Poor     0   (0%)




                                                      21
3E Excellent Top-20 Health Websites

 emedicine.net
 kidshealth.org
 mayoclinic.com
 medscape.com
 nih.gov
 webmd.com




 Ratings are accessible on www.healthratings.org



                                                   22
3F Website Ratings Page Example




                                  23
4 Conclusions

 Designing, producing & updating excellent
  health information websites is costly,
  complex, challenging

 Rating websites is equally challenging, but
  desirable & feasible

 6 of 20 most visited health websites, were
  rated “excellent” overall

 Quality of information is limited by state of
  medical science




                                                  24
4A Conclusions: Needed improvements
 Generally, rated websites need to improve
   – Contents - descriptions of editorial policies; also policies
     & procedures
      • Describe how select topics, search/grade information,
        develop contents, assure quality of articles/contents
      • Name authors/reviewers
      • Provide authors/reviewers’ credentials; must be
        appropriate to contents; disclose financial/other interests
      • State date last reviewed/updated
      • Refer to sources of facts/citations
      • Indicate criteria for linking to other websites
   – Design & Ease of use
   – Accessibility - especially for consumers with special
     needs
   – Currency & corrections


                                                                      25
4B Feedback
 HII/CRW welcome feedback on
   – Ratings’ utility to consumers
   – Suggestions for improving ratings website, process, criteria,
     etc.
   – Health websites to be rated

 Volunteer to rate health websites!




                                                                     26

				
DOCUMENT INFO
Shared By:
Categories:
Stats:
views:10
posted:3/9/2011
language:English
pages:26