Characterizing Audience for Informational Website Design A Case

Document Sample
Characterizing Audience for Informational Website Design A Case Powered By Docstoc
					                      Characterizing Audience for
                     Informational Website Design:
                             A Case Study

                                               Jennifer Turns, Ph.D.

                                               Assistant Professor,
                                                  Technical Communication
                                                    Faculty Affiliate
                     Program for Educational Transformation through Technology (PETTT)
                            Center for Engineering Learning and Teaching (CELT)



Acknowledgements: This work has been supported by the Program for Educational Transformation through Technology (PETTT). Many
people have contributed to this work including Scott Macklin, Tracey Wagner, Aaron Louie, Brett Shelton, Kristina Liu, Alice Tanada,
Jake Burghardt, Julianne Fondiller, Regina Yap, Ralph Warren, and Dr. Frederick Matsen.                                            1/30
   Overarching UCD Questions
• How do design teams incorporate information
  about users into their design process?

• When characterizing users/audience?
   –   Which dimensions are most significant?
   –   Which methods under which circumstances?
   –   How will the insights inform design?
   –   What are realistic expectations?



                                                  2/30
  Informational Website Design
Some of my projects in this domain…
• Arthritis Source:
      Medical information on the web
• Teaching challenges of engineering faculty:
     Information for educators
• Also
   – Legal information online
   – Information about architecture building methods


                                                       3/30
               Today’s Goal
Use a case study of audience analysis for website
design to reflect on decisions in audience analysis
• Describe Arthritis Source, informational website
• Describe our approach to audience analysis
• Present sample of results from our audience
  analysis, and their impacts on the design
• Discuss moving beyond the case study

                                                     4/30
             Arthritis Source
• Developed in 1995 by
  Dr. Frederick Matsen

• Focus on arthritis
• Authorized information
• User-centered information

• Research test bed

                                5/30
           Arthritis Source Content
• Articles as basic metaphor
   – Content is organized into articles
   – Examples: Rheumatoid Arthritis,
     Rotator Cuff Surgery, Pregnancy
• Templates underlie articles:
   – The template for each article is the set
     of questions answered in each article
   – All content is based loosely on five
     templates: conditions, surgery,
     treatment, living with, medication
   – Templates are informed by research
     on user questions
• Content is dynamically generated
                                                6/30
       Arthritis Source Authoring
• Authors
   –   Volunteer
   –   Are subject matter experts (e.g., doctors)
   –   Are not trained as technical communicators
   –   Create content for entire template or specific question
   –   Can create content online or in Word

• Authoring is actually distributed
   – Team creates templates based on user information
   – “Authors” create content
   – Administrator edits
                                                                 7/30
       Multiple Access Paths
• Multiple Access Paths
  – Navigation comes from
    article templates
  – Spotlighted content
  – Question-based search




                               8/30
 Embedded, Ongoing Evaluation
• Online survey
   – “Tell us about yourself”

• Quick polls
   – “How useful is this article”
• Online quizzes
   – “What do you know…”

• Online research studies
   – “Exploring use over time”

                                    9/30
    Arthritis Source Community
Community includes          Share responsibility for
• Users                     • Determining scope
• Technical Communication   • Creating content
  Professionals             • Evaluation quality
• Domain Experts (i.e.,
  doctors)
• Learning Scientists
• Developers
• Administrator


                                                   10/30
           Arthritis Source Timeline


       Design
                  ‘00    ‘01           ‘02

1995
       Audience
       Analysis



                                             11/30
            Audience Analysis
• Goals
  – Inform our own design and evaluation
  – Contribute to broader discussion


• Decisions
  – Dimensions?
     • Inform design, Speak to team, Theoretical traditions?
  – What methods?
     • Breadth/Depth, Acknowledge distributed nature of users


                                                                12/30
  Multidisciplinary Influences
• Dimensions                 • Theoretical Perspectives
  –   Roles                     –   Technical Communication
                                –   Reader Response Theory
  –   Goals
                                –   Cognitive Science
  –   Knowledge                 –   Constuctivism
  –   Circumstances of Use      –   Distributed Intelligence
  –   Culture                   –   Situated cognition
  –   Ergonomics                –   Socio-Cultural Theory
                                –   Human Factors




                                                               13/30
            Defining Dimensions

• Role – Dominant persona of users (job, affiliation)

• Goals – Reason for the interaction

• Knowledge – The extent and nature of prior relevant knowledge

• Circumstances of Use – Setting, resources, strategy, timing

• Culture – Group level beliefs, language, preferences

• Ergonomics – Relevant perceptual & motor abilities, skills
                                                                  14/30
             Method – Online Survey
  • Questions: Adaptive, ~25 questions

  • Participants:
       – Duration: 9/1/2000 – 7/2/2001 (10 months)
       – 472 respondents / 710 starts

  • Analyses1 –
       –   Descriptive Statistics
       –   Content Analysis
       –   Qualitative Coding
       –   Statistical Analysis
1Acknowledgments:   Tracey Wagner, Kristina Liu, Alice Tanada, Kristen Schuyler   15/30
         Method - Phone Interview
           About Visit                 About Knowledge of Condition

• Could you tell me about your         • Could you tell me what you think
  visit or visits to the Arthritis       arthritis is in general?
  Source?                              • Could you tell me how RA/OA
• Could you tell me what you             affects the body?
  were trying to do when you           • Do you know what contributes to
  visited the Arthritis Source?          getting RA/OA?
• Did you benefit from your visit      • Do you know how RA/OA is
  or visits to the Arthritis Source?     diagnosed? If no, Do you
• What kind of information do            remember what your doctor told
  you think other arthritis patients     you about your diagnosis?
  should know?                         • What is most difficult to
                                         understand about RA/OA?
                                                                            16/30
                         Phone Interview
  • Participants – 20 users (10 OA, 10 RA)

  • Analyses1
       – Conceptions/misconceptions
       – Overarching Goals
       – Specific Information Needs



1Acknowledgments:   Tracey Wagner, Kristina Liu   17/30
         Mapping Data to Categories
                                                        Circum-
                                                       stances of              Ergo-
Data and Sources              Role   Goals   Knowledge    Use       Culture   nomics
Online Survey
 Visitor Type                 XXX               XX                    X        X
 Age                                  X         X                     X        XX
 Home Community                                             X         XX
 Geographical Area                                          X         XX
 Type of Arthritis                    X                                        XX
 Level of Education                             XX
 Time since diagnosis                 X         X
 Name of Condition
 Why visiting                 XX     XXX                   X
 Came in from                                              XX
 Use of site in past                  XX                   XX
 Sources of information                          X         XX
Phone Interview
 Knowledge of Condition                        XXX
 Goals                               XXX
 Specific Information Needs          XXX




                                                                                       18/30
             Results - Overview
• What we learned…
   – 19% international (culture)
   – 26% rural (circumstances of use)
   – 21% over 60 (ergonomic, through vision implications)

• Highlight specific examples where results had
  identifiable impact on design
   –   Role
   –   Goals
   –   Knowledge
   –   Circumstances of Use
                                                            19/30
                                       Roles
      • Users with many roles              • “Person with arthritis”
                 Other
                                             is too simplistic…
                 20%
                                               – Person with pain
     Student
       1%                                      – Person with condition
 Researcher
    2%
                                                 that they do not
  Medical                                        consider arthritis
Professional
    5%
                                               – Person who is
      Relation
       10%                                       exploring whether they
                                                 have arthritis
                         Person with
                          Arthritis
                            62%
  n=462/472
                                                                          20/30
    Role – Design Implications
• Added new types of information
  – Differential diagnosis

• Identified writing guidelines…
  – Avoid statements that assume reader has the
    particular condition… (avoid -- “your
    condition” or “you need to…”)



                                                  21/30
  Knowledge – Misconceptions?
• From interview data, we identified several
  possible misconceptions:
  – Low bone density is associated with Osteoarthritis
  – Not drinking enough milk increases the risk of Osteoarthritis
  – Bone spurs cause arthritic pain
  – Osteoarthritis causes bone erosion.
  – OA is caused by the "wear and tear" of the joints due to overuse
    and aging.
  – There is little you can do




                                                                       22/30
Knowledge – Design Implications
• Added material to confront misconceptions

• Added “common myths” question to
  generic templates

• Developed Osteoarthritis knowledge quiz
  – 2513 Respondents for question 1
  – 470 Respondents for entire 7-question quiz

                                                 23/30
           Goals – Social Support?
Description                                                   Percentage
                                                                (n=458)
1. No indication for social support                              75%

2. Implicit desire for social support                            20%
   (mostly in the form of direct arthritis questions raised
   and requests for other links/sites)

3. Seeking doctor’s (or hospitals) referrals                     2%
4. General desire to talk to people                              3%
5. Request for live interaction on the site                      0%
6. Seeking local, face-to-face support                           0%


                                                                           24/30
    Goals – Design Implications
• On providing access to social support
   – Reduced priority for this within our system
   – Added content to help users find to support


• On learning from a surprise
   –   20% of statements were direct questions
   –   Indicates need to help users start with own questions
   –   Question-based search permits this directly
   –   Templates are based on user questions


                                                               25/30
         Circumstances of Use - Virtual
     • Coming from…                      • Implications

                Bookmark                    – Improve placement of
       Other       7%
       18%                                    site on search engines
                           Website
                            22%
                                            – Orient users coming
                                              from search engines -
                                              accelerated
                           Referred by
                               6%
                                              implementation of the
                                              navigation based on the
Search Engine                                 article templates.
    47%            n=372/472



                                                                        26/30
   Ongoing Audience Research
• Ongoing analysis of user questions (goals)
  – From emails
  – From question-based search
  – Collected earlier…


• Studying use over time through user online
  journaling (goals, circumstances of use,
  knowledge and learning)

                                               27/30
          Where we’ve been
• Starting Point:
     When characterizing users/audience…
     • Which dimensions are most significant?
     • Which methods under which circumstances?
     • How will the insights inform design?
     • What are realistic expectations?


• Contribution here: Case Study…

                                                  28/30
     Moving beyond case study
Framing audience analysis decisions
• Effort: Manager perspective
   – Resources to design, collect, analyze, interpret, decide


• Insight: Researcher perspective
   – Rigorousness, representativeness, triangulation?


• Impact: Designer perspective
   – Clarity of links to design, Persuasiveness of the data


                                                                29/30
   Designing Audience Analysis
• Effort: Manager perspective
   – Resources to design, collect, analyze, interpret, decide


• Insight: Researcher perspective
   – Rigorousness, representativeness, triangulation?


• Impact: Designer perspective
   – Clarity of links to design, Persuasiveness of the data

        Observation: Opportunities lie here…
                                                                30/30
 Web-based Health Information
• Site quality
   – Owner credentials, update dates (Hoffman, 2000)
• Quality of information
   – Comprehensiveness (e.g., Chen, 2000)
   – Accuracy (e.g., Chen, 2000)
   – Providing references (e.g., Hellawell, 2000)
• Findability of information
   – Time required (e.g., Gotwald, 2000)
   – Getting to real questions (e.g., Lechner, 1996)
• Need for evaluation methods
   – (e.g., Wu, 2000, Delamsthe, 2000, Charatan, 1999)

                                                         31/30
        When users are learners?
• Characteristics of learners
                                          Users
   –   Growth (Soloway, 1994)
   –   Diversity (Soloway, 1994)
   –   Motivation (Soloway, 1994)
   –   Prior Understandings (NRC, 1999)
                                                  Learners
• Thinking about implications
   –   What to know and why
   –   Knowing users over time
   –   Variations (how many to know)
   –   Time required to truly “know”
                                                             32/30