Listening to the Learners:
A Case Study in Health Information
Website Design
Jennifer Turns
Acting Assistant Professor
Technical Communication
Tracey Wagner
Educational Technology Analyst
Program for Educational Transformation Through Technology
Acknowledgements: This work has been supported by the Program for Educational Transformation Through Technology (PETTT).
Many people have contributed to this work including Scott Macklin, Aaron Louie, Kristen Shuyler, Brett Shelton, Kristina Liu, Alice
Tanada, Jake Burghardt, Julianne Fondiller, Regina Yap, Ralph Warren, and Dr. Frederick Matsen.
Background - Case Study - Results – Implications - Conclusion
Today’s Goal
Demonstrate how a characterization of users as learners can
inform the design of an informational website.
– Background
– Case Study
– Results
– Implications
– Conclusions
Background - Case Study - Results – Implications - Conclusion
Informational Website Design
• Examples: • Issues
– legal information – Type of content
– cooking information – Organization of
– home repairs and content
renovations – Style of content
– Access to content
Background - Case Study - Results – Implications - Conclusion
Audience Analysis
• ―Know thy user‖
• Common dimensions include
– Role – Dominant persona of users (job, affiliation)
– Goals – Reason for the interaction
– Knowledge – The extent and nature of prior relevant knowledge
– Circumstances of Use – Setting, resources, strategy, timing
– Culture – Group level beliefs, language, preferences
– Ergonomics – Relevant perceptual & motor abilities, skills
Background - Case Study - Results – Implications - Conclusion
When users are learners
• Learner centered design (Soloway et al, 1994)
• Key Challenges
– Diversity: learners are diverse
– Engagement: learners need support for
engaging in learning
– Growth: learning changes the learner
Background - Case Study - Results – Implications - Conclusion
Design Relevance?
• What is the nature of diversity, engagement,
and growth in a specific domain and with a
specific user populations?
• How can we characterize users along these
dimensions?
• How do we design informational web sites
to address these dimensions?
Background - Case Study - Results – Implications - Conclusion
Arthritis Source
• Origin: Developed in
1995 by Dr. Rick Matsen
• Goal: Help learners-at-
large satisfy their
knowledge needs with
authorized information.
• Challenges: Expand scope
and ensure effectiveness of
Arthritis Source
Background - Case Study - Results – Implications - Conclusion
Audience Analysis Methods
• Challenge
– Getting to actual users
• Our methods include
– Online Survey
– Phone Interviews
– Logfile Analysis
Background - Case Study - Results – Implications - Conclusion
Diversity
• Users with many roles • Many goals / questions
– Condition information (54%)
– Ways to minimize pain (7%)
Other
20%
– Seek clarification or a second
opinion on diagnoses (5%)
Student
1%
– Preparing for surgery (5%)
Researcher – Desire to better control arthritis
2% condition (2.5%)
Medical – …
Professional
5%
Relation
10%
Person with
Arthritis
n=462/472 62%
Background - Case Study - Results – Implications - Conclusion
Diversity
• Educational spread • Misconceptions
– Bone spurs cause arthritic
Doctorate
5% pain
Masters
15% – Joint injections are helpful
Secondary
35%
– Low bone density is
associated with Osteoarthritis
– There is little you can do
– (and 3 others)
Bachelors
22%
Associates
23%
n=399/472
Background - Case Study - Results – Implications - Conclusion
Diversity
International Use
- 128 countries
in 2½ months
Background - Case Study - Results – Implications - Conclusion
Engagement
– Motivated learners: 48% from search engine
– Personally engaged: 65% have arthritis
– Heavily one-time visitors: 80% first visit
– Moderate time on page: avg. visit ~ 7 mins.
– Common misconceptions and questions suggest
a need for longer engagement periods and
deeper interaction with information
Background - Case Study - Results – Implications - Conclusion
Growth
– Windows into growth
• Many statements of information needs
• Users at different points in patient careers
– Opportunities for growth
• Efforts to articulate questions
– Ongoing interest in exploring Growth
Background - Case Study - Results – Implications - Conclusion
Design Challenges
• Diversity
– Support users with variety of questions/goals
– Support users in different contexts
– Support users with different knowledge
• Engagement
– Help users find relevant information quickly
– Find ways to engage users in knowledge that might be
useful to them (e.g., correcting misconceptions)
• Growth
– Support user concerns and knowledge over time
Background - Case Study - Results – Implications - Conclusion
Design Direction – Content
• Content via Templates
– Common organization
– Linked to questions
– Ensures coverage
• Links to
– Diversity
– Engagement
– Growth
Background - Case Study - Results – Implications - Conclusion
Design Direction – Search
• Question-based Search
– Start with question
– Choose from ―similar‖
questions
– Enter content at point where
question is answered.
• Links to
– Diversity
– Engagement
– Growth
Background - Case Study - Results – Implications - Conclusion
What we are working on
• Building on these strategies
– Expanding content
– Improving search
• Evaluation Studies – Is it working?
– User evaluation of content
– ―Longitudinal‖ Study - Impact of extended site
interaction on knowledge and health outcomes
Background - Case Study - Results – Implications - Conclusion
Moving in the Right Direction:
Evidence of Growth?
• Sequences of searches showing growth?
• 8:26 am "my husbands leg bone below knee is
splinering and knee has moved over it,is there a
surgery to save his leg"
• At 9:22 am "aproximately 6inches below knee cap
the long bone is spintering is there a surgery that can
cover this and help support the leg bone―
• 10:01am "can a femur tibia splintering of the bone
be surgicially be repaired ?―
Background - Case Study - Results – Implications - Conclusion
Concluding Remarks
• Learner-Centered Design - Diversity, engagement, growth
• Learner-Centered Design and Audience Analysis
– One strategy for audience analysis
– Complements other audience analysis approaches (e.g., roles, tasks)
• Learner-Centered Design and Our Case Study
– Variety of data can be organized via this framework
– The dimensions are powerful organizer
– Diversity is easy to capture
– Engagement & growth are challenging to capture for web populations.
– Using this framework informed our design
Background - Case Study - Results – Implications - Conclusion
Method – Logfile Analysis
• Analyze record of users interactions with site
• Available data include
– Entry/exit pages, pages viewed, time on page, country of
origin, etc.
Table 1: Sample log file data
1/1/96 0:09:30 dial18.chemek.cc.or.us. :bonejoint:kkakkkkk2_1.html
1/1/96 0:09:32 dial18.chemek.cc.or.us. :bonejoint:gif:Clip.GIF
1/1/96 0:10:03 dial18.chemek.cc.or.us. :bonejoint:mov:ScopeACLTear.mov
1/1/96 0:10:47 dial18.chemek.cc.or.us. :bonejoint:mov:ACLgraft.mov
1/1/96 0:10:56 dial18.chemek.cc.or.us. :bonejoint:Arthritis.idx.html
1/1/96 0:13:01 pm5-00.magicnet.net. :bonejoint:nzzzzzzz1_2.html
1/1/96 0:15:00 pm5-00.magicnet.net. :bonejoint:xzzzzyzz1_1.html
Background - Approach – Results - Implications
Method - Phone Interview
About Visit About Knowledge of Condition
• Could you tell me about your • Could you tell me what you think
visit or visits to the Arthritis arthritis is in general?
Source? • Could you tell me how RA/OA
• Could you tell me what you affects the body?
were trying to do when you • Do you know what contributes to
visited the Arthritis Source? getting RA/OA?
• Did you benefit from your visit • Do you know how RA/OA is
or visits to the Arthritis Source? diagnosed? If no, Do you
• What kind of information do remember what your doctor told
you think other arthritis patients you about your diagnosis?
should know? • What is most difficult to
understand about RA/OA?
Background - Case Study - Results – Implications - Conclusion
Method – Online Survey
• Questions: Adaptive, ~25 questions
• Participants
– Duration: 9/1/2000 – 7/2/2001 (10 months)
– 472 respondents / 710 starts
• Analyses
– Descriptive Statistics
– Content Analysis
– Qualitative Coding
– Statistical Analysis
Background - Case Study - Results – Implications - Conclusion
What we are working on
• Supporting goals (finding/using content)
• Support evolving content
• Evaluating effects
– Learning
– Health outcomes
– Use patterns
– Various changes over time—growth
Background - Case Study - Results – Implications - Conclusion
Method – Online Survey
• Questions: Adaptive, ~25 questions
– Demographics
– Role
– Goal
– Successful during today’s visit
• Participants
– Duration: 9/1/2000 – 7/2/2001 (10 months)
– 472 respondents / 710 starts
Background - Case Study - Results – Implications - Conclusion
Method - Phone Interview
• Questions:
– About your visit (5): Could you tell me what you were trying
to do when you visited the Arthritis Source?
– Knowledge of condition (5): Could you tell me how RA/OA
affects the body?
• Participants – 20 users (10 OA, 10 RA)
• Analyses
– Conceptions/misconceptions
– Overarching Goals
– Specific Information Needs
Background - Case Study - Results – Implications - Conclusion
Method – Logfile Analysis
• Analyze record of users interactions with site
• Available data include
– Entry/exit pages, pages viewed, time on page, country of
origin, etc.
Sample Logfile Data: 2 requests
cpe-65-172-233-177.sanbrunocable.com [29/Apr/2002:00:01:07 -0700]
"GET /shoulder_elbow/shoulder/roughness/management/exercise/02/exercise06
HTTP/1.1" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312461)"
cpe-65-172-233-177.sanbrunocable.com [29/Apr/2002:00:01:11 -0700]
"GET /shoulder_elbow/shoulder/roughness/management/exercise/02/exercise05
HTTP/1.1" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312461)"