Docstoc

GOMS Analysis Web Site Usability (PowerPoint)

Document Sample
GOMS Analysis Web Site Usability (PowerPoint) Powered By Docstoc
					  Empirically Validated Web Page
          Design Metrics




Melody Y. Ivory, Rashmi R. Sinha, Marti A. Hearst
UC Berkeley
CHI 2001
    The Usability Gap

    196M new Web                                               ~20,000 user
    sites in the next 5                                        interface
    years [Nielsen99]                                          professionals
                                                               [Nielson99]




2              Empirically Validated Web Page Design Metrics
    The Usability Gap

    196M new Web                                               A shortage of
    sites in the next 5                                        user interface
    years [Nielsen99]                                          professionals
                                                               [Nielson99]


                  Most sites have inadequate
                  usability [Forrester, Spool, Hurst]
                  (users can’t find what they want 39-
                  66% of the time)
3              Empirically Validated Web Page Design Metrics
    The Usability Problem

       NON-professionals need to create websites
       Guidelines are helpful, but
        –   There are MANY usability guidelines
                Survey of 21 web guidelines found little overlap [Ratner et al. 96]
                Why?
                   –    One idea: because they are not empirically validated
        –   Sometimes imprecise
        –   Sometimes conflict


4                      Empirically Validated Web Page Design Metrics
    Possible Solutions:
    Tools to Help Non-Professional Designers

       Examples:
        –   A “grammar checker” to assess guideline
            conformance
                Imperfect
                Only suggestions – not dogma
        –   Automatic comparison to highly usable pages/sites
        –   Automatic template suggestions
       How to create these?

5                   Empirically Validated Web Page Design Metrics
    Current Design Analysis Tools
       Some tools report on easy-to-
        measure attributes
         –   Compare measures to thresholds
                 Stein (Rating Game), Theng &
                  Marsden, Thimbley (Gentler)
                 Not empirically validated
         –   Guideline conformance
                 CAST (Bobby), Scholtz & Laskowski
                  (WebSAT), Lift Online
         –   Perceptually based heuristics
                 Faraday (Design Advisor)
       Small subset of features [Brajnik00]
         –   Simplistic
         –   Not empirically validated

6                     Empirically Validated Web Page Design Metrics
    The WebTANGO Approach

       Models of good design by looking at existing
        designs
        –   Empirical foundation for easy-to-measure attributes
        –   Focus on information-centric sites
       First work to take a large set of sites and
        analyze them



7                Empirically Validated Web Page Design Metrics
    The Investigation

       Using quantitative measures to predict Web site ratings
        –   Followup investigation [HFW00]
       Given
        –   1898 pages from 400+ sites
        –   11 quantitative measures to assess various Web page aspects
       Questions
        –   Which features distinguish well-designed web pages?
        –   Can metrics predict ratings?
        –   Are there differences for categories of pages?


8                  Empirically Validated Web Page Design Metrics
    Webby Awards 2000

       2000 sites initially
        –   27 topical categories
                We studied sites from information-centric categories
                   –    Finance, education, community, living, health, services

       100 judges
        –   International Academy of Digital Arts & Sciences
        –   3 rounds of judging



9                      Empirically Validated Web Page Design Metrics
     Webby Awards 2000
        6 criteria
          –   Content
          –   Structure & navigation
          –   Visual design
          –   Functionality
          –   Interactivity
          –   Overall experience
        Scale: 1-10 (highest)
        Nearly normally distributed
        What are judgments about?


10                    Empirically Validated Web Page Design Metrics
     Webby Awards 2000

        Content criterion is best
         predictor
        Visual design criterion is
         worst predictor
        User study of 57 sites
         –   Ratings reflect usability




11                  Empirically Validated Web Page Design Metrics
     Quantitative Measures:
     Aspects Impacting Usability

        Identified 42 attributes from the literature
        Roughly characterized:
         –   Page Composition
                 words, links, images, …
         –   Page Formatting
                 fonts, lists, colors, …
         –   Overall Page Characteristics
                 information & layout quality, download speed, …


12                   Empirically Validated Web Page Design Metrics
     Quantitative Measures:
     Word Count




13          Empirically Validated Web Page Design Metrics
     Quantitative Measures:
     Body Text %




14          Empirically Validated Web Page Design Metrics
     Quantitative Measures:
     Emphasized Body Text %




15          Empirically Validated Web Page Design Metrics
     Quantitative Measures:
     Text Positioning Count




16           Empirically Validated Web Page Design Metrics
     Quantitative Measures:
     Text Cluster Count




17           Empirically Validated Web Page Design Metrics
     Quantitative Measures:
     Link Count




18           Empirically Validated Web Page Design Metrics
     Quantitative Measures:
     Page Size (Bytes)




19           Empirically Validated Web Page Design Metrics
     Quantitative Measures:
     Graphic %




20          Empirically Validated Web Page Design Metrics
     Quantitative Measures:
     Graphic Count




21           Empirically Validated Web Page Design Metrics
     Quantitative Measures:
     Color Count




22           Empirically Validated Web Page Design Metrics
     Quantitative Measures:
     Font Count




23           Empirically Validated Web Page Design Metrics
     Characterizing Measures:
     A View of Web Site Structure                                   [Newman et al. DIS00]




        Information design
         –   structure, categories of information
        Navigation design
         –   interaction with information
             structure
        Graphic design
         –   visual presentation




                                                                              Courtesy of Mark Newman
24                  Empirically Validated Web Page Design Metrics
Characterizing Measures:
Web Site Structure Assessed
     Study Method

        The Webby factor
         –   Principle components analysis of the 6 criteria
                 Accounted for 91% of the variance
        Two comparisons
         –   Model 1: Highly rated sites (top 33%) vs. the rest
                 Using the overall Webby score
         –   Model 2: Highly rated sites vs. bottom 33%
                 Using the Webby factor


26                   Empirically Validated Web Page Design Metrics
     Findings

        We can accurately classify web pages
         –   Linear discriminant analysis
         –   Model 1: For highly-rated sites vs. rest
                 67% correct when not considering content categories
                 73% correct when taking content categories into account
         –   Model 2: For highly-rated sites vs. bottom
                 65% correct when not considering content categories
                 80% correct when taking content categories into account



27                   Empirically Validated Web Page Design Metrics
     Findings

        Top vs.
         bottom
         –   Webby factor
        Linear
         discriminant
         analysis
        Better for
         categories



28                 Empirically Validated Web Page Design Metrics
     Deeper Analysis

        Which metrics matter?
         –   Linear regression analysis
                 Backward elimination until adjusted R² reduced
         –   All metrics played a role
        Compared small, medium, and large pages
         –   Across the board (preliminary profiles)
                 Good pages had significantly smaller graphics percentage
                 Good pages had less emphasized body text
                 Good pages had more colors (on text)


29                   Empirically Validated Web Page Design Metrics
     Small pages (66 words on
     average)
        Good small pages have                               Examples
         (according to beta
         coefficients)                                         –     Services (Home Pages)
         –   Slightly more content                                       Top
         –   Smaller page sizes                                          Bottom
         –   Fewer graphics
         –   More font variations
        Suggests that these pages
         –   Have faster download times
                 corroborated by a download
                  time metric
         –   Use different fonts for
             headings vs. the rest of the
             text
30                   Empirically Validated Web Page Design Metrics
     Medium pages (230 words on
     average)

        Good medium pages
         –   Emphasize less of the body text
         –   Appear to organize text into clusters (e.g., lists and
             shaded table areas)
         –   Use colors to distinguish headings from body text
        Suggests that these pages
         –   Are easier to scan



31                 Empirically Validated Web Page Design Metrics
     Large pages (827 words on
     average)

        Good large pages have
         –   More headings
         –   More links
         –   Are larger but have fewer graphics
                 Probably attributable to style sheets
        Suggests that good large pages
         –   Are easier to scan and facilitate information seeking



32                   Empirically Validated Web Page Design Metrics
     Why does this work?

        Content is most important predictor
        BUT there’s predictive power in other aspects
         –   Visual and navigation design
         –   Verifies preliminary investigation                    [HFW00]

        Possibly: Good design is good design all over
        Note: we are NOT suggesting we can
         characterize:
         –   Aesthetics or subjective preferences


33                 Empirically Validated Web Page Design Metrics
     How might we use this?

     Web Site Design                                            Comparable Designs
                                           Profiles


                                           Analysis
                                           Tool
     Favorite Designs
                                                •Prediction
                                                •Similarities
                                                •Differences
                                                •Suggestions
34              Empirically Validated Web Page Design Metrics
     Future work

        Distinguish according to page role
         –   Home page vs. content vs. index …
        Better metrics
         –   More aspects of info, navigation, and graphic design
        Site level as well as page level
        Category-based profiles
         –   Use clustering to create profiles of good and poor sites
         –   These can be used to suggest alternative designs



35                  Empirically Validated Web Page Design Metrics
     In Summary

        Automated tools should help close the Web Usability
         Gap
        We have a foundation for a new methodology
         –   Empirical, bottom up
        We can empirically distinguish good pages
         –   Empirical validation of design guidelines
         –   Can build profiles of good vs. poor sites
        Eventually build tools to help users assess designs



36                  Empirically Validated Web Page Design Metrics
     More Information

        http://webtango.berkeley.edu




37              Empirically Validated Web Page Design Metrics

				
DOCUMENT INFO
Shared By:
Categories:
Stats:
views:27
posted:6/19/2011
language:English
pages:37