User Interface Design and Evaluation - PDF by ddh19362

VIEWS: 169 PAGES: 36

									User Interface Design and 
        Evaluation




                             1
                Administrivia
  What ifs
• What ifs
  – What if the TAs tell me to focus on something else 
    after the poster session?
  – What if I run my project by our users and they 
    hate it?
  – What if my project doesn’t include any nifty 
    technology?




                                                      2
User Interface Design and 
        Evaluation




                             3
               Quick Review
               Quick Review
  Instrumental Interaction
• Instrumental Interaction
  – Degree of indirection
    Degree of integration
  – Degree of integration
  – Degree of compatibility
  Mental Models
• M t lM d l




                              4
              Models of System
              Models of System
  Designer s Model
• Designer’s Model
  – How the designer of system believes system should be 
    used
• System image
  – The system itself
• User’s Model
  – How the user of a system believes system should be used
• Designer and User communicate via system
  – Goal is to have both images align as closely as possible


                                                               5
             Aligning Models
             Aligning Models
  Guiding usage
• Guiding usage
  – Done by applying certain design principles to UI
    Allows users to perceive what should be done, to 
  – Allows users to perceive what should be done to
    map action onto display
  Evaluating system design
• Evaluating system design
  – Beginning evaluation
    Project phase 3
  –P j t h        3



                                                        6
            UI Design Principles 
        (from Preece, Rogers, Sharp)
        (f                     h )
•   Affordance 
    Affordance
•   Mapping
•   Constraints
    C        i
•   Visibility/Feedback
•   Consistency




                                       7
                    Affordance

•   Attribute of an object that allows people to know how to use it

                                        • Coined by Norman
                                            – Means “to give a clue”
                                            – Book: The Design of 
                                              Everyday Things
                                          Norman’s current 
                                        • N     ’        t
    Push                Pull              argument:
                                            – Should not pay too much 
                                              attention during UI design
                                              attention during UI design
                                            – Objects have “real 
                                              affordances”
                                            – Screen widgets have 
                                              “      i d ff d
                                              “perceived affordances””
            Waterloo                        – Learned conventions
                                                                       8
  Mental Models and Affordance
  Mental Models and Affordance
  Recall thermostat examples
• Recall thermostat examples
  – My flower version
Mental Models and Affordance
Mental Models and Affordance

• Consider thermostat
  – What was wrong with my flower model?




                                           10
                          Mapping
• Physical actions performed on the device must be mapped 
  onto on‐screen effects
• Instruments operations must be mapped onto objects




  Recall instrumental interaction
• Recall instr mental interaction
   – Degree of integration
   – Degree of compatibility


                                                             11
      Physical Versus Virtual
      Physical Versus Virtual

• Some things work well in physical, not in virtual
                                             Virtual
   Physical




                              This user interface (UI) is simply hideous. 
                              Form and function have been sacrificed to 
                              F        d f ti h          b         ifi d t
                              looks, in the sense that it's appearance is 
                              mimicking a physical piece of equipment. 
                              Trying to adjust the parameters with the 
                              mouse is difficult and error‐prone, and I 
                              am not brave enough to try using the 
                              keyboard ‐ I don't even know how I would 
                              do that. 
                                                                      12
                Constraints
  Guide user by preventing certain actions, 
• Guide user by preventing certain actions
  enabling others




                                               13
              Constraints

• Physical
   – Only one way to 
     connectt
   – Disabled 
     buttons/menus
• Logical
   – My Documents, My 
     Pictures, etc.
     Pictures etc
• Cultural
   – Examples?

                            14
         Visibility/Feedback

• Recall principles of direct manipulation
                                     • Continuous 
                                       representation
                                     • Actions on objects of 
                                       interest
                                     • Fast, incremental 
                                       operations with 
                                       immediate feedback
                                     • Layered 
                                       learning/self‐
                                       revealing

                                                         15
                Consistency
  Allows users to leverage control from familiar 
• Allows users to leverage control from familiar
  onto new




                                                16
                           Summary
• Various ways we can guide usage
   – Affordances suggest functionality
   – Mappings allow users to control on‐screen objects via physical devices, 
     and to use widgets (instruments) to manipulate data (objects)
   – Constraints limit actions
   – Visibility/feedback allow users to find objects of interest and activate 
     commands
   – Consistency allows users to leverage knowledge of familiar apps when 
     beginning to work with new apps.
• How do we know if we have it right?
   – Evaluation




                                                                            17
                       Evaluation
                    yp p
• Evaluation for many purposes
• Two forms
  – Quantitative
     • Data involves numerical measures that can be contrasted 
  – Qualitative
     • Data is narrative and observational in form
• Can combine
  – Mixed methods
     • Pragmatic approach to research 
     • Data involves both observation and numerical data



                                                                  18
           Goals of evaluation
           Goals of evaluation
  Design versus implementation
• Design versus implementation
  – Formative evaluation is used during development
    Summative evaluation is used for finished product
  – Summative evaluation is used for finished product
• Can help to align models
  – Designer’s model
  – User’s mental model




                                                    19
               Goals of evaluation (2)
               Goals of evaluation (2)
•   To assess extent and accessibility of systems functionality
                   d       h?                  f        ?
     – Does system do enough? Can users access functions?
•   To assess users’ experience of interaction
     – Do they like it?  Do they understand it?
•   To identify specific problems with system
    To identify specific problems with system
     – Is something done wrong? Can aspects be improved?
•   To understand real world
     – How do users use technology?  Can design be improved, can work be 
               d          h l           l           ?
       automated, can we help a potential user group?
•   To compare designs
     – Best/better/worse Essential features
•   To engineer toward a target
    To engineer toward a target
     – Is design good enough?
•   To check conformance to a standard
     – Microsoft design guidelines, Mac interface guidelines


                                                                            20
        Quantitative Evaluation
        Quantitative Evaluation
  Postpositivist claims and testing
• Postpositivist claims and testing
• Experimental method
  – Hypothesis
  – Typical measures
  – Test
  – Conclusion
• Confounds
  – Example


                                      21
            Quantitative Evaluation
            Quantitative Evaluation
• Can be hard to control for confounds
• Solution?
   – Punt
     Usability engineering
   – Usability engineering
   – Define metrics
      •   Time to accomplish a task
      •   Error rate
          E       t
      •   User satisfaction
      •   Etc.
     Keep re‐engineering until you reach metrics
   – Keep re engineering until you reach metrics
   – Note that metrics can interact


                                                   22
          Quantitative Evaluation
          Quantitative Evaluation
  Generally useful late in design
• Generally useful late in design
   – Given two systems, can we evaluate their relative 
     performance
   – Need careful metrics
• Also used for novel interaction techniques
   – Given a new way of selecting, is it faster, less error prone, 
     etc.
  Not typically used in design
• Not typically used in design



                                                                      23
             Qualitative Evaluation
             Qualitative Evaluation
•   Constructivist claims
•   Very common in design
•   Can be used either during design or after design complete
•   C    l b        db f    d i t        d t d
    Can also be used before design to understand worldld
•   Many, many techniques
•             g
    Broad categories
    – Interpretive
    – Predictive
    – Walkthroughs/thinkalouds




                                                                24
             Interpretive Evaluation
             Interpretive Evaluation
•   Need real‐world data of application use
•   Need knowledge of users in evaluation
•   Generally precede or follow design, not during
•   Techniques
    – Contextual Inquiry
        • Similar to for user understanding, but applied to final product
    – Cooperative and Participative evaluation
        • Cooperative evaluation allows users to walkthrough selected tasks, 
          verbalize problems
        • Participative evaluation also encourages users to select tasks
      Ethnographic methods
    – Ethnographic methods
        • Intensive observation, in‐depth interviews, participation in activities, etc. 
          to evaluate
        • Master‐apprentice is one restricted example of evaluation that can yield 
          ethnographic data
          ethnographic data

                                                                                       25
          Predictive Evaluation
          Predictive Evaluation
  Avoid extensive user testing by predicting 
• Avoid extensive user testing by predicting
  usability
• Includes
  – Inspection methods
  –U          d li
    Usage modeling
  – Person down the hall testing




                                                26
         Person down the hall testing
         Person down the hall testing
•   Common in the real world; purpose of this poster session
•   At each poster
     – Should be only one person present, two max.
     – Others should be circulating
•   When people come to your poster
    When people come to your poster
     – Select someone to walk through the interaction
     – Others watch
     – Have post‐it notes and pens on hand for feedback
     – Walk visitors through task, how users work now, and how you are changing 
       work
     – Then show prototypes
•                 p
    At classmates poster
     – Walk through the prototypes or observe
     – Evaluate how prototypes work, provide comments on post‐its/notepads




                                                                                   27
                       Thinkalouds
• Two methods
  – Retrospective
     •   Capture video of users using system
     •   Watch video with users
         Watch video with users
     •   Users comment on their actions and present their thinking
     •   Very common with Difficult‐to‐evaluate systems like ATC
     •   Can introduce post‐hoc rationalizations
         C i t d           th      ti   li ti
  – Concurrent
     • Very typical during design
     • You will do this




                                                                     28
              Concurrent Thinkalouds
              Concurrent Thinkalouds
•   Observe user using your prototype
•   Encourage them to “think‐aloud”
     – Express what they are thinking and wondering at each moment
•   When user is not having problems they work fast
       Faster than they think
     – Faster than they think
•   When user is having problems, they slow down
     – Think aloud can reveal aspects of bad mental models, poor affordances, 
       insufficient constraint, poor feedback, etc.
•   Sometimes, when under heavy load, user will pause
     – Essential to continue to encourage them to think‐aloud, but in a friendly way
•   Tasks can be specified (“Could you schedule a reservation?”) or open‐
    ended (user chooses what he/she would like to do with system)
    ended (user chooses what he/she would like to do with system)
•   Informal technique – creating an informal atmosphere will result in more 
    successful session



                                                                                       29
    Conducting concurrent think alouds
    Conducting concurrent think‐alouds
•   Settle on task
              l h           l        ?
     – Vertical or horizontal testing?
•   Settle on exactly what you want to tell user
     – You want to give appropriate level of direction
       If using Anoto pen, need to communicate how technology works
     – If using Anoto pen need to communicate how technology works
     – If using a traditional interface, need to communicate purpose of system
•   Think about how much help you want to give
     – You want an honest assessment
•   Two people maximum at think‐aloud
•   The interface, not the person, is under scrutiny
     – How they work is how they work
       You want an interface that will be easily incorporated into work practice
     – You want an interface that will be easily incorporated into work practice
     – Let them know that you will be providing only limited help, and apologize for 
       this in advance



                                                                                    30
  Conducting concurrent think‐alouds 
                 (2)
                 ( )
• One of you take the lead and greet the person
   – Put them at ease, describe process, give them information on what 
     you are testing
   – Pleasant expression
  Person who greets should observe
• P       h      t h ld b
   – Maintain pleasant expression
   – Set up audio recording
     Get notebook ready and ask them to start (the task you give or the 
   – Get notebook ready and ask them to start (the task you give or the
     tasks they typically would do)
   – Take notes as they work (suplements audio recording)
           p       g
   – Prompt during silences
       • ASK:  What are you thinking now?
       • NOT:  Why did you do that?




                                                                           31
  Conducting concurrent think‐alouds 
                 (3)
                 ( )
• After they finish, debrief
   – Look to your notes for points you would like clarification on
   – Ask them for overall impressions of the system
       • Biology example
  Thank your users
• Th k
• After session
   – Get together with your group asap
   – Walk through your notes, use audio, and make an affinity diagram of 
     data
   – Look for themes you can use to improve prototype
  Iterative on prototype (if possible) and conduct walkthrough 
• Iterative on prototype (if possible) and conduct walkthrough
  with other participant



                                                                            32
  Conducting concurrent think‐alouds 
                 (4)
                 ( )
• Advantages
   – Not limited to paper prototypes
        • Mathbrush
   – Rapid, high‐quality qualitative feedback
   – Data is as rich as with contextual inquiry
        • Observations, hearing
   –   Can interact with subject to get complete information
   –   Can help subject if it becomes necesary
   –   Flexibility in initiative
   –        g             pp     y g           p
       Doing, so less opportunity to give rote positive assessment
• Disadvantages
   – Limited sample


                                                                     33
                  Walkthroughs
• A series of sketches
• Walk user representatives through different screen 
  shots
• Ask users what they would do on each screen
• Advantages
   – Even faster than thinkalouds
• Disadvantages
     Feedback limited by no  doing
   – Feedback limited by no “doing”
   – Only works for early stage sketches


                                                        34
               Finish off final phase
               Finish off final phase
• Converge on a final prototype
• Implement a functional/semi‐functional version
   – You can woz aspects
     p       ,            p     yp
• If possible, demo final prototype for users
• Create a video of your final prototype
   – You act
        y
   – May need to add external video footageg
   – Use camtasia (30 day trial) or camstudio (free)
       • Camtasia better for editing
   – Windows Movie maker for editing, etc.
   – Due on last day of class
       • Graded on day of final exam
• Set up for final “poster” session, do final write‐up

                                                         35
                                Final Write up
                                Final Write‐up
•   Introduction                                        •   Evaluation results
     –   Summary of users, including current work 
         Summary of users including current work             –   Walk the affinity of feedback you collected
                                                                 Walk the affinity of feedback you collected
         practice                                            –   Include feedback from poster session 2 in 
     –   Problem you addressed                                   your evaluation
     –   Early designs                                  •   Overview of final design
     –   Overview of your evaluation results (most           –   Provide an overview of final design
         significant information)
         significant information)                            –   Include pictures and screenshots, no low 
     –   Overview of your final design                           fidelity
•   Evaluation Plan                                          –   Discuss:
     –   Provide an overview of how you evaluated                  •   How well it met user needs
         your design, whether you did a horizontal                 •   How thorough/complete you were
               ti l     l ti       d h th
         or vertical evaluation, and whether user or               •   How realistic design is for your users
                                                                       H      li ti d i i f
         designers took initiative                      •   Future Work
     –   Describe number of sessions, how long,              –   Any remaining improvements to your 
         etc.                                                    design
     –   As you provide these overviews, make sure           –   Any other significant problems of the user 
         you justify decisions
         you justify decisions                                    h      / h ld b      l d
                                                                 that can/should be solved
                                                        •   Conclusion




                                                                                                                36

								
To top