ESE 250

Document Sample
ESE 250 Powered By Docstoc
					                             User Interfaces

“Guest” Lecturer: Professor Norm Badler, CIS
  Credentials
  Introduction
  Devices
  Design principles
  Applying the principles

12/9/2009                    2
  Credentials
  Introduction
  Devices
  Design principles
  Applying the principles

12/9/2009                    3
  Computer Graphics, Animation and Simulation
  User Interfaces in 2D and 3D
  Virtual reality systems
  Commercial software product: Jack human modeling
     system developed here at Penn (now owned, extended
     and marketed by Siemens):

  Teaching Computer Graphics, including programming
     with graphics user interfaces (CIS277, CIS460).

12/9/2009                                                                                                 4
  Credentials
  Introduction
  Devices
  Design principles
  Applying the principles

12/9/2009                    5
User Interface Design?
  It’s important to our well being…

            Dilbert strip:   6
The Problem is that Humans are
Half of the User Interface

As Even if Intuitive and Clear, User
Interface Might not be so Good…

12/9/2009   8
And it’s not just about our
Workstation Interfaces…

12/9/2009                                                                      9
Poor Design can Cost!
  Toyota will recall about
     3.8 million cars and
     trucks to reshape and/or
     replace the accelerator
     pedals. The design of
     the accelerator pedal in
     combination with loose
     floormats may result in
     the accelerator pedal
     getting stuck.
                 Photo and source: autoblog, via Bright
12/9/2009                                                 10
Who’s to Blame for Usability
  Most Returned Products Work Fine, Study Says Only 5
   percent of returned products are genuinely defective:
   Yardena Arar, PC World, June 2, 2008 4:00 pm
  Only 5 percent of consumer electronics products returned
   to retailers are malfunctioning --yet many people who
   return working products think they are broken, a new
   study indicates.
  The report by technology consulting and outsourcing firm
   Accenture pegs the costs of consumer electronics returns in
   2007 at $13.8 billion in the United States alone, with return
   rates ranging from 11 percent to 20 percent, depending on
   the type of product.
12/9/2009                                                                            11
It could be the User Interface
Designers– Or their Managers
  It took 43 people to come up with the 15 different ways of
   shutting down Windows Vista
  According to a former worker at Microsoft, 43 persons where
   involved in designing and implementing the new shut down
   feature in Windows Vista.
  Their work was exceptionally fruitful. On a laptop you get Off,
   Lock, Switch User, Log Off, Restart, Sleep, Hibernate, and Shut
   Down. Combined with all the keyboard shortcuts for the various
   options, plus pressing the power button and closing the lid, this
   gives us 15 different ways of shutting down a laptop.
  In defense of Microsoft, it should noted that you will only see the
   Off and Lock options, unless you choose the advanced menu.
12/9/2009                                                                             12
There Oughtta be a Law…
  According to Jef Raskin in his book The Humane
   Interface, there are two laws of user interface design,
   based on the fictional laws of robotics created by Isaac
  First Law: A computer shall not harm your work or,
   through inactivity, allow your work to come to harm.
  Second Law: A computer shall not waste your time or
   require you to do more work than is strictly necessary.

                               Physical or Mental??

12/9/2009                                                                      13
  Credentials
  Introduction
  Devices
  Design principles
  Applying the principles

12/9/2009                    14
Displays and Wearable Devices
  Familiar 2D video and computer screens
  Mobile phones
  Cameras (still and video)
  Unobtrusive wearable displays
  Motion capture suits (and less obtrusive methods)
  The Nintendo Wii
  Body state sensing devices and feedback

12/9/2009                                              15
Mobile “Smartphones”
 Not just a phone, music player, and
 But a computing and interaction
 Touch screen, voice, video, inertial
  (accelerometer) orientation sensor,
 New applications.

12/9/2009                                16
Wearable “Smart” Cameras
 Microsoft Cambridge SenseCam.
 Takes low-res wide angle photo every
  30 sec. or when triggered by various
  on-board sensors (sound,
  temperature, motion).
 Visual record of experience.
12/9/2009                                                                                           17
Wearable Displays
    “Sliver” display
    Wearable computer

12/9/2009                18
Motion Capture
 Active Suit, e.g., Xsens Movens
  (accelerometers and magnetometers)
 Passive, e.g. Vicon (Cameras and LED
 Video only, e.g. OrganicMotion

Motion Capture Uses
  Enables data-driven animations (games)
  3D motion data is re-usable, modifiable, and adaptable
   to other characters.
  User inputs for interactivity and virtual worlds.

12/9/2009                                               20
Nintendo Wii
  Accelerometer and optical motion sensing

12/9/2009                                     21
Psychophysical Experiences
  Non-immersive or desk-top “VR”: The 2D computer
   screen; recent 3D display monitors.
  Immersive: “Through the window” display (airplane
  Immersive: Head-mounted displays (HMD) (helmets,
   goggles, glasses).
  Whole body immersive: CAVE® or other multi-
   projection rooms with imagery on walls, ceiling, floor.
  Augmented (Mixed) Reality: See the virtual
   superimposed on and registered with the real world.
12/9/2009                                                    22
Virtual Reality (VR) Displays
  VR goggles or helmets
     Limited field of view
     Cybersickness issues
     Cautious navigation
     User interfaces generally use hand

12/9/2009                                  23
Virtual Reality (VR) Displays
(Multiple Screens)
 One or more people can see
    and experience

                    “CAVE”

12/9/2009                       24
Augmented Reality (AR) Displays

 AR with video

 AR with optical

12/9/2009                         25
Augmented Reality Wearable
 Video version
 Scene limited to video
  resolution (not so good).
 Primary issue is
  real/virtual image

12/9/2009                     26
Augmented Reality with Portable
Devices (Smartphone)
 Use the embedded camera and overlap synthesized
  images and animation.
 Need real-time feature tracking for registration.
    “Invisible Train”:
    Schmalstieg and
    Reitmayr, 2004

12/9/2009                                                                27
Multi-Modal Interfaces
  Various challenges to do this effectively (cost, latency,
   technology, encumbrances,...)
  Sound input “easiest”, though speech recognition is
   still far from perfect.
  Eye gaze interfaces -- tricky to control; useful in
   assistive technology:

12/9/2009                     28
Electroencephalography (EEG)
  “Thinking” about something to control it.

12/9/2009                                                                                            29
EEG for Home Use is Coming…
  “Emotiv” mind controller:,future-tech-interview-with-emotiv-co-founder-nam-do-and-the-science-behind-mind-control.aspx

12/9/2009                                                                                                                                 30
Tactors and Haptics
  Deliver end effector forces and torques:

  Deliver skin vibration

  We are a long way from the Star Trek Holodeck in
     terms of the physicality of an interactive experience.
       Though visually, we’re doing great…
12/9/2009                                                     31
  Credentials
  Introduction
  Devices
  Design principles
  Applying the principles

12/9/2009                    32
User Interface Design:
Who do you Turn To?
  Academic and industry leaders.
  People with insight and experience
   whom one can learn from: Gurus!
  Best known: Donald Norman,
   through his several books, including
   The Design of Everyday Things.

  Others mention here include UI
     practitioners and even IBM.

12/9/2009                                 33
Donald Norman: UI Guru
 Referring to Norman’s book: Design of Everyday Things
  Visibility –visible functions aid user awareness; invisible
   functions are more difficult to find and know how to use.
  Feedback – return information about what action has been
   done and what has been accomplished.
  Constraints –restricting the kind of user interaction that can
   take place at a given moment.
  Mapping –the (functional, geometric, appearance) relationship
   between controls and their effects in the world.
  Consistency – use similar operations and use similar elements
   for achieving similar tasks.
  Affordance –an attribute of an object that allows people to
   know how to use it.

12/9/2009   34
Jakob Nielsen: UI Heuristics (1)
 Visibility of system status
  The system should always keep users informed about what is going on, through
   appropriate feedback within reasonable time.
 Match between system and the real world
  The system should speak the users' language, with words, phrases and concepts
   familiar to the user, rather than system-oriented terms. Follow real-world
   conventions, making information appear in a natural and logical order.
 User control and freedom
  Users often choose system functions by mistake and will need a clearly marked
   "emergency exit" to leave the unwanted state easily. Support undo and redo.
 Consistency and standards
  Users should not have to wonder whether different words, situations, or actions
   mean the same thing. Follow platform conventions.
 Error prevention
  Even better than good error messages is a careful design which prevents a
   problem from occurring in the first place. Either eliminate error-prone
   conditions or check for them and present users with a confirmation option
   before they commit to the action.
12/9/2009      35
Jakob Nielsen: UI Heuristics (2)
 Recognition rather than recall
  Minimize the user's memory load by making objects, actions, and options
    visible. The user should not have to remember information from one part of
    the dialogue to another. Instructions for use of the system should be visible or
    easily retrievable whenever appropriate.
 Flexibility and efficiency of use
  Accelerators -- unseen by the novice user -- may often speed up the interaction
    for the expert user. Allow users to tailor frequent actions.
 Aesthetic and minimalist design
  Dialogues should not contain information which is irrelevant or rarely needed.
    Every extra unit of information in a dialogue competes with the relevant units
    of information and diminishes their relative visibility.
 Help users recognize, diagnose, and recover from errors
  Error messages should be expressed in plain language (no codes), precisely
    indicate the problem, and constructively suggest a solution.
 Help and documentation
  Best if the system can be used without documentation, but it may be necessary
    to provide it. Any such information should be easy to search, focused on the
    user's task, list concrete steps to be carried out, and not be too large.
12/9/2009       36
Constantine and Lockwood
  The structure principle. Your design should organize the user interface purposefully, in
   meaningful and useful ways based on clear, consistent models that are apparent and
   recognizable to users, putting related things together and separating unrelated things,
   differentiating dissimilar things and making similar things resemble one another. The
   structure principle is concerned with your overall user interface architecture.
  The simplicity principle. Your design should make simple, common tasks simple to do,
   communicating clearly and simply in the user’s own language, and providing good
   shortcuts that are meaningfully related to longer procedures.
  The visibility principle. Your design should keep all needed options and materials for a
   given task visible without distracting the user with extraneous or redundant information.
   Good designs don’t overwhelm users with too many alternatives or confuse them with
   unneeded information.
  The feedback principle. Your design should keep users informed of actions or
   interpretations, changes of state or condition, and errors or exceptions that are relevant
   and of interest to the user through clear, concise, and unambiguous language familiar to
  The tolerance principle. Your design should be flexible and tolerant, reducing the cost
   of mistakes and misuse by allowing undoing and redoing, while also preventing errors
   wherever possible by tolerating varied inputs and sequences and by interpreting all
   reasonable actions reasonable.
  The reuse principle. Your design should reuse internal and external components and
   behaviors, maintaining consistency with purpose rather than merely arbitrary
   consistency, thus reducing the need for users to rethink and remember.
12/9/2009                  37
Schneiderman: 8 Golden Rules
 1. Strive for consistency.
    Consistent sequences of actions should be required in similar situations;
    identical terminology should be used in prompts, menus, and help screens;
    and consistent commands should be employed throughout.
 2. Enable frequent users to use shortcuts.
    As the frequency of use increases, so do the user's desires to reduce the number
    of interactions and to increase the pace of interaction. Abbreviations, function
    keys, hidden commands, and macro facilities are very helpful to an expert user.
 3. Offer informative feedback.
    For every operator action, there should be some system feedback. For frequent
    and minor actions, the response can be modest, while for infrequent and major
    actions, the response should be more substantial.
 4. Design dialog to yield closure.
    Sequences of actions should be organized into groups with a beginning,
    middle, and end. The informative feedback at the completion of a group of
    actions gives the operators the satisfaction of accomplishment, a sense of relief,
    the signal to drop contingency plans and options from their minds, and an
    indication that the way is clear to prepare for the next group of actions.

12/9/2009   38
Schneiderman: (2)
 5. Offer simple error handling.
    As much as possible, design the system so the user cannot make a serious
    error. If an error is made, the system should be able to detect the error and
    offer simple, comprehensible mechanisms for handling the error.
 6. Permit easy reversal of actions.
    This feature relieves anxiety, since the user knows that errors can be undone; it
    thus encourages exploration of unfamiliar options. The units of reversibility
    may be a single action, a data entry, or a complete group of actions.
 7. Support internal locus of control.
    Experienced operators strongly desire the sense that they are in charge of the
    system and that the system responds to their actions. Design the system to
    make users the initiators of actions rather than the responders.
 8. Reduce short-term memory load.
    The limitation of human information processing in short-term memory
    requires that displays be kept simple, multiple page displays be consolidated,
    window-motion frequency be reduced, and sufficient training time be allotted
    for codes, mnemonics, and sequences of actions.
12/9/2009   39
Mayhew’s “General Principles” (1)
 1. User compatibility.
    The most fundamental principle is to know the user.
 2. Product compatibility.
    Across-product compatibility allows the user to adapt to a new system quickly.
 3. Task compatibility.
    The structure and flow should match and support the task.
 4. Work flow compatibility.
    A system should be organized to facilitate transitions between tasks.
 5. Consistency.
    Similarities within a product allow reasoning by analogy and predict how to do
    things they have never done before.
 6. Familiarity.
    Concepts, terminology, and spatial arrangements should be familiar.
 7. Simplicity.
    Make the interface relatively simple.
 8. Direct manipulation.
    Directly perform actions on visible objects.
12/9/2009   40
Mayhew’s “General Principles” (2)
 9. Control.
    User prefer to feel a sense of mastery and control over any tool at their disposal,
    thus present a tool-like interface.
     What you see is what you get (across other media, too)
 11. Flexibility.
     Accommodate variations in user skill and preferences.
 12. Responsiveness.
       Give users feedback as soon as possible and let users be aware of the progress.
 13.   Invisible technology.
       Only present the functionality that users need to know.
 14.   Robustness.
       A system should tolerate common and unavoidable human error.
 15.   Protection.
       "Undo" or other recovery measures should be provided.
 16.   Ease of learning and ease of use.
       It should be easy to learn for the novice and efficient and easy to use for the expert.
12/9/2009    41
IBM's "Design Principles for
 1.  Simplicity: Don't compromise usability for function
 2.  Support: User is in control with proactive assistance
 3.  Familiarity: Build on users' prior knowledge
 4.  Obviousness: Make objects and their controls visible and
 5. Encouragement: Make actions predictable and reversible
 6. Satisfaction: Create a feeling of progress and achievement
 7. Accessibility: Make all objects accessible at all times
 8. Safety: Keep the user out of trouble
 9. Versatility: Support alternate interaction techniques
 10. Personalization: Allow users to customize
 11. Affinity: Bring objects to life through good visual design
12/9/2009   42
  Credentials
  Introduction
  Devices
  Design principles
  Applying the principles

12/9/2009                    43
What Can We Do With Principles?
 Principles are generally:
  Descriptive, comparative and analytical (i.e., how
   alternatives compare; test and refine paradigm)
  Not constructive (i.e., do not define process of
   developing user interface design)
     No automated (good) interface design tools exist
      (e.g., that could have predicted the iPod user
      interface design)

12/9/2009                                                44
What Can We Do With Principles?
  Principles define an optimization problem where the
     (target) user population is not uniform in skill,
     cognitive ability, needs, experience, learning style, or

12/9/2009                                                       45
Task-Centered Design
Identify users                           Analysis
                                                          Note ESSENTIAL iterative loops!
           Choose tasks

                       Look at other

                                     Turn tasks
                                   into scenarios                                        Design

                                               Turn scenarios
                                              into storyboards


Make sure testers are REAL
                                                                      Build the design
users from REPRESENTATIVE user
populations, not just SELF!                                                          Evaluate


User Analysis
  If you don’t understand what the users want to do with
   a system, you have no realistic prospect of designing
   an effective interface.
  User analyses have to be described in terms that users
   and other designers can understand.
  Scenarios where you describe typical episodes of use,
   are one way of describing these analyses.

12/9/2009      Ian Sommerville: Software Engineering, 7th Ed., 2004   47
Analysis Techniques
  Task analysis
     Models the steps involved in completing a task.
  Interviewing and questionnaires
     Asks the users about the work they do.
  Ethnography
     Observes the user at work.

12/9/2009       Ian Sommerville: Software Engineering, 7th Ed., 2004   48
Task Analysis
  Need to determine what tasks must be accomplished.
  Understand user’s intentions: What user wants (or
   needs) to do, not necessarily how to do it.
  Task analysis breaks problem into individual task
  Top-down task analysis.
  Tasks still need to be wrapped in a UI.

12/9/2009                                               49
Example Task Analysis: Using PowerPoint
                                               0 Prepare a slide      Plan 0:
                                               using PowerPoint       1-2-3-4

       1 Start                                                              3 Draw
                                            2 Enter text                                           4 Save slide
     PowerPoint                                                            diagrams
                                    Plan 2: According              Further redescribed         Further redescribed
                                    to requirements

      1 Enter text                          2 Copy text               3 Add text to
    using keyboard                          from a slide              existing slide
                                             Plan 2.2:             Further redescribed

                            2 Choose copy                3 Choose insert              4 Choose paste
  1 Select text block
                              command                       position                     command
 Further redescribed

 12/9/2009                          50
  Design semi-structured interviews based on open-
   ended questions.
  Users can then provide information that they think is
   essential; not just information that you have thought
   of collecting.
  Group interviews or focus groups allow users to discuss
   with each other what they do.

12/9/2009      Ian Sommerville: Software Engineering, 7th Ed., 2004   51
  Involves an external observer watching users at work
   and questioning them in an unscripted way about
   their work.
  Valuable because many user tasks are intuitive and
   they find these very difficult to describe and explain.
  Also helps understand the role of social and
   organizational influences on work.

12/9/2009      Ian Sommerville: Software Engineering, 7th Ed., 2004   52
Ethnographic Records
      Air traffic control involves a number of control ‘suites’ where the suites
      controlling adjacent sectors of airspace are physically located next to
      each other. Flights in a sector are represented by paper strips that are
      fitted into wooden racks in an order that reflects their position in the
      sector. If there are not enough slots in the rack (i.e. when the airspace
      is very busy), controllers spread the strips out on the desk in front of the
      When we were observing controllers, we noticed that controllers
      regularly glanced at the strip racks in the adjacent sector. We pointed
      this out to them and asked them why they did this. They replied that, if
      the adjacent controller has strips on their desk, then this meant that
      they would have a lot of flights entering their sector. They therefore tried
      to increase the speed of aircraft in the sector to ‘clear space’ for the
      incoming aircraft.

12/9/2009             Ian Sommerville: Software Engineering, 7th Ed., 2004           53
Insights from Ethnography
  Controllers had to see all flights in a sector. Therefore,
   scrolling displays where flights disappeared off the top
   or bottom of the display should be avoided.
  The interface had to have some way of telling
   controllers how many flights were in adjacent sectors
   so that they could plan their workload.

12/9/2009       Ian Sommerville: Software Engineering, 7th Ed., 2004   54
So Where Does UI Design Begin?
  How should information from the user be provided to the
   computer system?
  How should information from the computer system be
   presented to the user?
  User interaction and information presentation may be
   integrated through a coherent framework such as a user
   interface metaphor or stylistic pattern.

12/9/2009      Ian Sommerville: Software Engineering, 7th Ed., 2004   55
 Interaction Styles
  Direct manipulation
  Menu selection
  Form fill-in
  Command language
  Natural language

12/9/2009         Ian Sommerville: Software Engineering, 7th Ed., 2004   56
Interaction Styles
Style            Main Advantages              Main Disadvantages                Applications

Direct           Fast and intuitive           Only suitable where there is a    Video games; CAD
manipulation     interaction; easy to learn   visual metaphor for tasks and     systems
Menu selection   Avoids user error; little    Slow for experienced user; can    Most general purpose
                 typing required              become complex if many menu       systems
Form fill-in     Simple data entry; easy to   Takes up much screen space;       Ordering
                 learn; checkable             causes problems where user
                                              options do not match the form
Command          Powerful and flexible        Hard to learn; poor error         Operating systems,
language                                      management                        command and control
Natural          Accessible to casual user;   Requires typing; NL               Information retrieval
language         easily extended              understanding systems may be      and Q/A systems

12/9/2009                Ian Sommerville: Software Engineering, 7th Ed., 2004                      57
User Interface Prototyping
  The aim of prototyping is to allow users to gain direct
   experience with the interface.
  Without such direct experience, it is impossible to
   judge the usability of an interface.
  Prototyping may be a two-stage process:
       Early in the process, paper prototypes may be used;
       The design is then refined and increasingly
            sophisticated automated prototypes are then developed.

12/9/2009            Ian Sommerville: Software Engineering, 7th Ed., 2004   58
Paper Prototyping
  Work through scenarios using sketches of the
  Use a storyboard to present a series of interactions
   with the system.
  Paper prototyping is an effective way of getting user
   reactions to a design proposal.

12/9/2009      Ian Sommerville: Software Engineering, 7th Ed., 2004   59
Prototyping Techniques
  Script-driven prototyping
     Develop a set of scripts and screens using a tool such as
      Macromedia Director. When the user interacts with
      these, the screen changes to the next display.
  Use PowerPoint as a substitute for an editable script.
  Visual programming
     Use a language designed for rapid development such as
      Visual Basic. See Chapter 17.
  Internet-based prototyping
     Use a web browser and associated scripts.

12/9/2009       Ian Sommerville: Software Engineering, 7th Ed., 2004   60
 User Interface Evaluation
  Some evaluation of a user interface design
   should be carried out to assess its suitability.
  Full scale evaluation is very expensive and
   impractical for most systems.
  Ideally, an interface should be evaluated against a
   usability specification. However, it is rare for such
   specifications to be produced.

12/9/2009      Ian Sommerville: Software Engineering, 7th Ed., 2004   61
 Sample Usability Attributes
Attribute         Description

Learnability      How long does it take a new user to become productive with
                  the system?

Speed of          How well does the system response match the user’s work
Operation (use)   practice and task requirements?

Robustness        How tolerant is the system of user error?

Recoverability    How good is the system at recovering from user errors?

Adaptability      How closely is the system tied to a single model of work?

12/9/2009         Ian Sommerville: Software Engineering, 7th Ed., 2004         62
 Simple Evaluation Techniques
  Questionnaires for user feedback.
  Video recording of system use and subsequent tape
  Instrumentation of code to collect information about
   facility use, timings and user errors.
  The provision of code in the software to collect on-line
   user feedback.

12/9/2009      Ian Sommerville: Software Engineering, 7th Ed., 2004   63
Analytical Tools
  Eye Tracks
   (Scan paths,
   time on target)
  Similar with
   mouse tracks

12/9/2009   64
Analytical Tools
 Biosensing
     galvanic skin
     heart rate
     repetitive
 Estimate subject
 Excitement,
  frustration, etc.
12/9/2009   65
The Next Disruptive Technology for
User Interfaces?
  Flexible displays
  Organic Light Emitting
   Diodes (OLED)
  Sony
        0.3mm thick
        24 bit color
        120×160 pixels

  Samsung:
     0.05mm thick
     480×272 pixels
     full video color gamut
12/9/2009                        66
User Interfaces in the Future?
  Trend toward more body sensing to determine user
     emotional, intentional and cognitive states. Video
     observation coming?
    But will this make anticipatory interfaces that are helpful
     or just more annoying?
    Invasiveness: Should you turn your phone off or should it
     know you are in class and route the call to voicemail?
    Would you give up your automobile’s user interfaces (for
     driving) to computer control? Sometimes? When you fall
    Very blurred connection between the user interface and the
     thing itself (Cooper): the User Interface is the artifact.
12/9/2009                                                      67
Thank you!

12/9/2009    68