Document Sample
evaluation+1+ppt Powered By Docstoc
					Topic 7 Evaluation

Evaluation and Designing
Examine the following products
         Product Evaluation
• The following photographs show some of
  the different lemon squeezers that have
  been made through history. They range in
  size, price, material, effectiveness and
• Look at the pictures of the lemon
  squeezers below, the designs may have
  changed over the years but does this
  make the newer versions more effective?
A wooden fruit squeezer (above) - this simple squeezer is a modern but
the design is Victorian
Starck's Juicy Salif (above)- The juicer was designed in 1988 by the
French designer Philippe Starck
A modern novelty juicer
A modern representation of a Victorian glass lemon squeezer
A modern simple hinged fruit squeezer
A simple modern plastic lemon squeezer
          Product Evaluation
• Does this equipment encourage people to
  prepare and cook food?
• Is one of the lemon squeezers more efficient
  than the others?
• Is there a difference in price?
            Evaluation Models
Try answering these questions:
   1. Describe how each of the lemon squeezers
   2. Evaluate the different squeezers using the
      evaluation model ACCESS FM:
      • Aesthetics
      • Cost
      • Customer
      • Environment
      • Safety
      • Size
      • Function
      • Materials
          Evaluation Models
• Evaluate the different squeezers using the
  evaluation model PIM’s
• Evaluation is the systematic assessment of
  the worth or merit of some object
• Evaluation is the systematic acquisition and
  assessment of information to provide useful
  feedback about some object
• Goals of evaluation
  • most evaluations is to provide "useful feedback“
  • the major goal of evaluation should be to
    influence decision-making
             Types of Evaluation
• Formative evaluation includes several evaluation types:
   • needs assessment determines who needs the program, how
     great the need is, and what might work to meet the need
   • evaluability assessment determines whether an evaluation is
     feasible and how stakeholders can help shape its usefulness
   • structured conceptualization helps stakeholders define the
     program or technology, the target population, and the possible
   • implementation evaluation monitors the fidelity of the program
     or technology delivery
   • process evaluation investigates the process of delivering the
     program or technology, including alternative delivery procedures
             Types of Evaluation
• Summative evaluation can also be subdivided:
   • outcome evaluations investigate whether the program or
     technology caused demonstrable effects on specifically defined
     target outcomes
   • impact evaluation is broader and assesses the overall or net
     effects (intended or unintended) of the program or technology
     as a whole
   • cost-effectiveness and cost-benefit analysis address questions of
     efficiency by standardizing outcomes in terms of their dollar
     costs and values
   • secondary analysis reexamines existing data to address new
     questions or use methods not previously employed
   • meta-analysis integrates the outcome estimates from multiple
     studies to arrive at an overall or summary judgment on an
     evaluation question
        Evaluation Strategies
• Qualitative Evaluation
  • Is an assessment process that answers the
    question, "How well did we do?"
• Quantitative Evaluation
   • Is an assessment process that answers the
     question, "How much did we do?"
• Tests, models and experiments are used at the
  design development stage of the design cycle
• Evaluating ideas, before developing a chosen
               Literature Search
• The process of carrying out a
systematic (and usually exhaustive)
search of the literature on a given
topic. It is usually the first step in a
research project.
• A review of the literature this is
important for a research project
because it enables you to
acquire an understanding
of your topic, with its key
issues, and an awareness
of relevant research
that has already been
                User Trial
• Experimental investigation in which a
  group of users test versions of a product
  under controlled conditions (Pheasant
  1996), usability trial within research
• Systematic heuristic (trail and error) or
  experimental evaluation of the interaction
  between people and the products,
  equipment and environments they use
                        User Research
• When undertaking user research, you are
  effectively asking questions of users. The types of
  questions you want to ask define or limit the
  appropriate type of research. Questions for users
  •   How well is the existing product supporting them? What do they like and dislike?
  •   Do they have a wish-list for what the product should do?
  •   Do they understand the meaning of a specific function, page, menu or screen?
  •   Can they complete the tasks they want?
  •   What is their emotional reaction to a design concept or product?
  •   Do they value a product (whether existing or proposed)?
  •   What alternative or additional methods, channels or tools are they using?
  •   Are they using any workarounds?
  •   What are their concerns about a product?
  •   Do they understand the navigation, terminology and behavior of the product?
            Expert Appraisal
• Is the evaluation of a product or service by
  someone who has the professional training or
  experience to make an informed judgment on
  the design. Ideally, this person should not be
  biased by former involvement with the product
  since familiarity with any product or task makes
  it seem simpler and easier. Expert appraisal can
  be used to
  • Identify possible causes of design exclusion
  • Suggest improvements to reduce this exclusion
  • Increase user satisfaction
     Advantages / Disadvantages
                               ADVANTAGES                             DISADVANTAGES
Literature Search Many sources of information are              An abundance of data which can
                   available. Use of ICT to access             be time consuming.
                   information, online books, periodicals
                   enhances the search of information,
                   speed and cost , storage & security are
                   all considerations.
User Trial         User trail data is collected by observing   “Users” may carry out tasks in
                   users behaviour. The “user” is a non-       different ways from those
                   specialist which makes trials easier and    expected and be inexperienced
                   cost effective.                             in data collection.
User Research      User research is collected by obtaining     The data is usually qualitative.
                   users’ responses to questions. Data is
                   relatively easy and cheap to obtain.

Expert Appraisal   Expert knowledge and advice are             The expert may be biased
                   gained.                                     Locating an expert may be
                                                               difficult and expensive. The data
                                                               is usually qualitative.

Shared By: