Test Plan Prototype Usability Testing

Document Sample
Test Plan Prototype Usability Testing Powered By Docstoc
					                                                                       st
FINAL Test Plan                                   July 28-29, 2004 21 Century Prototype Usability Testing



Testing Location:        OTIVO, INC. 451 Hayes St, San Francisco
Document Date:           July 26, 2004


Objectives
        1. Evaluate usability of design and functionality in the prototype
        2. Understand how many coverage options are desired (4 or more or less)
        3. Understand which next steps are desired (purchase online, purchase by phone, save quote) and why –
           particularly whey someone would choose one step over another or not continue with a next step
        4. Gather feedback about live chat
        5. Gather feedback about help
        6. Evaluate overall appeal of quotes and coverage options

What will be tested
Participants will review a static HTML prototype provided by and installed locally on the computer that the participants will
use.

Current prototype URL: http://dev.url.com/dir1/dir2/prototype/ (login/pwd: login/pwd)

The prototype assumes that this scenario:
        2 drivers who are married
        2 vehicles (specific vehicle type to be determined in delivered prototype)
        Driver 1 has a single at-fault accident
        Driver 2 has a single ticket
        Driver 1 is an Engineer
        Driver 2 is an occupation to be decided (in delivered prototype)
        1 vehicle is financed so participant can search for financing company
        2 additional household members: 3yearold daughter, grandmother living at home
        Coverage levels are adjustable so that participants can change coverage

The prototype will have these and other pages:
        1. ZIP Code
        2. Getting Started
        3. Vehicle Info
        4. Business Use (only appears if specifies that a vehicle is for business use)
        5. Driver Info
        6. Marital Status (only appears if they need to verify marital status).
        7. Incident Info (only appears if user admits to incidents)
        8. Discounts (only appears if user qualifies for a discount)
        9. Coverage Options
        10. Quote
        11. Driver Detail
        12. Verify Driver Info
        13. Vehicle Detail
        14. Verify VIN
        15. Lien Holder
        16. Quote Summary


                                       st
OTIVO Test Plan: July 28-29, 2004 21 Century Prototype Usability Testing
Filename: 6bde4284-8125-4727-93de-a82362023030.doc                                                                   Page 1
       17. Payment Plan
       18. Payment
       19. Confirmation

Scenarios suggested:
   1. Assume subject was referred by current policy holder to trigger the Refer Good Friend function
   2. Have subject save and start to retrieve their quote
   3. Get feedback on the Information Disclosure section = privacy policy
   4. Feedback on enable/disable questions – Vehicle Info: what is primary use of car? Next two questions only “hot” if
       choose commuting. Does that make sense or is it confusing?
   5. Feedback on new questions –
           a. Do you currently have auto liability insurance?
           b. If yes, how long have you had auto liability insurance without a lapse in coverage?
   6. Trigger Marital Status page (user selects 1 driver but indicates that he/she is married)
   7. Use printer friendly function
   8. Use ‘Exclude’ function on the Driver Detail page to exclude household member who is over 18
   9. Assess the impact of requiring the user to save quote to get past to Driver Detail page
   10. Have the subject select that their vehicle is co-registered (on the Vehicle Detail page) and enter in the co-
       registered owner
   11. Feedback on Quote Summary page (since we will now show this page to all users)
   12. Feedback on Quote Summary printer friendly version (if available)
   13. Coverage Calculator (how much coverage do I need) and Evaluator (how does compare to other carriers)
       integration into eQuote – tools in green on right sides


Dates and Facilities

Testing Dates: July 28-29, 2004 9:30am-5:45pm at OTIVO

User schedule:
       Floater 1: 9:30am-11:45am
       Participant 1: 9:30am-10:30am
       Participant 2: 10:45am-11:45am
       Floater 2: 1pm-3:15pm
       Participant 3: 1pm-2pm
       Participant 4: 2:15pm-3:15pm
       Floater 3: 3:30pm-5:45pm
       Participant 5: 3:30pm-4:30pm
       Participant 6: 4:45pm-5:45pm

Facility: OTIVO, 451 Hayes St, San Francisco, CA

Incentive Payment: The incentive payment for each user will be $75 and payment for each floater will be $100
Equipment provided by the facility:
    OS: Windows 2000
    BROWSER: Microsoft Internet Explorer 6.0
    MONITOR SIZE: 17 inch LCD
    MONITOR RESOLUTION: 1024x768
    COLOR DEPTH: millions of colors




                                      st
OTIVO Test Plan: July 28-29, 2004 21 Century Prototype Usability Testing
Filename: 6bde4284-8125-4727-93de-a82362023030.doc                                                             Page 2
Test Recording and Documentation:
The moderator conducting the session will record notes during the session. Based on these notes, a report of findings
and recommendations will be developed and delivered. Picture in picture video will be recorded in Windows Media format
on CD as well as on miniDV tape.

Moderator
        Leanne Waldal OTIVO, INC. office: 415-626-9939 cell: 415-812-4916 email: lwaldal@otivo.com

Facility Review Checklist
          Confirm that audio/video is working properly
          Verify hardware/software setup
          Verify link in toolbar to first page of prototype (21 ) and to Quote page (Quote)
                                                                st

          Remove address bar in browser
          Facility will collect signed informed consent forms from users
          Facility will give participants their incentive payments and ask for signature indicating receipt of payment

Greeting participants, Introductory Remarks
       While greeting and leading the user to the room, the moderator suggests they make themselves comfortable,
       switch the mouse or keyboard position if they need to, and explains that they will be reading from a script.
        The moderator will then proceed to follow the script below.




                                       st
OTIVO Test Plan: July 28-29, 2004 21 Century Prototype Usability Testing
Filename: 6bde4284-8125-4727-93de-a82362023030.doc                                                                  Page 3
  User’s Name:                                              Date and Time:



 My name is … and I don’t work for the web site whose design you’ll use today – I’m an independent consultant.

 You were invited here today to review a design for a web site and give your feedback about the design and content. The
 design is a prototype so some things function and some don’t. As we go through the prototype I’ll let you know what
 works and doesn’t work and ask you questions regarding what you think of the design and how you think the pages will
 work.

 There are people sitting behind this mirror (motion) who are interested in your opinions and feedback about the Web site
 that we’ll discuss.

 We would like to videotape/audiotape your session so other people involved in the design can benefit from your
 feedback. Is it okay with you to be videotaped today?

 Your participation here today is voluntary. If any time you'd like to stop or take a break, for whatever reason, just let me
 know.

 The NDA that you signed means that you agree not to tell anyone, including family members, detailed information about
 your visit and about the designs you see during this hour. What you can say is that you participated in a study to help
 improve a Web site.

 I’m going to read from this script and take notes as we go along. At some points I might need to ask you to stop for a
 moment while I jot something down.

 Do you have any questions before we start?



Introductory Questions
We’ll start with some questions about you:

        1. Who’s your current auto insurance provider?
        2. How did you choose that provider? What information do you look for?
        3. Do you know, off the top of your head, what you currently pay for 6 months’ premium?
        4. What methods have you used to purchase auto insurance? Phone, web, responded to snail mail
        5. If/when you’re shopping for insurance on the Web, where do you go or what do you do to compare prices?
        6. Have you ever shopped for insurance by …
           Clicking on an ad banner?
           Responding to an email?
           Using a search engine?
        7. Have you ever shopped for anything, besides insurance, by …
           Clicking on an ad banner?
           Responding to an email?
           Using a search engine?
        8. Have you ever clicked on an ad banner?
        9. Have you heard of 21st Century Insurance? If yes, what do you know about them?




                                        st
OTIVO Test Plan: July 28-29, 2004 21 Century Prototype Usability Testing
Filename: 6bde4284-8125-4727-93de-a82362023030.doc                                                                    Page 4
First Impression
Please make yourself comfortable in front of the computer.

When we talk, if there is something on the screen you want me to see, please use the mouse and pointer to show me.
Also, please try to be as vocal as you can, think out loud and tell me whatever you'd like to tell me about the site. If you
get too quiet, I'll prompt you for discussion.

We’re going to review some prototype design for a car insurance web site. Since it’s a prototype some things might look
like a link, but they won’t be clickable and some things might look like a link and they will be clickable. Additionally, when
you fill out a form I’d like you to use your own information, however when you submit the form the information on the next
page won’t be what you typed in because this is just a prototype. I’ll remind you of that as we go along.
                                                                                                                           st
We’ll start with the assumption that you’re shopping around for car insurance quotes and a friend recommended 21
                          st
Century so you went to 21 Century Insurance’s web site to get a quote.
                                                                                                               st
As we go along, let me know what you think of each page. Here’s the first page you’d see if you went to 21 Century’s
                                  st
web site – click on the link to 21 in the bar.

Note throughout session where they want to click on or get help.


Zip Code
        10. What do you think of this? (do they notice retrieve saved quote?)
        11. What would you do here?

Getting Started

        12. What do you think of this? (do they notice quote number? do they notice right side boxes?)
        13. What would you do here?
        14. Go ahead and fill out the form with your information. (do they notice/comment about privacy policy?)
        15. What do you think of the questions? Are any of them confusing?
        16. Why do you think there’s a privacy policy? (do they think it’s for the site or their info? note: content not
            available)
        17. What would you do next? (see if they notice Save your quote at the bottom)

Vehicle Info

Keep in mind this is a prototype so sometimes data will be populated in the form fields that you didn’t necessarily enter.

        18. What do you think of this? (do they notice quote number? do they notice right side boxes?)
        19. What would you do here?
        20. Go ahead and fill out the form with your information. (do they notice/comment about primary car question
            revealing other questions? Do they notice gray? Do they try to enter data when it’s gray?)
        21. What do you think of the questions? Are any of them confusing?
        22. What did you think of the question about primary use and how many days per week the car is driven? (did
            they notice/comment about daysperweek/avgdrivingdistance not being available?)
        23. Was there anything missing on this page related to your vehicles?
        24. What would you do next? (see if they notice Save your quote at the bottom)




                                        st
OTIVO Test Plan: July 28-29, 2004 21 Century Prototype Usability Testing
Filename: 6bde4284-8125-4727-93de-a82362023030.doc                                                                     Page 5
Business Use
       25. Let’s say you said, on the previous page, that one of your cars was for business use. This is what you’d see
           next. What do you think of this? (do they notice quote number? do they notice right side boxes? Do they
           notice grayed out at top?)
       26. What do you think of the questions? Are any of them confusing?
       27. What would you do next? (see if they notice Save your quote at the bottom)

Driver Info

       28. What do you think of this? (do they notice quote number? do they notice right side boxes?)
       29. What would you do here?
       30. Go ahead and fill out the form with your information. And remember that in this scenario a friend who has
           insurance with 21st Century referred you.
       31. What do you think of the questions? Are any of them confusing?
       32. What do you think of the questions about currently having auto liability insurance (in middle of page) and
           about how long you’ve had auto liability insurance without a lapse in coverage?
       33. Don’t forget that you were referred by a friend (towards bottom of page)
       34. Would you complete the Save your Quote section? Why (not)? Would you come back later?
       35. Why do you think you need to provide an email address and password to save a quote?
       36. What do you think would happen if you saved your quote?
       37. If saving your quote was required to get a quote, would you save it? Why(not)? (do they mention email
           address?)
       38. (help – prompt if they don’t rollover it) What do you think of this – that it shows help when you rollover?
       39. (help) Is it in a good place on the page? Do you mind that the help shows up when you rollover? (note if they
           mention the text is “more info” not “help”)
       40. What would you do next? (see if they notice Save your quote at the bottom)

Marital Status

       41. Because this is a prototype there is prepopulated data for a specific scenario. So in this case we’ll assume
           that you’re married and you want to add your spouse as a driver.
       What do you think of this so far? Does this make sense?

Incident Info

       42. Again, because this is a prototype there is prepopulated data here that says the drivers are Ron and Megan.
           Let’s assume that Ron Smith has a single at-fault accident. Megan has a single ticket.
       43. What would you do next?

Discounts

       44. Here we’ll assume that Ron had indicated his occupation as an engineer and that he’s a student and a mature
           driver and that one vehicle has a lojack device. What do you think of this?
       45. What would you do here?

Coverage Options

       46. What do you think of this page? (I think this is a quote for 2 people and 2 cars – but it’s not clear)
       47. Are there too many, too few, or just the right amount of options? If too many/few, how many would you like to
           see?
       48. Is there enough text or too much or too little? How much do you want to read about each option at this point?
           a. (if too little) What’s missing?


                                      st
OTIVO Test Plan: July 28-29, 2004 21 Century Prototype Usability Testing
Filename: 6bde4284-8125-4727-93de-a82362023030.doc                                                                Page 6
        49. Which coverage option would you choose? (if lowest) Is that the most appropriate coverage? Why(not)?
        50. For this scenario in this prototype, choose Common Coverage.

Quote
        51. What do you think of this page?
        52. What would you do here? (change options, tools on right, help) Go ahead and change anything that’s
            appropriate for you.
        53. Have you noticed the Contact Us box on the right?
                                    st
        54. Would you contact 21 Century? If you called, what information would you give to the person on the phone if
            you had a question about the quote on this site? (would they give quote number?)
        55. Would you use live chat? Why (not)? Have you seen or used live chat before on a site like this?
        56. Would you call them or ask them to call you?
        57. (tools – prompt it they don’t click on it) “How much coverage do I need?” Coverage calculator – what do you
            think this does? What do you think of it?
        58. Coverage Calculator – New Questions: Here’s the first page you’d see if you clicked on the link to How
            much coverage do I need? What would you do here?
        59. Coverage Calculator – New Questions: What do you think of these questions? Do you think they’ll be
            helpful in figuring out the right coverage for you? Go ahead and click on View Results
        60. Coverage Calculator – Our Recommendations: Keeping in mind that this is still under development
            (gibberish at top of page) – look over this and let me know what you think.
        61. Coverage Calculator What would Apply and Return do? Which would you choose?
        62. Coverage Calculator What did you think of this? (do they think it is useful/helpful?) Would you use it?
                                                                    st
        63. (tools – prompt if they don’t click on it) “How does 21 compare to other carriers” Evaluator – what do you
            think this does? What do you think of it?
        64. Evaluator – Your current 6mo. premium: Look over this page, assume your current carrier is AAA, and let
            me know what you think of it. What would you change (if anything)? Why?
        65. Evaluator – What would you do next? (view, return to quote) Go ahead and View so you can see the next
            screen in this prototype. (do they realize they’d need to choose and input their current coverage data?)
                              st
        66. Evaluator – 21 /AAA comparison – Let’s say that your current insurance is AAA and you entered the
            values that you see here. What do you think of this page? Is it helpful?
                                                             st
        67. Evaluator – Now go back to your original 21 quote.
        68. Evaluator What did you think of this? (do they think it is useful/helpful?) Would you use it?
        69. Would you print this? How would you print this? (do they notice printer friendly link?) if printer friendly page is
            available, click, look at it, ask what they think of it
        70. Now that you’ve seen your quote, gone through some tools to evaluate your coverage and compare with your
            current coverage, what would you do next? Purchase online, purchase by phone, save quote, print, quit …
            Why?
        71. Would you like more or other options at the bottom of this page?
        72. (if wouldn’t buy online) Would you save the quote? Why (not)?
        73. Let’s say you save the quote for now so you can check some other web sites and references and come back
            to this later.

Zip Code – retrieve saved quote

        74. We’re going to pretend that a few days have passed and you’re coming back to this site to look at the quote
            that you saved. So close the browser. Now open the browser again (doubleclick on the blue e) and click on
                           st
            the link to 21 in the links bar. This is the page you’d see after a few days when you came back to the site.
            What would you do to see that quote again? (Retrieve saved quote, Other)
        75. (email page) What would you do here? What do you think you’d see?
        76. If you logged in, you’d get the quote that you just saw – to see it click on the link to Quote in the links bar.
            (note: login link doesn’t go back to the quote – it assumes you saved earlier in the quote process and goes to
            vehicle info)




                                        st
OTIVO Test Plan: July 28-29, 2004 21 Century Prototype Usability Testing
Filename: 6bde4284-8125-4727-93de-a82362023030.doc                                                                    Page 7
Quote (“Retrieved”)

        77. How comfortable are you buying insurance online?
        78. Do you prefer to call someone at the insurance company before purchasing? (if yes) Why?
        79. How do you like to buy insurance? online, over phone, through a broker …

Driver Detail
        80. Let’s say you decided to purchase online. What do you think of this page?
        81. What do you think of the requirements in the form? Too much, too little, just right?
        82. Fill out this page ……. and assume that there is one additional household member who was born in 1940.
        83. What would you do if you didn’t want the person born in 1940 to be on this policy? (See if they figure out how
            to exclude from policy)
        84. (if they don’t see how to exclude) What does “Exclude” mean?
        85. What do you think of the process so far? (note: this page requires saving which wouldn’t be there if had
            already saved quote)
        86. What would you do next?

Verify Driver Info

        87. What do you think of this page? (do they understand why they need to verify drivers license info?)
        88. What would you do next?

Vehicle Detail

        89. What do you think of the design of this page? (do they comment on VIN input form?)
        90. Let’s assume that one vehicle is financed and both vehicles are co-registered. Fill out the form.
        91. What would you do next? (they can make up a VIN)

VIN Error

        92. Let’s say you entered the VIN incorrectly. Does this make sense?
        93. What would you do next? (don’t need to re-enter – just hit next)

Lien Holder 1

        94. Let’s say you don’t remember the exact name of the company that financed your vehicle. What would you do
            here? (search – note: Next goes to quote summary))

Lien Holder 2

        95. What would you do next? (type in letters, select from list) Let’s say you typed an A in the box.
        96. What do you think of this?
        97. Let’s say that A&E Credit Union is the right one – select it.

Lien Holder 3

        98. What do you think of this?
        99. What would you do next if this was the right financial company? (next, search again, search)

Quote Summary




                                       st
OTIVO Test Plan: July 28-29, 2004 21 Century Prototype Usability Testing
Filename: 6bde4284-8125-4727-93de-a82362023030.doc                                                                 Page 8
        100.    Keep in mind that this is a prototype so the items might not exactly match what you selected on previous
            pages. What do you think of the layout of the information on this page?
        101.    Do you want to see any more or less information on this page? (if yes) What more/removed?
        102.    What if you wanted to change some of the coverage options in this quote? How would you do that? (do
            they notice breadcrumbs? Or just hit back button?)
        103.    Would you print this? How would you print this? (printer friendly version at top, bottom)
        104.    (printer friendly version if available) What do you think of this? Would it print well? If not, what would
            you change?
        105.    What would you do next?

Credit Report

        106.    What would you select here? (any comments about payment plans, credit report)
        107.    Is there enough information to help you choose? (if not) What do you want to see here?
        108.    What would you do next? Let’s say you choose to pay in full now.

Payment Plan Method

        109.    What do you think of this form?
        110.    What would you do next to pay in full with a credit card?

Policy Confirmation

        111.    Take a moment to review this page and let me know if it gives you what you need at this point.
        112.    Is there anything you particularly like or dislike about this page?

Closing Questions

        113.    Did this process ask for too much, too little, or just the right amount of information? If too much, what
            questions would you rather not answer? If too little, what did you want to be asked?
        114.    What stands out in your mind?
        115.    What's your overall impression of this design?
        116.    Was there anything you particularly liked or disliked? Anything confusing?
        117.    How would you describe the designs using a few words?
        118.    How would you describe the quote? (reasonable, expensive, etc.)
        119.    If you could change anything about the designs you saw, what would you change?
        120.    How does this compare with other sites you've used to get insurance quotes?
        121.    Before we finish, do you have any comments or questions about anything I didn’t show you or anything I
            didn’t ask you?

That’s the end of my questions for you. Thank you very much for participating in this web site design review. Your
responses will be very helpful and everyone really appreciates the time you took to come here today.

(did they ever notice the quote number?)




                                       st
OTIVO Test Plan: July 28-29, 2004 21 Century Prototype Usability Testing
Filename: 6bde4284-8125-4727-93de-a82362023030.doc                                                                   Page 9

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:4
posted:4/18/2012
language:English
pages:9