Docstoc

Usability Test Report

Document Sample
Usability Test Report Powered By Docstoc
					         CheckFree Integration Solution
              Usability Testing Report




                     Prepared By:
                      Leo Primero
              Updated last October 8, 2011




Page 1
Executive Summary


The purpose of this test is to learn how well a representative sample of CheckFree
Investment Services can interpret and use the CheckFree Integration Solutions. Areas of the
site that will be evaluated for performance and overall user satisfaction include the
architecture, navigation, and its ability to meet expectations.

Overall task performance:
   Participants find it fairly easy to figure out
   Dashboard is well received
   A participant states it is a better version of NAWF
   A participant states 4/5 out of 10 in terms of functionality, look and feel

User difficulties and frustrations with the site:
    Select Profile page and Dashboard is not intuitive enough and lack visual prompts
    The links presented (total number) do not stand out
    ID and Sponsorship terminology gets confusing
    “Back” browser feature does not work
    The search result page is cropped in IE 7.0
    XML link is not intuitive and specifics of the XML page do not display what the cause
      of the failure is
    The Dashboard and Reports tabs are not prominent & intuitive

Significant usability findings:
    Select Profile link is not visible
    Dashboard and Reports tabs are not perceived as tabs initially
    Lack of titling/branding to tell that it is CheckFree Integration Solution
    Texts are small and hard to read


Methodology

What happened during the usability test
The usability evaluation of the CheckFree Integration Solution was conducted by Leo
Primero in Newark, NJ on December 13 & 14, 2007.

During the usability evaluation, three participants were asked to spend 45 minutes with the
site. During this time, participants:

        Answered questions about initial site impressions
        Performed real-world tasks on the site while thinking aloud
        Answered questions about their overall satisfaction




Page 2
Who were tested


Three participants from the CheckFree Newark office were invited. Two were from the office
of the Project management team who are barely new employees of CheckFree and have
limited exposure to CheckFree’s technology, and one QA lead that has been with CheckFree
for several years and who has a background of how NAWF functions.

What participants did


The participants met with the study facilitator approximately 45 minutes each and
performed five prepared scenarios. They were welcomed and briefed of what the usability
testing is for, given introduction to the task they need to perform, and then were asked to
sign the video waiver form.

Testing environment
The following is a summary of the participants’ computing environment:

        URL of website:      http://ewr-cr01:12001/CcisWebApplication/Login.aspx
                  Room:      Washington St Conference Room
     Computer platform:      Dell Latitude D620 Laptop
        Browser tested:      Internet Explorer 7.0
      Screen resolution:     1440 X 900
      Operating system:      Windows XP Pro
      Connection speed:      Shared T1
     Capturing software:     Camtasia 4.0

Captured mouse and keyboard screencast URLs
Participant 1:
http://10.170.100.198/screencast/Usability_Testing_CIS/Participant1/index.html
http://10.170.100.198/screencast/Usability_Testing_CIS/Participant1_Video/index.html
Participant 2:
http://10.170.100.198/screencast/Usability_Testing_CIS/Participant2/index.html
Participant 3:
http://10.170.100.198/screencast/Usability_Testing_CIS/Participant3/index.html


Major Findings and Recommendations

Major Findings:

 #       Description                                                       Marker
 1       Report task is relatively easy to complete.                       POSITIVE
 2       Select Profile link is not visible.                               USABILITY
 3       Dashboard and Reports tabs are not perceived as tabs initially.   USABILITY
 4       “Back” feature of the browser resulting to a “Webpage has         BUG
         expired” error.




Page 3
  5      Response time is good.                                              POSITIVE
  6      Fairly easy to configure once you get used to it.                   POSITIVE
  7      A better version of NAWF.                                           POSITIVE
  8      Help needs to be current. It does not present the information       BUG
         sought.
  9      Lack of titling/branding to tell that it is CheckFree Integration   USABILITY
         Solution.
 10      Texts are small and hard to read.                                   USABILITY
 11      The site performance is fast.                                       POSITIVE
 12      The scenarios used resulted to a less ideal outcomes due to non-    INFORMATION
         existence new account information.

The following is a description of those markers:

USABILITY             This marker denotes where a potential usability issue may be present.
POSITIVE              A positive comment made by the user.
INFORMATION           This is typically an observation or comment that was made as a result of
                      a question asked by the facilitator or general observation of the user. It
                      may lead to design considerations.
BUG                   This denotes that a comment or behavior was observed, but the results or
                      comment could be discounted due to a constraint of the prototype

Recommended Solutions:

 #       Description
 1       Select Profile, and other global links (Home, Change Password, Help, and Logout links
         should be moved below and grouped just above Dashboard and Reports tabs. Effort
         should be done that Select Profile and Home links be visible to the users.
  2      Dashboard and Reports tabs should be made to immediately control the focus of users.
         Introductory texts maybe needed.
  3      “Back” feature of the browser should be allowed to work.
  4      Main Help should be made current.
  5      Titling/branding of CheckFree Integration Solutions visual elements should be
         implemented to show the site identity to users.
  6      Cascading Style Sheets should be adjusted to use 10 point or larger for all texts.
  7      Alternate tags should be added to all links.
  8      Search result page column headers should be made more prominent.




Detailed Data Findings & Recommendations

Introductory questions & tasks

      Summary of introductory questions

 Questions                                       Responses
 Have you heard of CheckFree Integration         Two of the participants have not heard of the




Page 4
 Solution?                                        site while one has but very little information.
 Just from looking at this site, what kinds of    One participant say roles, responsibility and
 information do you think you could get           security might be available. Another says
 from this site? Please be specific.              client partners, sponsors similar to NAWF from
                                                  looking at the title of the page. Last one says
                                                  can’t really say much (no clue).
 Who do you think this site is designed for?      First participant says for associates, IT
 Why?                                             infrastructure. Next one says developed for
                                                  clients and for internal (admin) purposes. Last
                                                  one says can’t really say much.
 Who manages this site?                           One participant says someone in HR;
                                                  technology in IT. Another says that it is
                                                  CheckFree. Another guesses EPL.


Scenario 1 – New Account Opening
You received word from the marketing department that a new relationship with ZZZ sponsor
means that your firm received approximately 25 new accounts yesterday via CIS and fed
into APL. How do you confirm the number of accounts that were sent and how many were
actually loaded into APL?

 Number of participants        3
 Percent successful            33%

 Findings/Notes                                  Recommendations
 Participant 1 completed the task with           Make client ID and name be distinct from the
 difficulty and help. He was initially lost in   other. Make Select Profile page link be more
 the Select Profile page, got confused           visible or possibly aligned with the Dashboard
 with client ID and Name. He was not             and reports tabs.
 able to get back to the Select Profile
 page without help.

 Participant 2 did not complete the task.        Make the sponsor and ID more distinct and
 He cannot find the sponsor from the grid        recognizable.
 drop down. When he gets to the
 dashboard that has all zeroes value on
 them, he gets stuck on what to do next.
 He got confused on ID and sponsorship.
 Participant 3 did not complete the task.        Make the total number for account requests be
 He tried going through the report but           most intuitive that they should know that it is a
 gets a dead end. ZZZ & XYZ is confusing         link to be delved into. The first thing that users
 to him.                                         go for is the drop down and does not even
                                                 realize that it is only for reports. There should
                                                 be introductory texts for these tables.




Page 5
Scenario 2 – Failed Requests
You are the source of the messages and recently your messages failed the XSD validation
stage the whole of last week. Can you check to see why these messages are failing and
where would you find this information?

 Number of participants     3
 Percent successful         100%, one completed with difficulty or with some help

 Findings/Notes                               Recommendations
 Participant 1 completed the task easily.     Make the back feature of the application work.
 However, he got lost when he                 Make available alternate tags for hypertext
 inadvertently hit the “Back” button of the   links.
 Browser resulting to a “Webpage has
 expired” error. He commented that it
 would have been useful if there a mouse
 over description on the links.
 Participant 2 completed the task with        Make dashboard entries more intuitive.
 difficulty or help. He finds the dashboard
 doesn’t show anything and expects more
 visual prompts from the dashboard.
 Participant 3 completed the task easily.     Make Select Profile be available visually in the
 He cannot find Select Profile at first but   same level or slightly above Dashboard and
 when prompted and directed to where it       Reports. Make the search result pages more
 is, does the task easily. XML as link is     user-friendly.
 not as prominently presented.


Scenario 3 – Account Failed to Load to APL
As the sponsor, you sent 10 accounts to XYZ and 15 accounts to RJA last Monday. You
received replies for each Request ID associated with the failed accounts, 5 of the XYZ
accounts and 5 of the RJA accounts last Tuesday. How do you find the accounts and the
reason for failure?

 Number of participants     3
 Percent successful         100%, two completed with difficulty or with some help

 Findings/Notes                               Recommendations
 Participant 1 completed the task easily.     Make the back feature of the application work.
 On the results page, the page is cropped.    Make available alternate tags for hypertext
 [This result page cannot be viewed in full   links.
 using Internet Explorer 7.0]                 Make sure that the site is compatible with IE
                                              7.0.
 Participant 2 completed the task with        Make sure that XML is intuitive to be clicked on
 difficulty or help. The dashboard does       and display reason of failure.
 not show enough information on how to
 proceed. The XML when clicked does not
 show the information what the reason is




Page 6
 for the failure.
 Participant 3 completed the task with         Make sure that XML is intuitive to be clicked on
 difficulty or help. The search did not        and display reason of failure.
 reveal anything and the XML when
 clicked do not explain anything.


Scenario 4 – Reports
Your firm would like to find out the number of events that occurred each hour for last week.
They want you to hand in a report analysis of which errors were generated and when the
peak processing occurred. They are also requesting that this information be sent to them in
an excel spreadsheet. How would you go about getting this information?

 Number of participants      3
 Percent successful          100%

 Findings/Notes                                Recommendations
 Participant 1 completed the task easily.      If possible, make reports have variations.
 States variation of the report would be
 nice. Task is easy enough to find.
 Participant 2 completed the task easily.      Make sure that first column has a label. Column
 Notices that first column of the report is    header titles should be made more prominent.
 not labeled.
 Participant 3 completed the task easily.      Make the “Back” browser feature available.
 When hitting the back button of the
 browser, error occurs.


Scenario 5 – Self Directed Activity
Now, we’d like to hear about how you might use this site. Please identify an item that you
would be interested in finding on this site. Please state your question and then search for
the answer. Let me know when you’ve found the information.

 Number of participants      3
 Percent successful          100%

 Findings/Notes                                Recommendations
 Participant 1 completed the task easily.      None.
 States that he wants to use this site for
 production specific not client specific.
 Participant 2 completed the task with         Make main Help current.
 difficulty or help. The help system was
 working but incomplete still.
 Participant 3 completed the task with         None.
 difficulty or help. He wanted to see if the
 reports are working and was able to get
 results.




Page 7
Exit Questions/User Impressions

 Questions                     Responses
 What is your overall              Fairly easy to figure, like the dashboard.
 impression to the site?           A better version of NAWF.
                                   4/5 out of 10 in terms of functionality, look and feel.
 What is your impression           Fine. It collects the data.
 of the search capability?         General Help search does not give out enough
                                    information.
                                   Doesn’t have many options/parameters to play around
                                    with. Not enough information listed on the screens to let
                                    you feel comfortable.
 Do you feel this site is          Can use some more work. Not able to use the “Back”
 current? Why?                      button.
                                   Yes, it is.
                                   It’s OK, not so appealing but satisfactory.
 What did you like best            Reporting is quick.
 about the site?                   Dashboard because ID and name is shown and can
                                    become driving points for clients.
                                   The response time is good.
 What did you like least           Small texts/wording. Dashboard | Reports tabs should
 about the site?                    be bigger.
                                   Main help should be made current.
                                   Not user friendly.
 If you were the website           Texts should be made larger and give more emphasis.
 developer, what would be          Lay-out and usefulness of business should be made
 the first thing you would          better. Help should be made better.
 do to improve the                 The functionality – the whole site does not give much to
 website?                           play around with.
 Is there anything that you        Hard to tell.
 feel is missing on this           Not really in terms of function.
 site?                             The branding of the product, what kind of system. The
                                    titling should grab you where you are.
 If you were to describe           The site performance is fast. UI stuff is good.
 this site to a colleague in       This best describe an Integrated Solutions.
 a sentence or two, what           It has limited capabilities. Does not live to its name.
 would you say?
 Do you have any other               None.
 final comments or                   Nope.
 questions?                          None.




Page 8

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:5
posted:10/9/2011
language:English
pages:8