Working title VeriSign Manager UI by cuiliqing


									Working title: VeriSign Manager UI Usability Evaluation
Author: Ishita Shah
Date: January 28th, 2004


VeriSign, in collaboration with Research Insights, conducted a usability
evaluation on January 20-21, 2004, of a new user interface design for
Payment’s VeriSign Manager Web application. The purpose of the study was
to obtain feedback on the proposed new design for VeriSign Manager, and to
understand users’ needs and problems.

The study included 12 current VeriSign Payment Services users, recruited by
[company name]. The sessions were approximately one hour in length and
were conducted at VeriSign’s onsite usability testing laboratory.

This evaluation was focused primarily on the following elements of the

      Existing users’ needs and problems
      Home page design and functionality
      Overall layout and navigation
      New functionalities:
       - Users and roles
       - Billing information
       - Service renewals


Five of the twelve users are current Payflow Pro customers, and the
remaining seven users are customers of Payflow Link. Almost all had prior
experience using VeriSign Manager on an average of one to two years. Five of
the twelve users were part of an IT department working for an enterprise
company (greater than 50 employees). Their job titles were system engineer
or IT specialists and they had an average of four years of Internet experience.
The remaining seven users were owners of or executive team members
working for organizations of less than 50 employees, and had two to four
years of Internet experience.

In the survey conducted as a part of this evaluation, nearly 60% of users said
that they use the existing VeriSign Manager on regular basis. More than 70%
of those users were satisfied using the current VeriSign Manager to manage
their Payflow products. However, the current design is not scalable for adding
new products and services. Also the purpose of the redesign is to enhance
existing customers satisfaction while trying to design a product that can used
by customers who currently depend on their home-grown systems.

This report presents the findings and recommendations arising from the
usability test.

Payment Manager Usability Testing            Page 1              VeriSign Confidential

In general, the prototype has a pleasing appearance and a good overall
structure. Users found that the application structure contained the main
sections that they would expect to find, and liked the general look of the blue
color palette and the layout of second and third level pages.

However, there are top five usability issues that should be addressed if the
application is to achieve its goals:

1) VeriSign Manager is too complicated. The main complaint of users is
   that the VeriSign Manager is “too complicated.” Typically, users made
   comments such as, “Why do I need to learn about the payment industry
   when at the end of the day the only thing I care about is the dollar
   amount earned,” or “Give me a simple solution like Paypal.” New users of
   all skill levels had a difficult time understanding the concepts of VeriSign
   Manager; then with repeated use, they got accustomed to it.

2) Reports do not meet user needs. The next greatest complaint of users
   is with the Report section, which is the most frequently used feature of
   the VeriSign Manager. Users want to save a customized report and its
   preferences. Users also want to get a report on transactions older than
   three months. Users also complained about the fact that they have to
   uncheck the preferences in the customized report page every time they
   want to run a new custom report.

3) Search feature does not allow wildcard searches. One of the
   complaints regarding the Search Transaction section was that users
   cannot do wildcard searches using partial information provided by the
   bank. For example, when users get a report from the bank, for security
   purposes in the report only a partial credit card number is shown as
   XXXXXXXXXXXX1111. The VeriSign Manager Search function is not able to
   do a partial search using information provided by bank to reconcile the
   information. To solve this problem, users have built home-grown
   applications that allow them to do wildcard searches. This situation forces
   users to maintain two different applications, one VeriSign Manager and
   the other a home-grown application.

4) Users were not interested in having news on the home page. While
   users liked the idea of having a message center for important messages,
   they noted that they seldom read marketing newsletters and promotions.
   On the home page of the new design, users noted that the Upgrade
   Center was more Verisign marketing-focused and not user-focused. A few
   of the users commented that the Upgrade Center made them feel
   pressured to buy. One user suggested that he would rather call a
   representative if he wanted to add features.

5) Unfamiliar and inconsistent terms and features confuse users. The
   naming conventions for Billing Information and Recurring Billing were
   confusing to users. For example, many users do not have a clear idea
   about what the Recurring Billing Service is, thinking instead that this

Payment Manager Usability Testing             Page 2              VeriSign Confidential
   represents their bill from VeriSign. The references to Account
   Administration and Service Settings tabs were also confusing.

Each of these issues is discussed in the body of this report, and specific
recommendations are made for addressing them.

Payment Manager Usability Testing             Page 3              VeriSign Confidential

The following sections of the report provide more detailed descriptions of the
issues arising from the usability testing, along with recommendations for each
item. Before redesigning the UI, the major product strategies and approach
need to be revised on based on customers’ needs.

It is recommended that the product owner and design team should:

1)     Design the Manager in such a way as to offer specific features only to
       customers who need them or may want them. For example, make the
       application dynamic so that a beginner user who does not want to see
       every available feature will only see the functionality he/she needs.
       Consider adding more help for beginners in the form of flash demos,
       tutorials, and more intuitive context-sensitive help.

2)     Conduct usability testing with subjects who have never used VeriSign
       Payment Services and find out if they are able to easily use the
       product. This will help enhance customer satisfaction by enabling new
       users to ramp up quickly instead of having what the customers
       describe as a long learning curve.

3)     Provide savable custom reports and wildcard searches for transactions.

4)     Divide the Message Center into two different sections: Message Center
       and Alerts. Display important messages that require users’ attention in
       the alert section. Display marketing updates in the Message Center
       and let the user move it to a less prominent spot on the page.

       Change the concept of the Upgrade Center. In addition to offering
       users a chance to buy, it would be nice if VeriSign can offer a chance
       to learn more or to try options for new and additional products.

5)     Find better terminologies for product names and functionalities across
       applications. In the scenarios such as Account Administration >
       Service Settings, and Recurring Billing > Billing Information, one way
       to find appropriate terminology is to ask customers what names make
       sense and how they would organize information.

Payment Manager Usability Testing            Page 4              VeriSign Confidential
Detailed Findings

The following pages contain detailed findings and recommendations from the
usability sessions. The primary sections include:

    Existing users’ needs and problems
    Home page design and functionality
    Overall layout and navigation
    New functionalities:
         - Users and roles
         - Billing information
         - Service renewals

Each issue is described, and a recommendation is made. The relevant pages
from the prototype have been listed in a separate column. A severity rating
has been assigned to each issue, according to its perceived impact on the

#     Issue                                     Recommendation                                     Relevant   Severity
      Existing users needs and problems
1.    A beginner user with less than one        Redesign the application by focusing on all        All        High
      year of Internet experience mentioned     customers’ needs and skill levels.
      that Payflow product is “too
      complicated.” He would like something     Make product more dynamic and
      simple like Paypal. He said, “I don’t     customizable by users. Give more power to
      want to learn about the Payment           users to customize and personalize the
      industry. What is important to me at      application based on their needs, in a way
      the end of the day is my revenue -- the   similar to a site such as .
      dollar amount my business has             Introduce features like “Make this my home
      earned.”                                  page,” where users can choose exactly what
                                                information they need and where they’d like to
      Another user with a similar level of      put it.
      experience mentioned that she tries to
      understand the functionality but if she
                                                For new and inexperienced users, introduce
      does not understand, then she will call
                                                “how to’s,” Flash guides, demos, tutorials and
      first call the customer support. She
                                                more intuitive help in addition to making the
      would not spend her time trying to find
                                                application more dynamic for each users
      information to help her understand the
                                                based on their needs.
      concept of functionalities.

Payment Manager Usability Testing                             Page 5                      VeriSign Confidential
2.   Users noted that Report is the most        1)   Make report customizable and savable.       *Report    High
     commonly used functionality on an          2)   Introduce a functionality and technology    Sections
     everyday basis.                                 where users can do search back in
                                                     history till one year.                      *Custom
     Most used reports are “Daily Activity,”    3)   Improve UI for custom report. Click here    Report
     “Transaction Summary,” and “Custom              for a proposed new design: (http://ishah-
     Report.” Users mentioned that the               sun:8020/Manager/Manager/reportCus
     most problems they have with reports            tom.jsp) for example.
     are:                                       4)   Consider eliminating features that
     1) Cannot save customizable report              customers do not use. Extra functionality
     2) Cannot do search for longer time in          adds confusion and makes the Manager
     history than 3 months.                          more complicated.
     3) Have to un-check more than 20
     preferences to run different type of
     custom report.
     4) Hardly use “Payflow Link Order”
     report and do not understand the
     difference between standard report
     and “Payflow Link Report” as 2nd level
     5) Hardly or ever use “Shipping &
     Billing Report”
3.   Users noted that 2nd most common           Add wildcard search capability to Manager.                  High
     function they use inside VeriSign
     Manager is “Search Transaction” to
     find out any problems with
     transactions. Users expected that in
     the new design they should be able to
     do wildcard searches using partial
     information provided by bank, such as
     Reference # or partial credit card #.
     Home Page Overall Design
4.   In general, users liked the clean look     Re-design home page layout to emphasize          Home       High
     of the Home page. However, many            most used sections and the links most used       Section,
     users referred to “a lot of information”   by customers, and de-emphasizing sections        home.jsp
     and “lot of choices.”                      such as upgrade center.
     They cited a lack of clarity for day-to-
     day users activities such as Report,       Do not provide all the links regarding buying
     Search and Perform Transaction             but rather give “learn more” and “Try a new
     shortcut link from home page.              product” as a link. Move buying section, or
                                                include up sell of new products as part of
     Noted that too much valuable real          Renewal section.
     estate was devoted to Message
     Center, and marketing upsell for
     Upgrade Center. Wanted the Message
     Center to emphasize messages that
     need users’ attention.

     In Upgrade Center, users would like to
     learn about new products first and then
     try those products, and then would like
     to buy.

Payment Manager Usability Testing                             Page 6                      VeriSign Confidential
5.   Even though users liked the idea of        Divide Marketing promotions and important        Home       High
     Message Center, users noted that it        messages and alerts in two different sections    Page,
     was too marketing-oriented. Users          on home page.                                    home.jsp
     mentioned that when they login to
     VeriSign Manager, usually they are         Give more real estate to alerts section that
     there to do report on transaction or       should have what users want to see as
     solve problem, and credit back not         important messages and messages that
     read marketing newsletter. They will       requires their attentions. Such as service
     rarely pay attention to it.                outages, renewal notice, service outages,
                                                password and credit card expiration. Once
     Users mentioned that they would like       user have acted on that warning / alerts,
     to see messages in message center          remove those alerts from their areas.
     that need their attention action, such
     as password change, credit card            In Marketing promotion area, put Marketing
     expiration, renewal notice, and service    newsletters, promotions of new products and
     outage.                                    more. Give them an option to delete the
                                                archived messages when they want to.
     When asked if they can differentiate
     between bold hot link and un-bold hot      To find out if it is a new or unread message,
     link, users were confused and thought      put some sign or image icon to differentiate.
     that only marketing newsletters get        Consider link color-coding.
     displayed in BOLD FONT FACE.

6.   Some users noted Payflow Services          Change the title of the table to – “”Purchased   Home       High
     table is bit confusing. They did not       Services,” “Services in your Account,” or        Page
     realize that those are the services that   “Services Paid for.”
     users had bought.
7.   When users clicked on Payflow              Create intermediate information page with        Home       Mid
     Service table links on home page, they     details about services, such as when they will   Page
     were confused to see a Service             expire, little more information about service,
     Settings and configuration section.        how it works, how much they have paid,
                                                documents. Also give a link to how to
                                                configure settings for services.
8.   Users noticed “Update Primary Users        Put these two links under Account                Home       High
     Information” and “Change Login             Administration tab.                              Page
     information” when they were on the
     home page. But when users were not         Change company information tab to Contact
     on the Home page, they did not             Information tab and in 3rd level navigation
     understand how to change their             introduce 3 other links:
     primary information.                       1. Primary Information
                                                2. Company Information
                                                3. Change Password

                                                If user logs in to Manager and if is not the
                                                primary contact person, then give him / her an
                                                option to change information in same tab.

Payment Manager Usability Testing                             Page 7                      VeriSign Confidential
9.    Users liked the idea of a “Transaction     Display following items in the “Transaction      Home           High
      Summary” table, but added that it          Activity Summary” table as default display and   Page,
      would be nice if they can personalize      give permission to users to change it and        home.jsp
      the summary table based on their           customize it based on their business need.
      needs. Users would like additional         1. Total $ amount, daily revenue
      functionality such as getting dollar       2. Be able to customize
      amount, rejecting transaction rates,
                                                 3. Transaction errors, problems, warnings
      and to be able to customize based on
      their needs.                               4. Be able to see by credit card type
                                                 5. Authorization, credits, settlements, break
                                                      down by transaction types, summary of
      Users liked the graphical
                                                      rejected transactions
      representation with pie chart of the
      transaction summary.                       6. Summary of transactions by card types:
                                                      such as what card type is dominant, how
                                                      many transactions merchant’s business
                                                      is doing based on cc type
                                                 7. Rejections rates based on cc type --
                                                      currently do through custom report but it
                                                      will be nice to see a snap shot on home

10.   When users were asked to look for          Lack of familiarity with products made user      Billing        --
      their billing statement from VeriSign,     misunderstand the use of the Recurring Billing   Information
      they were confused and pointed to          tab. Naming convention for Billing Information
      Recurring Billing tab and said they will   and Recurring Billing was confusing to users.
      find out more information from             Find better terminology for product name and
      Recurring Billing section as billing is    functionality across applications.
      the heading they were searching for.
11.   Majority of primary users put              Change terminology for those two tabs. One       Account        --
      themselves as Admin and repeatedly         way to find appropriate terminology is to ask    Administrati
      got confused between “Account              customers what names make sense and how          on and
      Administration” and “Service Settings”     they would organize information under each       Service
      tabs. They thought that these were         tab. Another solution is combining those two     Settings
      synonyms of each other and wondered        tabs into one and give that tab a meaningful
      why there are two different tabs.          name.

      Some users thought they could find
      the bank information under “Service

      Some users thought Renewal should
      be under “Service Settings” tab.

      Most of the users expected that
      “Account Administration” tab would
      have update contact, company, and
      billing information along with changing
      the login password information.

Payment Manager Usability Testing                              Page 8                     VeriSign Confidential
12.   Most users were comfortable with           I suggest that at this point it is not worth      Over all     Low
      “VeriSign Manager” name for Payflow        changing the name. Customers are used to
      Management application because they        this name.
      are used to that name.

      One user who had experience using
      other VeriSign products such as SSL
      certificates mentioned that in the
      beginning he did not feel that this
      name matched the application’s
      purpose. He suggested some other
      names such as “Payflow Manager.”
      Navigation & Graphics
13.   One user noticed that unselected tabs’     Choose a different color for 3rd level            Over all –   High
      background color in main navigation is     navigation. Have four different colors or         tab
      the same color as 3rd level navigation’s   shades of blue for                                structure
      background color. She was confused              1. Selected 1st Navigation tab
      about where she was in the navigation           2. Unselected 1st Navigation tab
                                                      3. Selected 2nd level Navigation
                                                      4. Selected 3rd level Navigation
14.   One user noted that 2nd Navigation         Make all link underlined                          Over all –   High
      was not noticeable. It was confusing                                                         tab
      for her because she did not see 2nd                                                          structure
      level navigation links as hot links. She
      did not know that she could click on
      those links and go to other
      functionality. She always kept on going
      back to the 1st navigation tab’s home
      page and clicked on the links with
      descriptions and found that annoying.

Payment Manager Usability Testing                              Page 9                        VeriSign Confidential
15.   One user was confused when all              Open new windows for new tasks. Always          Global       Mid
      global navigation links were opened in      give them “On focus” so they will stay on top   Navigation
      same pop-up window, which was not           of other browser windows already opened.        structure

      Following were her steps:
           1. She clicked on
                “Documentation” link and
                opened a new window.
           2. She went back on the home
                page and clicked on
                “Support” global navigation
                link. She thought it would
                open one more new window.
           3. She was confused why
                clicking on the link did not
                open the support section in
                new window.
           4. After some clicks she
                realized that the content was
                changing in the window
                originally opened the first
                time for Documentation.

16.   When users were asked for their             Add another table to Support page. In that      Support      Mid
      expectation for “Support” Global            table give customer support contact             landing
      navigation link they said it should have    information such as phone and e-mail            page
      customer support phone # and e-mail         address.

      When they clicked and did not find it, it
      was a bit confusing to them.

      Users liked the idea of knowledge
      Base and opening a bug / problem
      tracking option in the support tab.
17.   Most users did not see a point to           Combine Documents and Help.                     Global       Mid
      having different Global Navigation links                                                    Navigation
      for Documentation, Downloads,               Change Downloads name to Software
      Support and Help                            Downloads.

                                                  Change Support name to Customer support or
                                                  add Customer Support link to Help library.
      New Features

Payment Manager Usability Testing                               Page 10                    VeriSign Confidential
18.   Users & Roles:                            1) Design / offer Users & Roles only to those   Users &   --
      - Existing users who had problems         users who need this functionality.              Roles
          with sharing passwords very
          much liked the idea of Users &        2) Remove alphabetical search
      - But there were some small               3) Reorganize all buttons’ functionality and
          business owners who were single       drop down close to check box selections.
          owner/operators who did not need
                                                4) Remove Action column from the table.
          this functionality at all. They
          thought this involved too much
          extra learning and made VeriSign
          Manager too complicated.
      - Users did not see a need for
          alphabetical search on the
          Manage Users page.
      - Some users were confused and
          did not see any relation between
          the list of users and functionality
          buttons such as “Delete,” and
          “Reset Passwords,” as well as the
          dropdown “Change Roles to” and
          “Change Status to.”
      - Couple of users clicked on Action
          column “Update” link as well as
          on “User Name” link and they
          found out that both links goes to
          same page, which was confusing
          to users. Why two links goes to
          same location?

Payment Manager Usability Testing                             Page 11                     VeriSign Confidential
19.   Billing: When users are asked how          Enable users to pay their bill online,             Billing       --
      they pay for their Verisign services,      particularly from inside the application.          Information
      they seemed puzzled and answered
      that either they did not remember or       There was no testing done for billing
      they have paid for the whole year.         functionality because there was no prototype
      Few users mentioned that right now         done. When mockups are created for billing
      they have to get a PO cut from a           functionality, usability testing is recommended.
      different department in their company
      and it takes some time and effort. So it
      will be nice to see online Billing
      Payment as part of the VeriSign
      Manager. Users said they think paying
      online will save them time.
      Users would like to see the following
      features implemented for Billing
      information sections:
            1) Summary of all products
            2) How much they have paid
                 for all the products
            3) When those products are
                 getting expired and how long
                 those services are valid for.
            4) Past one years’ billing
                 statement / invoices
            5) Online Payment method
                 using credit card
20.   Renewal functionality:                     There was no testing done for Renewal              Renewal       --
                                                 functionality because there was no prototype
      When users were asked where they           done.
      would go to renew their service, many
      said under Service Settings. Other         When mockups are created for renewal
      users thought they would find renewal      functionality, usability testing is recommended.
      with billing information because they
      are going to pay with a credit card and
      they knew that credit card information
      was found under billing information.

      Users would like to see following
      features implemented for Renewal

           1)   Which service they are
                renewing for how long
           2)   How much does it cost
           3)   It will be nice to see options
                to upgrade to new services
                while they are renewing
           4)   For how long they can renew
                their services. Plan options.
           5)   Be able to renew online
                using Payment method by
                credit card

Payment Manager Usability Testing                               Page 12                      VeriSign Confidential
      Additional Comments and
21.   Most existing users have learned        1.   Flash demo                                     Over all      Low
      VeriSign Manager by poking around       2.   Tutorials
      and using a trial-and-error method.     3.   Easy access to what they need it and
                                                   when they need it by giving short cuts to
                                                   relative documents on functionality page
                                              4.   Error code, result code should be part of
                                                   pdf guide as well as there should be html
                                                   version link on the front end page as
                                                   cheat sheet or short curt

22.   One user mentioned that it would be     Create cheat sheet, tips, easy access html          Error         Mid
      nice to have easy access to “Result     pages for such critical functionalities and offer   pages,
      Code” or “Error code” apart from the    the link on the front end – UI pages.               Result
      pdf and guides.                                                                             pages,
                                                                                                  short cuts

23.   One user – with 2 years of VeriSign     Do not offer extra features that are not needed     Perform       Mid
      Manager everyday user experience –      or used by users.                                   Transaction
      mentioned that they hardly use “Auto                                                        s Section
      Delayed Capture – rather does not use   Or give users more control over personalizing
      it at all”.                             their application by letting them choose what
                                              to show and where to show and whom to let
                                              use that functionality.

Payment Manager Usability Testing                            Page 13                      VeriSign Confidential

To top