Docstoc

ACE Release 2 - PowerPoint Presentation

Document Sample
ACE Release 2 - PowerPoint Presentation Powered By Docstoc
					              TRADE SUPPORT NETWORK
RELEASE 2 / USER ACCEPTANCE TEST (UAT)
Presenters:   John Hill, CBP
              Jim Stevens, eCP
              Kelly Ausherman, Boyds Collection
              Terri Ellis, Hewlett Packard
              Bill Inch, CBP
Panel:        Mike Boshes, eCP
              Michael Hall, eCP
              Marc Newkirk, eCP
Date:         September 16, 2003
                                        Customs and Border Protection
                                               Modernization
                                            New World, New Tools
                      Agenda / Overview

 Pilot / UAT Overview      John Hill, CBP

 Pilot / UAT Activities    Jim Stevens, eCP
    Training
    Overall User Satisfaction
    UAT Test Case User Satisfaction


 Trade Feedback            Kelly Ausherman, Boyds Collection
                            Terri Ellis, Hewlett Packard

 Account Expansion         Bill Inch, CBP
  Goals
                                                                1
    RELEASE 2
Pilot/UAT Overview


   John Hill, CBP


                    Customs and Border Protection
                           Modernization
                        New World, New Tools        2
                 Pilot / UAT Overview

 Release 2 Pilot period is a 90-day test of the system prior to
  formal acceptance by the government
   • Period of time allows for proof of system readiness through successful
     deployment, operations support, and user training and support;
   • Included a future workload simulation with 1380 concurrent users.
 UAT is an 8-week system test following training that utilizes a
  journal of test case scenarios designed to calculate users
  ability to use ACE Portal and to capture their satisfaction and
  comments
    66 Trade Account Users representing 41 Trade Accounts attended
     training
    UAT Journals containing 25 Test Case Scenarios reflecting the various
     capabilities of current Release of ACE Portal distributed to each
     participant at end of 3 Trade Training sessions; an Overall User
     Satisfaction Survey was provided
    70% of completed UAT Journals received from participants
                                                                         3
Pilot/UAT Activities


  Jim Stevens, eCP



                 Customs and Border Protection
                        Modernization
                     New World, New Tools        4
                                Training


 Three training sessions for ACE Trade Account Owners:
     July 08-10, 2003    15 Accounts    Overall Course Effectiveness 3.43
     July 15-17, 2003    11 Accounts    Overall Course Effectiveness 3.56
     July 22-24, 2003    15 Accounts    Overall Course Effectiveness 3.77

   Role-based training scenarios focused on CBP Account Managers,
    CBP Client Representatives, and Trade Account Owners. Training
    scenarios for each position with CBP Subject Matter Experts

   CBP Client Representatives and Account Managers present during the
    third day of the Trade Account Owner training to provide for interactive
    set-up of the account information

   Training was well received overall, earning an aggregate 3.59 on a 5
    point scale, Good to Very Good, for overall course effectiveness

                                                                               5
                 User Acceptance Test Journal

 UAT Test Journal consists of Test Cases
    Developed in conjunction with the Training Scenarios
    Based on input and feedback from CBP Subject Matter Experts
 Tests the various capabilities in Release 2 of the ACE Secure
  Data Portal
    Account Management, Significant Activity Log, Reports, etc.
 Tests ACE in real world environment using customer’s data.
 Tests Case questions provide insight into the User’s
  satisfaction with the various components of ACE
    Ease of Use, Ease of Learning, Number of Steps, Amount of
     Time, Performance, Compared to Current Method, etc.
 User Survey Questionnaire provide overall satisfaction
                                                                   6
                         UAT Test Case Example

   Test Case – Manage Importer of Record (IR) Information
      View IR Details
      Edit IR Details


   Before you start the test case, please answer the following questions:
      Is this a task you perform (or should perform) as part of your job?
      How frequently do you perform this task? How important is this task?
   Were you able to complete this test case?
      How satisfied are you with the Online Help, User Guide, Help Desk Support,
       Training for this task?
   Based on your experience doing this task:
      A typical user would be able to successfully use the ACE Portal for this task
       after…?
      How satisfied are you with the ACE Portal for this task with the:
          ease of learning to use; ease of using;
          usefulness; number of steps;
          amount of time; application performance
      Compared to the way you do this task now, the ACE Portal is …?                  7
      Overall, how satisfied are you with the ACE Portal for this task?
                                       Overall User Satisfaction


                                                                                  User Satisfaction Survey
                              ACE Trade UAT User Satisfaction                      provided to UAT Participants at
                                                                                   the conclusion of training (prior
                                     Before UAT                  After UAT         to start of UAT)
                    5
                                        4.23                        4.15
                           3.91                                                   User Satisfaction Survey was
User Satisfaction




                    4                                 3.7                          also part of the UAT Test
                                                                                   Journal (at the conclusion of
                                                                                   the Test Cases)
                    3

                                                                                  Results are positive
                    2

                    1                                                             Scale for Responses:
                                                                                       5 Very Satisfied
                        Trade 7/10   Trade 7/17   Trade 7/24    Trade UAT as
                                                                                       4 Satisfied
                                                                   of 8/25
                                                                                       3 Neutral
                                       Trade User Sessions                             2 Dissatisfied
                                                                                       1 Very Dissatisfied




                                                                                                                   8
                UAT Test Case User Satisfaction

   47 of 65 UAT Journals Returned From Trade Participants – 72%


   25 total test cases developed in collaboration with trade representatives
      Results from 12 Test Cases scored between 4 and 4.5
      Results from 13 Test Cases scored between 3.5 and 4


   Test Case User Satisfaction: Composite Mean of all Test Cases = 4.00
      High = 4.43; Low = 3.51



   38 of the 41 Trade companies represented provided feedback via the UAT
    Test Journal or the ACE Portal Feedback – 93%


   Based on results from UAT Test Journals and Portal Feedback,
    enhancements to the ACE Secure Data Portal will occur in future releases    9
        UAT Evaluation


Kelly Ausherman, Boyds Collection



                        Customs and Border Protection
                               Modernization
                            New World, New Tools        10
             UAT Evaluation

 Overall Evaluation
   Ease of Use
   Easy to Learn
   Training Preparedness
 Functionality Likes & Dislikes
    Limited Training for New Users
    Limited Number of Roles
    Location of the PDF icon in HTS
    Location of Error Messages
    Confirmation of Deactivation of User
 Technical Capability
    Stability

                                            11
Value and Benefits of ACE


  Terri Ellis, Hewlett Packard



                         Customs and Border Protection
                                Modernization
                             New World, New Tools        12
           Value & Benefits of ACE

 Visibility of Data
   Individual IR or Aggregate
   Entry Info
   Compliance Info
   Bond Info
   Immediate access to info
   Fewer FOIA Requests
   Reference Materials
       HTS
       Rulings
    Significant Activity Log
    Action Plan                     13
Account Expansion Goals


      Bill Inch, CBP



                       Customs and Border Protection
                              Modernization
                           New World, New Tools        14
             Account Expansion Goals
Reach 1,100 total ACE accounts by Release 4
 Phase I: Release 2 – 200 new importer accounts

 Phase II: Release 3 – Brokers for initial 41 importer accounts

 Phase III: Release 3 and 4 – 125 new trade accounts each month
      Release 3                                     Release 4
      • Importers and brokers managed by            • Additional importers, brokers and carrier
        national and port account managers            accounts
      • Selected land border carrier accounts


 Release 2                                   Release 3                             Release 4
 41     + 59     + 50      + 50   200     Eval.
                                                              +125/month Eval.             1,100
 Add Importers………………………………..….Add Brokers……………….………………Add Carriers
                              for Initial 41
                              Accounts
                                                                                                   15
                 Phase I                 Phase II                     Phase III
Visit the Modernization & ACE
     site on www.cbp.gov
         or write us at:
       CBPMO@dhs.gov

                                16

				
DOCUMENT INFO