Audit Program

Document Sample
Audit Program Powered By Docstoc
					Corporate Audit Services (CAS) Project Development Audit Plan Submitted by Allan.Conley@bcbsnc.com on 4-4-2002. General guidelines for participating in 2002 Target Project .
1. Contact business owner(s) responsible for the target project and inform them of your involvement in the project. 2. Arrange to attend project planning meetings. 3. Request and review copies of project plans, action plans, issues lists, resource plans, status reports, flow charts, procedure write-ups etc., to determine that sufficient documentation exists to control and monitor the project. 4. If the above requested documentation is not available or insufficient, immediately bring those to the attention of the responsible manager and recommend the documentation be created as appropriate. Also note the recommendation on the CAS Project Management Assessment Report. 5. Continue receiving and reviewing updated versions of the above documents throughout project. 6. Assess actions being taken and progress being made to address project issues. If progress is not being made (target dates not met, functionality not achieved) immediately bring those problems to the attention of the responsible manager, and recommend additional actions as appropriate, on the CAS Project Management Assessment Report. 7. Look for risks and gaps that may not have been identified or not considered (missed) by the area. If found, immediately bring your concerns to the attention of the responsible manager and the project leader. Share with the manager any recommended actions you think should be taken to address the risks/gaps, and also note them on the CAS Project Management Assessment Report. 8. As appropriate, test samples of source documents processed by the system or manually to ensure that systems and processes are working as intended. If errors are detected, immediately bring the errors to the attention of the responsible manager. Share your recommendations with the manager and also note them on the CAS Project Management Assessment Report. 9. Update the CAS Readiness Assessment Report every two weeks for presentation to the New Blue Strategy Team meeting. Specific schedule to be determined.

D:\Docstoc\Working\pdf\beaadb89-d666-4d66-a4be-4ead6a0dafd2.doc

11/01/09 2:25 AM

Page 1 of 19

Corporate Audit Services (CAS) Project Development Audit Plan

Project Development Audit Program
Section A. Feasibility Study OBJECTIVES: To determine if there was a feasibility study prepared that meets corporate guidelines/requirements and details the project plan as required. Step 1     Determine if a project feasibility study has been conducted and approved by client management and Information Services.

Y, N N/A

W/P Ref

Date

By

Does the study detail the scope of the project? Has a project budget been included? Does the budget appear realistic? Has the appropriate level of management reviewed and approved the study (including Information Services)?

Section B. Project Management OBJECTIVES: To determine if there is an adequate level of project management support. Step 1    Determine the makeup, authority, and support of the project team.

Is the Project Leader well versed in project management methodology? Is the appropriate level of management involved in the project? Does the project team include members from the user areas (all affected departments including Information Security) as well as systems development, , computer operations, audit, legal, compliance, Quality Assurance, vendors and all other appropriate areas? Does the project team have the level of authority to make the decisions concerning the project? Does the project team have the appropriate level of expertise including the technical (computer) area, and business area? Determine if a formal project request has been initiated.

 

Step 2   

Was a formal project requestgenerated and approved by upper management? Does the request include documentation of the expected benefits to be achieved? Is a cost/benefit analysis included? Determine if the Feasibility Study or the formal project request for the new process contains all relevant information, including:

Step 3



- reasons for the project - scope of the project - constraints of the project - costs and benefits of the project - plans and schedules - user requirements. Is the documentation for support of the project in accordance with corporate, and if applicable, State and/or Federal established procedures?

D:\Docstoc\Working\pdf\beaadb89-d666-4d66-a4be-4ead6a0dafd2.doc

11/01/09 2:25 AM

Page 2 of 19

Corporate Audit Services (CAS) Project Development Audit Plan Project Development Audit Program
Section C. Project Plan OBJECTIVES: To determine if a comprehensive Project Plan has been developed. Step 1      If a written project plan is available: Y, N N/A W/P Ref Date By

Are the critical phases determined? Does the plan require management/user approval at specified points? Do the time frames appear realistic? Can the project be canceled at early enough points? For major tasks and milestones, does it contain: - Start and end dates - Task duration - Prerequisites - Dependencies - Resource assignments Determine if the project plan covers the following:

Step 2 

  

The major phases of project development, including: - test phase - training for users - conversion - implementation. All applications and areas concerned? All vendors? All interfaces to/from the application? Review project administration activities for the following:

Step 3  

 

   

Are periodic status meetings held? If they are, determine if the following are covered: - documenting progress and highlighting variances from plan - preparing revised plans - budgetary control - work status reporting and review Are Action Items listed and Issues logged, maintained, reviewed, and updated during status meetings? Determine if the project plan is being followed, and that any deviations are documented, including extensions of the schedule. - Are all deviations documented? - Are all extensions approved by the project team and management? - Are all relevant parties notified of any extensions or changes to the project plan? Has an acceptance procedure been defined and documented (e.g., formal acceptance/approval of major phases, and of the total project prior to production implementation)? Does everyone involved in the project understand their level of involvement, and their roles and responsibilities? Has a project change procedure been established (approval method, forms, cut-off dates, etc.)? Are original and newly-identified risks documented, and are contingencies developed to address them?

D:\Docstoc\Working\pdf\beaadb89-d666-4d66-a4be-4ead6a0dafd2.doc

11/01/09 2:25 AM

Page 3 of 19

Corporate Audit Services (CAS) Project Development Audit Plan Project Development Audit Program
 Do manual work-arounds, needed to bridge missing functionality, include a staffing plan, a timeframe for termination, and projected volume capacity (how much work can we handle manually before we exhaust our capability to keep up timely)? Have any budget provisions been made for overruns, delays, or changes? Y, N N/A W/P Ref Date By



Section D. General Requirements OBJECTIVES: To determine that the application was adequately designed to meet the functional business requirements. Step 1     Review any changes to the application during all phases of the project to determine if they significantly change the project's original goal.

Are all changes documented? Are all changes reviewed and approved by the project team? Are all changes which affect the project scope approved by upper management? For any significant changes, is the project re-evaluated to determine the feasibility/costs/benefits? Determine if there are any legal or regulatory considerations which must be taken into account.

Step 2    

Will the new system comply with all known legal and regulatory requirements? If the new system is purchased from a vendor, are there legal/contractual requirements to ensure the system complies with current and future legal or regulatory requirements? If there any legal or regulatory reporting required of the system, has the responsible liaison group reviewed and approved them? Are HIPAA impacts considered and coordinated with the HIPAA Governance Committee? Determine if the software and hardware was purchased and evaluated in accordance with any established corporate, State or Federal guidelines.

Step 4

 

Was the purchase of the software and hardware approved by the appropriate department and upper management? If the purchase does not meet the purchase approval standards, was a waiver approved by the appropriate level of management/entity? Determine if the legal department was involved in the purchase of the software and hardware.

Step 5  

Were all contracts associated with this project reviewed by the Legal department? Is a copy of all contracts on file within the proper area?

Section E. Design Phase OBJECTIVES: To determine if the design meets the stated technical and functional requirements and fulfills the scope of the project.

D:\Docstoc\Working\pdf\beaadb89-d666-4d66-a4be-4ead6a0dafd2.doc

11/01/09 2:25 AM

Page 4 of 19

Corporate Audit Services (CAS) Project Development Audit Plan Project Development Audit Program
Part 1 Technical and Functional Specifications. Y, N N/A W/P Ref Date By

OBJECTIVES: To determine if the system design and business requirements have been sufficiently detailed and approved. Step 1       Determine if detailed user requirements have been developed and documented in the Functional Specifications.

Have Functional Specifications been approved by all interested parties (including Information Security if appropriate)? Are report specifications and frequency included? Is a Transaction (Audit) Trail to identify who changed what and when, included as a requirement? Have daily balancing requirements been identified? Are security access requirements defined? Is system response time included? Determine if the design of the system/new process is thoroughly documented in the Technical Specifications (accepted/approved by the major stakeholders).

Step 2

   

 

Are regular design sessions scheduled? Are all business areas covered for each application interfacing with the new system? Is the old system documented and understood? (If yes, this will help facilitate a smooth integration of the new system.) Are the following documented in the specifications: - data files - interfaces - procedures - screens - reports - imported and exported files - documents Are all existing accounts, products and services which impact the new system identified and considered in the new system‟s process ? Have Technical Specifications been approved by all interested parties (including Information Security)? Computer Program Design.

Part 2

OBJECTIVES: To determine if the Technical Specifications‟ computer program design is sufficiently detailed to permit the programmers to code the system. (This step might be omitted if we are dealing with a vendor purchased system.) Step 1   Determine if the program specifications are complete and consistent.

Do the Technical Specifications include descriptions of all programs? Are all program modules documented as to inputs, outputs, record layouts, purpose, flows, etc? Determine if the program documentation is complete.

Step 2 

Does the program documentation include all required information, including: - system flowcharts - data flow diagrams
11/01/09 2:25 AM Page 5 of 19

D:\Docstoc\Working\pdf\beaadb89-d666-4d66-a4be-4ead6a0dafd2.doc

Corporate Audit Services (CAS) Project Development Audit Plan Project Development Audit Program
- decision tables - program narratives Part 3 Input Design Y, N N/A W/P Ref Date By

OBJECTIVES: To determine if the input requirements are adequately defined and documented to ensure the integrity of data input into the system. Input can include: manual input via terminals from forms; tapes; or disks. Step 1  Review the documentation for the input requirements of the system.

Does the documentation include: - editing and validation/verification - balancing - security provisions - control totals - appropriate authorization? Determine if the input file definitions are defined and documented.

Step 2      

Have the input files been defined, including all record layouts for imported and master files? Have the databases been defined and record layouts documented? Determine if input data has been classified as per the Security Policy/Plan. Information can be classified as SECRET, PRIVATE, INTERNAL and UNCLASSIFIED. Have the security levels been established & defined for file & database access? Can access security be applied at the data element level? Are imported files date/time stamped? Does the application allow for batch or control totals? If yes:

Step 3   

Are the totals logged? Can the control totals be reconciled between input and output? Is the reconciliation documented? Determine if provisions have been made for data preparation and computer processing errors to be reviewed and re-entered correctly.

Step 4    

Can errors be detected and corrected prior to completion of the processing cycle? Has an individual been identified who will perform the review process? Has the frequency of the review been established? Has an individual been identified who will perform the re-entry function? Determine if provisions have been made for any internal tables or parameter files to be periodically reviewed by the users for accuracy.

Step 5   

Has the frequency of review of critical internal parameters/tables been established? Has an individual been identified who will make changes to these files? Is a procedure in place to correct errors?

D:\Docstoc\Working\pdf\beaadb89-d666-4d66-a4be-4ead6a0dafd2.doc

11/01/09 2:25 AM

Page 6 of 19

Corporate Audit Services (CAS) Project Development Audit Plan Project Development Audit Program
 If an error is found, do procedures address situation assessment and up/downstream implications/consequences? Determine if errors are/will be reviewed to determine the extent and type of outstanding errors for trend analysis purposes. Y, N N/A W/P Ref Date By

Step 6   

Has an effort been made to determine the types of errors to be reviewed? Has an individual been identified who will perform the analysis? Has the frequency of review been established? Determine if defaults and/or hard-coded values can be displayed for user review & approval.

Step 7  

Have the default values to be used been identified? Has an effort been made to determine when and how they can be changed?

Part 4 Processing Design OBJECTIVES: To determine if the processing requirements are defined and documented adequately. Step 1  Determine that the transactions and account balances are properly recorded on the Accounting systems, if applicable.

Are there operating procedures to verify that financial accounting records/accounts are properly posted? Step 2 Review the written daily processing balancing procedures. Are balancing procedures in place to ensure that the daily production process is properly controlled (through to the accounting systems if applicable)? Step 3 Determine if written procedures have been prepared to address processing errors. Do operating procedures explain all error codes and messages, and corrective actions for each? (Error codes for operators as well as data entry should be included.) Step 4 Determine if the application has provisions that prevent concurrent file/record updates (two people trying to update the same record simultaneously).   Is the file/record locked when one user is accessing in update mode? Are there appropriate error/warning messages provided? Determine if the application has controls to check for processing data integrity (for example run-to-run controls, etc. ) Determine if the system-generated transactions (e.g., discounts, fees, accruals, and other calculations) can be traced back to the source for reconciliation.  

Step 5

Step 6

  

Are these transactions readily identifiable? Are there adequate transaction (audit) trails for tracing purposes? Are extracted/exported files created from the most current data?

D:\Docstoc\Working\pdf\beaadb89-d666-4d66-a4be-4ead6a0dafd2.doc

11/01/09 2:25 AM

Page 7 of 19

Corporate Audit Services (CAS) Project Development Audit Plan Project Development Audit Program
 Are date and control totals included in extracted files? Y, N N/A W/P Ref Date By

Part 5 Output Design OBJECTIVES: To determine if the output requirements are adequately defined and documented. Step 1 Determine if output data has been classified according to the Security Policy/Plan. Information can be classified as SECRET, PRIVATE, INTERNAL and UNCLASSIFIED.

Step 2 



Review the adequacy of the documentation for output requirements. (Output includes reports as well as files.) Does the documentation include: - organization of the output - who is to receive the reports - retention of reports - retention of files - transaction (audit) trail considerations Are all departments' concerns considered? Determine if the output provides the users with the ability to control and ensure the completeness, accuracy, and authorization of the data.

Step 3

    

Do the reports include the ability to trace to the originator of each transaction? Do the reports include control totals, if applicable? Is there a means to verify the information included on the reports? Have the calculations used to develop data (accruals, fees, rates, etc.) been checked for accuracy? Have report routing and distribution procedures been established? Determine if there are adequate controls in place over negotiable instruments generated by the application. Negotiable items include checks, warrants, drafts, etc. Determine if provisions have been made for the user to scan output reports/datasets/files to detect obvious errors. These can include incomplete/missing runs, missing files, unreasonable values, incorrect report dates, formats, etc. Determine if Service Level Agreements between Operations, the user, and/or the vendor are in place or are being negotiated. These would include: response time, and system "up" time.

Step 4

Step 5

Step 6

Part 6 Interfaces OBJECTIVES: To determine if there is adequate security and controls over interfaces to the new application. Step 1   Determine if the application interfaces with other systems.

Is an appropriate level of security wrapped around the interfaces? Have the New Systems‟ interfaces been documented?

D:\Docstoc\Working\pdf\beaadb89-d666-4d66-a4be-4ead6a0dafd2.doc

11/01/09 2:25 AM

Page 8 of 19

Corporate Audit Services (CAS) Project Development Audit Plan Project Development Audit Program
Section F. Management Control Reports OBJECTIVES: To determine if there are adequate and effective management control reports designed into the system. Step 1  Determine if control reports produced by the system are sufficient. Y, N N/A W/P Ref Date By

Review the schedule of reports to determine if they include: - error reports - balancing reports - transaction registers - logs of all logon attempts - logs of all invalid logon attempts Determine if the reports include all necessary information as determined by the user department and management.

Step 2   

Are the user departments satisfied with the information produced on the control reports? Will these reports meet the user and management‟s needs? Will the reports satisfy internal and external auditors and regulators‟ needs? Determine the security and integrity over the management control reports.

Step 3   

Is access security sufficient to preclude anyone from altering the control reports? Are the reports distributed and reviewed by the appropriate people? Is report production and distribution frequency adequate? Determine if the use of management control reports is sufficient to address accountability/auditability. Do critical management reports require appropriate signoff or other indication of review? Are they to be reviewed in a timely manner by management?

Step 4  

Section G. System Documentation OBJECTIVES: To determine if the system documentation is complete. Step 1     Determine if an IS operations manual has been prepared prior to implementation of the system.



Does the manual include complete instructions on the system operation? Does the manual include run books with all program documentation? Does the manual include all job steps, including job sequence and prioritization? Does the documentation include: - each program function - hardware requirements - description of all console messages and operator response - disposition of input and output - identification of output file labels - restart procedures - run-to-run control points Were the operator's manuals used in testing?
11/01/09 2:25 AM Page 9 of 19

D:\Docstoc\Working\pdf\beaadb89-d666-4d66-a4be-4ead6a0dafd2.doc

Corporate Audit Services (CAS) Project Development Audit Plan Project Development Audit Program
 Are the operator's manuals readily available to all operators? Review the user department's proposed procedures for the new system. Y, N N/A W/P Ref Date By

Step 2   

Do the procedures include adequate segregation of duties? Do the procedures include all aspects of balancing? Do the procedures include security procedures: - password changes logon/logoff - PC/ terminal control - diskette/ tape/ disk control - access to the workstation - key control - report distribution and control Determine if the responsibility for on-going, post-implementation tasks has been assigned.

Step 3    

Has an IS system maintenance organization been determined? Has the user responsibility for quality control been determined? Has the user (and IS if applicable) responsibility for system balancing been assigned? Are there system change control procedures established? (For postimplementation changes rather than changes during the design phase.) Review the IS system documentation to determine if it appears complete.

Step 4 

Does the system documentation include the following topics: - error codes - system operations - samples and descriptions of reports and forms - system flowcharts - descriptions of interfaces to other systems, if applicable - system operating instructions, including scheduling runs, database descriptions, and layouts - file descriptions and layouts - data dictionary - field and record layouts Determine if the new user procedures include adequate segregation of duties, particularly in: - data entry and review - data entry and approvals - verification of the data entered - error corrections - balancing - password maintenance, access authorizations, and system usage (see also, Data Security section).

Step 5

Section H. Data Security OBJECTIVES: To determine if there is adequate security in place over application programs, data files, transactions, and database files. Step 1 Determine whether the operating system, the application, or both have controls in place to prevent unauthorized access to the new system‟s programs, files, databases, and transactions.



Is there a security system in place?
11/01/09 2:25 AM Page 10 of 19

D:\Docstoc\Working\pdf\beaadb89-d666-4d66-a4be-4ead6a0dafd2.doc

Corporate Audit Services (CAS) Project Development Audit Plan Project Development Audit Program
Is all access controlled (through a security access control program either internal or external to the new system)?  Is both a password and a LogonId required for access to the system?  Does the security system lock out a LogonID after a certain number of invalid signon attempts? Step 2 Compare transaction-level data security access levels to corresponding job assignments. Determine if access levels are adequate (just enough to perform job) and provide proper segregation.   Are there varying levels of security access (e.g., inquiry only, update nonmonetary transactions, update financial transactions, add/delete records) for different types of transactions? Are the levels appropriately assigned to the user department staff? Management and staff personnel who approve transactions should not have the authority to input or change the data. Determine if there are controls in place to prevent unauthorized access to the application‟s source and object code.  Y, N N/A W/P Ref Date By

Step 3   

Is access to the new system‟s source and object code controlled by a software configuration management system providing strict and formal control over software copying, modification, and migration? Are programmers prevented from normal access to production code and data? Does management review all programmer accesses to application code? Determine who has the ability to issue/change passwords, and then review the control over password administration.

Step 4    

Considering separation of duties, is this an appropriate person? Are the password assignments controlled by the data security department? If password assignments are controlled by the user department, are „password issuers‟ prohibited from having authority to input transactions? Are all password distributions supported by a written or electronic approval from the data owner? Review the display and storage of passwords.

Step 5    

When typed or displayed on a screen or displayed on a report, are passwords masked/obliterated? Are password files encrypted? Are passwords stored in a visible (human readable) file? Are passwords encrypted during transmission? Determine if all invalid/failed system access attempts are logged.

Step 6  

Are all invalid signon attempts captured? Has management assigned an individual to review the invalid signon attempts?

Section I. Hardware OBJECTIVES: To determine if all required hardware components are in place prior to implementation.

D:\Docstoc\Working\pdf\beaadb89-d666-4d66-a4be-4ead6a0dafd2.doc

11/01/09 2:25 AM

Page 11 of 19

Corporate Audit Services (CAS) Project Development Audit Plan Project Development Audit Program
Step 1 Determine if all required components have been identified, including computer (PC, server, mid-range, etc.), printer, modems, telecom lines and equipment, etc. Ensure that corporate bid requirements were followed for hardware procurement. Determine if there is a written plan and schedule for hardware installation. Y, N N/A W/P Ref Date By

Step 2

Step 3     

Was the equipment ordered within the established project budget? Does the plan include an installation schedule? Does the plan appear reasonable? Will all components be installed prior to the scheduled implementation? Have contingency plans been developed for hardware replacement? Determine if the equipment has been ordered in time for installation prior to implementation.

Step 4 

Is there a written contingency plan in case of delays? Determine if the equipment and hardware were delivered and set up as required.

Step 5    

Was the equipment tested when installed? Was there an acceptance sign-off by the IS Support department? Was the user department(s) notified of the installation date? Was the inventory list updated? Has data center facilities-management considered the impact of the new hardware (e.g., new data center, physical space, connectivity, etc.)?

Step 6

Section J. User/Departmental Procedures OBJECTIVES: To determine if adequate documentation and user manuals are developed for proper operations of the system. Step 1   Determine if the user manuals are complete prior to implementation.

Have the user manuals been drafted and reviewed by the user department? Will the manuals be printed and ready for use when the system is implemented? Review the user manuals to determine if they appear to be complete and relatively easy to understand.

Step 2 

Does the user manual include all information needed for: - preparation of input documents - data entry - documentation of output - balancing procedures - error correction/resolution - error messages for on-line systems or error reports - timing and distribution of output reports - security procedures
11/01/09 2:25 AM Page 12 of 19

D:\Docstoc\Working\pdf\beaadb89-d666-4d66-a4be-4ead6a0dafd2.doc

Corporate Audit Services (CAS) Project Development Audit Plan Project Development Audit Program
- password - logon and logoff procedures - descriptions of terminal screens - description of report layouts and fields? Were the user manuals used in testing? Were the manuals distributed to the correct staff? Were the manuals distributed to all applicable departments? Are the manuals readily available to all applicable staff? Determine if the user manuals cover the normal day-to-day processing, including commonly occurring errors, as well as month/quarter/year-end processing. Y, N N/A W/P Ref Date By

   

Step 3

   

Do the manuals include back-up procedures? Do the manuals include the special procedures for month-end, quarter-end, and year-end processing? Do the manuals include a list of commonly occurring error messages, easily referenced? Are correction procedures covered? Determine if departmental documentation for the new system is up to date.

Step 4  

Are departmental Process Flows complete and adequate? Are departmental Process Procedures complete and accurate?

Section K. Training Plan OBJECTIVES: To determine if a training plan was developed for the project and if user training was adequate. Step 1 Step 2  Determine if a formal training plan was developed. Determine what the training plan contains:

 

Are all aspects of the system covered including: - data entry - backups - management reporting - disaster recovery - user operations - computer operators - balancing and reconciliations? Does the training include vendor-recommended techniques? Did the training plan incorporate feedback from the users? Review the training plan to determine if training will be completed prior to implementation of the system.

Step 3   

Will the most critical employees be trained first? Will end-users be used to train others in lieu of the Training Department? Are differences between training regions and the production system taken into account during training? Determine who will be trained and at what level.

Step 4

D:\Docstoc\Working\pdf\beaadb89-d666-4d66-a4be-4ead6a0dafd2.doc

11/01/09 2:25 AM

Page 13 of 19

Corporate Audit Services (CAS) Project Development Audit Plan Project Development Audit Program
   Will there be several levels of training: Management, data entry, supervisory? Will all appropriate levels of staff be trained? Will there be technical training for computer operators? Y, N N/A W/P Ref Date By

Section L. Testing OBJECTIVES: To determine if the test plan includes all aspects of the new system, the system is adequately tested prior to implementation, and all unexpected results are thoroughly resolved. Step 1    Determine if the project team has developed a test plan.

Is the test plan written? Does it include IS unit/functional testing, and user acceptance tests? Are the users involved in developing the testing plan (e.g., producing test scripts or scenarios)? Determine if all aspects of the system, as outlined in the detail Functional Specifications, will be tested, including, but not limited to: - data entry - editing - reports - calculations - error reporting - interfaces with other systems - network communications - print handling Are all critical functions tested? Are all existing capabilities tested? Are all changes tested? Is end-to-end system testing conducted prior to implementation? Determine when testing will take place and ensure it will be completed prior to implementation.

Step 2

   

Step 3 

Does the test plan allow for re-testing of errors and changes? Determine if a parallel test will be run. Note: this entails running the old and new systems with the same data, and comparing the results.

Step 4

 

Have the criteria for the termination of the parallel run been identified? Note: parallel runs may continue until both systems are in sync. Does it include Information Systems and user acceptance/signoff? Review month-end, quarter-end, and year-end tests. Note: If the new system performs different/special month-end, quarter-end, and year-end processing, then these processes should be forced and tested.

Step 5

D:\Docstoc\Working\pdf\beaadb89-d666-4d66-a4be-4ead6a0dafd2.doc

11/01/09 2:25 AM

Page 14 of 19

Corporate Audit Services (CAS) Project Development Audit Plan Project Development Audit Program
Step 6 Determine if volume and/or stress testing will be done. Note: Volume testing should include a “normal” processing day‟s transactions as well as a high-volume day‟s transactions, printing, backups, file transfers, interfaces, etc. Note: Stress testing should try to “overload” the system and provide an indication of maximum volume/thru-put the current configuration can be expected to handle. And, stress testing should test system response time in the stressed state. Y, N N/A W/P Ref Date By

Section M. Test Procedures OBJECTIVES: To determine if adequate test procedures have been developed. Step 1           Determine if test data have been prepared.

Does test data include all possible conditions, including errors? Have test scripts been prepared? Have the test files been defined? Are the appropriate regions between test and production synchronized? Have expected test results been defined prior to actual testing? Do test scripts include all expected test results? Have predetermined results been set up in advance (e.g., in a testing log)? Have procedures been developed to monitor test results (e.g., in a log)? Are the detail steps for the tests defined? Are the controls over the application interfaces, e.g., access, balancing, data integrity, etc., included in the testing plan?

Section N. Test Results OBJECTIVES: To determine if test results are consistent with expected results, and that unexpected results are monitored and addressed. Step 1     Determine if procedures have been developed to evaluate the test results.

Are the users included in the testing process and evaluation of the results? Are unexpected test results logged, monitored, and resolved? Is there a problem resolution process for those tests not meeting the expected results ? Note: Ask for a resolution-process flowchart. Is the logging and problem resolution consistent with other implementations? Review the test results and determine if unexpected results are handled properly.

Step 2    

Are unexpected test results evaluated to determine the reasons for the variance? If needed, are program corrections made, or manual work arounds established? Are the problems re-tested after correction? Have all testing issues been resolved, or are work-arounds in place prior to implementation?

D:\Docstoc\Working\pdf\beaadb89-d666-4d66-a4be-4ead6a0dafd2.doc

11/01/09 2:25 AM

Page 15 of 19

Corporate Audit Services (CAS) Project Development Audit Plan Project Development Audit Program
Step 3 Follow those unexpected results deemed critical, to ensure adequate resolution. Determine if the tests having unexpected results were adequately re-tested after correction (to the program, etc.). All results which deviated from the expected should be re-tested. Y, N N/A W/P Ref Date By

Section O. Conversion Plan OBJECTIVES: To determine if the system conversion plans are adequate. Step 1    Obtain and review the conversion plan.

    

Is the conversion plan written? Is the data conversion approach defined? Determine the type of conversion. Some conversion scenarios include: - full conversion - “shell accounts” and update later - interim and “bridge” process - combination Has the conversion plan been approved by management and user departments? Are all source systems identified? Are all components identified (e.g., disks, tapes, telecom connections, etc.)? Are the conversion rules and rationale documented? Is a contingency plan defined? Separation of duties.

Step 2  

Are the appropriate IS and user operations staff available? Are adequate separation of duties maintained during the conversion processes? Step 3 Review the plans and procedures for balancing         Are all needed files for the conversion identified? Are all fields slated for conversion mapped to the new system? Are there plans for verification of the data coming into the new system? Are there procedures developed for balancing the number of records and totals for all monetary data elements and accounts for both the source files (before the conversion) and the resulting files (after the conversion)? If so, are the procedures robust and adequate? Are there before and after file compares planned, i.e., are the input files/records reconciled to the receiving files/records? Note: This should include all dollar fields, record counts and accounts. Are appropriate edits and error reports in place to ensure data integrity? Is a before and after conversion run planed to compare the old to new systems? Determine if the rejects and errors will be re-entered and properly accounted for during the conversion.

Step 4   

Is there monitoring of all rejected transactions in the conversion? Are these rejects researched and re-entered prior to cut-over to the new system? Are the errors/rejects considered as reconciling items when balancing?

D:\Docstoc\Working\pdf\beaadb89-d666-4d66-a4be-4ead6a0dafd2.doc

11/01/09 2:25 AM

Page 16 of 19

Corporate Audit Services (CAS) Project Development Audit Plan Project Development Audit Program
Step 5 Determine if there will be a parallel run after the actual conversion. Ensure that the results of the parallel run will be reviewed prior to the actual switch-over to the new system. Y, N N/A W/P Ref Date By

 

Are run-to-run totals produced during parallel runs? Note: At the very least, after conversion, the new system should run the last day‟s processing so the results can be compared to the old system‟s. Were the results of the parallel run consistent with expectations? If a manual conversion, determine the procedures to verify all entries.

Step 6

Section P. Final Acceptance/Approval of Conversion Process. OBJECTIVES: To determine if criteria and procedures have been established for the new system‟s acceptance into the production environment.        Have standards for the final acceptance been established? Are all stakeholders required to approve final acceptance of the tested system before the conversion can take place? Has user department reviewed the system performance and operational test results, and approved the final results? Have all problems encountered in the parallel run been resolved prior to full conversion? After the conversion, was a sample of pre-selected records from the old system compared to the corresponding records on the new system? Has the user and other stakeholder departments signed off on the conversion (accepted the conversion results) prior to discontinuing the old system? Was audit involved in the conversion?

Section Q. Implementation Plan OBJECTIVES: To determine if the implementation of the system was adequately planned. Step 1 Determine if all software required for the successful implementation has been written (if coded in-house) or obtained from the vendor (i.e., every piece of necessary coding is in place). Determine if all required JCL and computer operations procedures have been written and included. Determine if any there are any required new or special forms.

Step 2

Step 3   

Have the forms been reviewed and signed off by all involved departments? Is there a working supply of special forms available prior to cut-over? Did audit have an opportunity to review the form(s) for comments? Determine if there is a written implementation plan in place.

Step 4 

Does the plan include responsibilities for all areas involved? Determine if there is a problem resolution scheme in place for the installation/conversion phase.

Step 5 

Will "help desk" personnel be available?
11/01/09 2:25 AM Page 17 of 19

D:\Docstoc\Working\pdf\beaadb89-d666-4d66-a4be-4ead6a0dafd2.doc

Corporate Audit Services (CAS) Project Development Audit Plan Project Development Audit Program
   Is the vendor or programmer(s) available for problem resolution? After installation and "shake-out," is the maintenance staff ready and able to take over? Do the users know where to go to get help? Review the backout/recovery plan. Y, N N/A W/P Ref Date By

Step 6     

Does the backout plan define when the backout would be invoked? Does the backout plan include procedures necessary to re-implement the old system, if needed? Does the backout plan require approval from management prior to backout implementation? Does the backout include procedures for all affected areas? Does the backout plan include a means to notify all areas/ users that the installation failed?

Section R. Post-Implementation Review OBJECTIVES: To determine if the results of the new system implementation met the original objectives. Step 1    At the project's conclusion determine whether the project met the objectives defined in the original proposal.

Were the expected benefits of the new system realized? Does the system perform as expected? If there were differences found between expectations and actual results, were they investigated and were the dispositions noted? Step 2 Determine if the cost/benefit analysis was correct. Compare the actual vs. budgeted costs and benefits. Were the actual costs within reason? Section S. User Satisfaction OBJECTIVES: To determine if the user is satisfied with the new system and evaluate what points should be considered in another project. Step 1 Determine if Project Management will obtain user satisfaction feedback. If feedback is obtained, determine if the user is satisfied with the operations of the new system. 

Step 2    

Is there a need for ongoing training? Have all problems been corrected? Does the system meet the user requirements? Does the system provide the user all the planned functionality and information? Determine if there are any suggestions for improvement

Step 3 

Did project management receive any suggestions such as: - improved or more efficient operations? - improved training? - improved reports - smoother installations?
11/01/09 2:25 AM Page 18 of 19

D:\Docstoc\Working\pdf\beaadb89-d666-4d66-a4be-4ead6a0dafd2.doc

Corporate Audit Services (CAS) Project Development Audit Plan Project Development Audit Program
- smoother conversion? Section T. Backup and Recovery Procedures OBJECTIVES: To determine if there are adequate backup and recovery procedures developed for the system (once it is in production). Step 1  Determine if disaster recovery and system restart procedures have been developed (for after the new system is in production). Y, N N/A W/P Ref Date By

Have the recovery and the restart procedures been written? Determine if there are procedures for periodic system backup.

Step 2     

Determine how often backups will be performed? How long will the backups be kept? On what media will the backups be kept? (Tape, disk, diskette) Will the media be stored off-site? Have the backup procedures been written? Review the procedures to determine if they are adequate.

Step 3  

Do the procedures include all foreseeable circumstances? Do the plans include recovery of - hardware and software? - PCs or terminals - printers or telecommunication lines to remote printers - software (and documentation) - modems - phone lines - office space and equipment - special forms Determine the retention period for the backups.

Step 4    

Determine how long the daily backups will be kept. Note: Should be at least two cycles of information at a minimum. Will the month-end backups be kept for a different period? Will the quarter-end backups be kept for a different period? Will the year-end backups be kept for a different period? If this is a critical business system, review the hot site contract and testing schedule. Is the hardware and supporting equipment for the new system included in the hot site contract? Will the new system be scheduled for recovery testing in the next hotsite test?

Step 5  



NOTE: DO WE NEED A SECTION ON THE DEPARTMENT/STAKEHOLDERS‟ BUSINESS CONTINUITY PLAN?

D:\Docstoc\Working\pdf\beaadb89-d666-4d66-a4be-4ead6a0dafd2.doc

11/01/09 2:25 AM

Page 19 of 19


				
DOCUMENT INFO