Document Sample
Auditing-Large-and-Complex-IT-Projects Powered By Docstoc
					      Auditor General
      State Of Florida

Auditing Large and Complex
         IT Projects
Basic IT project risks.
Example IT projects in Florida.
Issues and challenges to auditing large IT
Checklist of suggested audit questions.
Suggestions for further reading.
     Basic IT Project Risks
Contract management.
Development and Implementation.
  IT Project Risks - Planning
Lack of clearly defined goals, objectives, and
Lack of identification of project risks.
Poorly defined project management structure.
Unrealistic budget – time and dollars.
Lack of stakeholder & user buy-in.
Extent and impact of business process
reengineering not sufficiently addressed.
Inadequate basis for development of
procurement criteria.
– No established baseline for measuring cost savings
IT Project Risks - Procurement
Poorly defined statement of work.
Lack of basis for competitive solicitation.
Flawed evaluation of proposals.
Litigation after award.
     IT Project Risks – Contract
Contract terms that don’t adequately protect the state’s
Poorly defined deliverables.
Lack of agreed upon performance measures.
Payment without suitable performance.
Lack of recourse for non-performance or poor
performance by contractors.
Vague ownership provisions – data, software, hardware.
Vague or missing provisions regarding data security.
Poor or lacking termination & transition clauses to
protect state’s interest should either party cancel.
IT Project Risks - Development and
 Implementation of software that does not
 function properly & include good controls.
 Poor data security.
 Inadequate training and knowledge transfer,
 resulting in lack of sufficient knowledge to
 operate, maintain, and use the new system
 Cost overruns.
 Failure to meet implementation deadlines
 Over-customization of vendor software making
 future upgrades costly to obtain and implement.
 Loss of data integrity during conversion/cutover.
        IT Project Risks – Post
Security vulnerabilities as a result of security not
being hardened after implementation.
Integrity of system compromised through poor
maintenance and change control.
On-going reliance on contractors.
In-house knowledge not sufficient to maintain
system or infrastructure.
Unable to turn off old systems that were to be
Unable to sustain user buy-in.
   Example Florida Projects
My Florida Marketplace.
People First.
MyFlorida Alliance.
Various educational entities.
    MyFlorida Marketplace
A Web-based e-procurement system for state
Developed by customizing Ariba software.
Application Service Provider – Accenture.
Responsible State agency: Department of
Management Services.
Total contract cost : $93.9 million.
Contract term: October 2002 through November
Funded through 1% transaction fee on
MyFlorida Marketplace – Planning
    and Procurement Issues
Cost-benefit and risk analysis not
conducted prior to decision to outsource
and issuance of ITN.
Insufficient involvement of key end-users
and stakeholders in development of ITN.
Lack of significant baseline data for
planning and analysis.
No mechanism to capture and track
statewide costs associated with MFMP.
MyFlorida Marketplace – Contract
 and Project Management Issues
System not formally accepted prior to
Heavy reliance on ASP without sufficient
monitoring of performance.
 – Limited monitoring by DMS.
 – Third-party monitor’s duties strayed from
   monitoring ASP.
  MyFlorida Marketplace – IT
Functionality and Control Issues
Deficient change control process.
System performance and capacity
management needed improvement;
performance issues existed.
Disaster recovery plan not timely approved
and lacked important provisions.
Insufficient back-up provisions.
Deficiencies in security controls.
  MyFlorida Marketplace – IT
Functionality and Control Issues
Data integrity issues.
– Problems with accounting system interface.
– Problems with attached scanned documents.
Declining use of system by State
– System functionality issues.
– System performance issues.
– Workflow inefficiencies.
             People First
An HR outsourcing initiative.
Includes a Web-based enterprise-wide
ERP system supporting:
– HR administration.
– Benefits administration.
– Payroll administration.
– Staffing administration.
           People First
Service provider – Convergys.
9-year contract.
$349.9 million.
People First application built using SAP
System phased in between May 2003 and
January 2005.
Convergys provides Florida with a SAS 70
report on its service center.
   People First – Planning and
      Procurement Issues
Cost-benefit and risk analyses not
performed prior to release of ITN.
Inaccuracies in cost estimates within
Business Plan.
No system to track statewide project cost.
Deficiencies in evaluation and negotiation
People First – Contractual Issues
Legal records retention requirements not
included in contract.
No provision for State to approve new or
changes in subcontractors.
No provision for subcontractors to obtain
background checks.
Many deliverables not timely provided.
Additional amounts paid to third-party monitor
for performing services already required.
    People First – Operational
System functionality problems and errors.
Lack of written security guidelines.
Off-shoring of State employee personnel
Planned system components not
implemented, requiring workarounds by
the agencies.
       MyFlorida Alliance
An effort of the State Technology Office to
reengineer its IT functions and governance
structure through outsourcing many of its
primary functions.
STO was responsible for centralized
management of IT for the executive
branch of Florida government.
MyFlorida Alliance – Functions to
        be Outsourced
Enterprise e-communications.
Enterprise technology services desk.
Enterprise applications management.
Enterprise data center operations and
Enterprise portal, enterprise security,
various others planned.
      MyFlorida Alliance
Two prime contractors – Bearing Point and
Contracts signed August 13, 2003.
7 year term.
$324.7 million.
Procurement method: ITN.
All agreements since terminated by STO.
STO abolished in law effective 7/1/07.
       MyFlorida Alliance
Inadequate planning & documentation to support
decisions to outsource or use ITN method.
Deficiencies in proposal evaluation and
negotiation, limiting fairness and competition.
Contracts lacked provisions to protect the State
and did not pinpoint total cost to State.
Cost savings analyses questionable or not
Replacement for the State’s general ledger
accounting and cash management systems.
Project beginning date - 9/8/2003.
Original planned rollout in three waves of
agencies going live, from July 2005 through
December 2005.
Rollout schedule amended five times.
Project suspended 5/17/2007, with $89 million
spent to date.
IT Projects at Florida’s Educational
 University ERP implementations – for
 example, University of Florida.
 – Insufficient system testing
 – Insufficient staff training
 – Functional system problems
 – System governance issues
 – IT security and control issues
 Various community colleges and school
Issues and Challenges to Auditing
         Large IT Projects
When to audit.
 – During the project?
 – Postaudit?
Impact of outsourcing.
 – Authority to audit contractor & subcontractors
 – Contractor responsiveness to audit requests
 – Applicability of SAS 70 audits, if available, to non-financial audit
Lack of sufficient entity knowledge of system.
 – Difficulty in extracting data needed for audit
Availability of entity staff to timely respond to audit
requests during implementation or post-implementation.
 Suggested Audit Checklist
Contract Provisions and Management.
Project Management.
 Suggested Further Reading
MyFlorida Marketplace
– 2007-076 (IT Audit)
– 2006-015 (IT Audit)
– 2005-116 (Operational Audit)

People First
– 2007-087 (Operational Audit)
– 2005-047 (Operational Audit)
 Suggested Further Reading
MyFlorida Alliance
– 2005-008

University of Florida
– 2006-040 (Operational Audit)
– 2006-145 (IT Audit)

Chapter 2007-115, Laws of Florida
– Abolishes STO and creates AEIT

Shared By:
Tags: Audit, ing-L
Description: Auditing-Large-and-Complex-IT-Projects