IT Performance Assessment Calculation Tool

Performance Metrics Explanation of the Spreadsheets Customer Service measures how the agency's program areas view their agency's IT organization. Completed by: Program Area and Executive Staff (Determined by each agency) IT Governance addresses how IT decisions are made, who gets to make them, and who is held accountable. The Governance structure determines what the IT management team will be developing and implementing for the business program areas. Completed by: Agency Executive Staff (Determined by each agency) Customer Service Governance Delivery Delivery addresses the agency's IT operations, application support and security management. Completed by: Agency IT Managers and Lead Personnel (Determined by each agency) Project & Contract Management Project and Contract Management addresses how well the agency and its IT team(s) plan, manage, and control projects both with internal staff and through contracts. Completed by: Agency Program Areas and IT Managers (Determined by each agency) Financial accountability section focuses on how the agency's fiscal teams interact with their agency's IT organization and the Office of Budget. Completed by: Agency's Fiscal Team (Determined by each agency) Employee Development area measures how the agency's HR representative(s) address the their agency's IT organization requirements and initiatives Completed by: Agency's HR Representative(s) and IT Managers (Determined by each agency Financial Accountability Employee Development Statement Applicability Very Rarely Statement applicable 0-24% of the time Rarely Sometimes Usually Always Does not Apply Statement applicable 25-49% of the time Statement applicable 50-74% of the time Statement applicable 75-90% of the time Statement applicable 91-100% of the time Statement is not applicable Definitions 16d4685c-455c-4c7c-bbe5-761395f48142.xls 8/14/2008DRAFT Definitions 16d4685c-455c-4c7c-bbe5-761395f48142.xls 8/14/2008DRAFT To be completed by Program Area and Executive Staff IT Performance Measures Customer Service Strategy and Planning 0-24% Our IT Organization… CS1 Works collaboratively with us in the strategy and planning processes - understanding our needs and identifying ways in which technology can help. Actively engages our area in making key decisions and setting technology direction. CS3 In developing and enhancing applications, provides accurate cost and other resource estimates - for both development and on-going support. Involves the appropriate stakeholders in defining requirements and designing solutions to meet business needs. Very Rarely Statement Applicability 25-49% 50-74% 75-90% Rarely Sometimes Usually 91-100% Always Comments CS2 CS4 CS5 Performs thorough analysis of business processes (existing and planned) and designs comprehensive solutions. Project and Contract Management 0-24% Our IT Organization… CS6 Manages IT projects through a project management methodology that is clearly defined, understood, and executed. Maintains control of the decision making and other management functions when contracting a project to a vendor. Very Rarely Statement Applicability 25-49% 50-74% 75-90% Rarely Sometimes Usually 91-100% Always Comments CS7 Customer Service 8/14/2008DRAFT 0-24% Our IT Organization… CS8 Effectively monitors contract compliance, evaluates service and deliverable quality, reconciles deficiencies, and reports results. Effectively oversees projects as funds are being spent, to ascertain if stated objectives are being met and costs and schedules are on target. Provides solutions on schedule and within allocated budget. Very Rarely Statement Applicability 25-49% 50-74% 75-90% Rarely Sometimes Usually 91-100% Always Comments CS9 CS10 Service and Quality Management 0-24% Our IT Organization… CS11 Provides information and services in which we have confidence. CS12 Provides solutions that meet business requirements, are reliable, perform well, and are free of defects that disrupt service. Provides an adequate level of after-hours support to accommodate our business needs. CS14 Provides adequate and timely training when implementing new or changed systems. CS15 Resolves requests for workstation-related services to our satisfaction, in a timely manner. Very Rarely Statement Applicability 25-49% 50-74% 75-90% Rarely Sometimes Usually 91-100% Always Comments CS13 Communications and Problem Management 0-24% Our IT Organization… CS16 Is timely, comprehensive, and clear in communicating with our area. Very Rarely Statement Applicability 25-49% 50-74% 75-90% Rarely Sometimes Usually 91-100% Always Comments Customer Service 8/14/2008DRAFT 0-24% Our IT Organization… CS17 Provides sufficient notice of planned system outages. CS18 Is sensitive to business requirements and responds to problems in a timely manner, limiting disruption of services. Goes above and beyond in solving problems and preventing their reoccurrence. CS20 Through its help desk, makes a positive contribution to our ability to perform our mission - answering questions and resolving issues in an accurate and timely manner. Very Rarely Statement Applicability 25-49% 50-74% 75-90% Rarely Sometimes Usually 91-100% Always Comments CS19 Signature Date Customer Service 8/14/2008DRAFT To be completed by Agency Executives IT Performance Measures Governance Statement Applicability 0-24% 25-49% Rarely 50-74% Sometimes 75-90% Usually 91-100% Always Don't Know/ Does Not Apply Our Agency's approach to IT governance … G1 Ensures that the most critical IT projects in support of our agency's business goals will be approved and assigned resources. And related responsibilities and roles have been clearly defined, documented, and communicated within the agency. Actively engages business partners, senior executives, and key stakeholders. G4 Contributes to an enterprise-wide perspective and more cost-effective solutions for the agency. G5 Includes an effective means of communicating between our agency's executives, IT organization, and business partners. Develops/guides the IT funding strategy for the agency. G7 Supports adherence to the Information Technology Bulletins (ITBs), Memorandums of Understanding (MOUs), and Management Directives (MDs). Very Rarely Comments G2 G3 G6 Signature Date Governance 8/14/2008DRAFT To be completed by Agency IT Managers and Lead Personnel IT Performance Measures Delivery Operations Server Management Statement Applicability Our Agency … D1 Meets or exceeds the defined/documented availability requirements of the agency. D2 Establishes and uses monitoring protocols for applications and servers. D3 Works toward consolidating servers, seeking the appropriate support (e.g. knowledge or physically helping with the design) from within the agency, working with other agencies, and/or OA/OIT as needed. 0-24% Very Rarely 25-49% Rarely 50-74% Sometimes 75-90% Often 91-100% Always Does Not Apply Comments Internal Service Level Agreements (SLAs) No D4 Our IT organization has established and documented internal service level agreements (SLAs) and/or memorandums of understanding (MOUs) with our business users for mission critical services. Yes Don't Know Questions D5-D7 are specific to an agency with internal SLAs. If you responded "No" or "Don't Know" to D4, please proceed to D8. Statement Applicability Our IT Organization … D5 Shares the comparison of SLA and/or MOU targets to actual results with our business partners. D6 Deploys a corrective action process for SLA and/or MOU targets that are not met. D7 Meets or exceeds established SLA and/or MOU targets. 0-24% Very Rarely 25-49% Rarely 50-74% Sometimes 75-90% Often 91-100% Always Does Not Apply Comments Delivery 8/14/2008DRAFT External Service Level Agreements (SLAs) No D8 Our agency has documented SLAs and/or MOUs in contractual form with our critical service providers/ suppliers. Questions D9-D10 are specific to an agency with external SLAs. If you responded "No" or "Don't Know" to D8, please proceed to D11. Statement Applicability Our Agency … D9 Compares taregt and actual SLA and/or MOU results with suppliers. D10 Deploys a corrective action process for SLA and/or MOU targets that are not met. 0-24% Very Rarely 25-49% Rarely 50-74% Sometimes 75-90% Often 91-100% Always Does Not Apply Yes Don't Know Comments Continuity of Government No D11 Our agency has a documented plan to ensure the continuity of essential functions under emergency circumstances. Yes Don't Know Question D12 is specific to an agency with a Continuity of Government Plan. If you responded "No" or "Don't Know" to D11, please proceed to D13. Statement Applicability Our Agency … D12 Tests the Continuity of Government plan to ensure the correct steps are documented. 0-24% Very Rarely 25-49% Rarely 50-74% Sometimes 75-90% Often 91-100% Always Does Not Apply Comments Network No D13 Our agency has a documented procedure for monitoring and managing network performance. Questions D14- D15 are specific to an agency with a network monitoring plan. If you responded "No" or "Don't Know" to D13, please proceed to D16. Statement Applicability Our Agency … D14 Provides an annual 99.8% or better availability of network resources, except during scheduled outages. D15 Monitors and categorizes the number of failures within our agency's network and provides a corrective action plan to fix the root cause of failures. 0-24% Very Rarely 25-49% Rarely 50-74% Sometimes 75-90% Often 91-100% Always Does Not Apply Yes Don't Know Comments Delivery 8/14/2008DRAFT Data Management/Storage Management No D16 Our IT organization proactively addresses system and data integrity. Questions D17- D19 are specific to data and storage management. If you responded "No" or "Don't Know" to D16, please proceed to D20. Statement Applicability Our Agency … D17 Establishes a routine backup process for data. D18 0-24% Very Rarely 25-49% Rarely 50-74% Sometimes 75-90% Often 91-100% Always Does Not Apply Yes Don't Know Comments Develops procedures and provides for offsite storage for all backup data. Tests database backup recovery plans on an annual basis. D19 Application Support Statement Applicability Our Agency … D20 Provides production application availability during all regular business hours, except during scheduled outages. D21 Maintains application availability in the quality assurance (test) environment during all work hours, except during scheduled outages. Maintains system response times within an acceptable range during the critical periods of the day. Documents the time frame when maintenance or support will occur and communicates planned outages to business partners in a timely manner. Clearly defines the steps required to move changes from test to production. 0-24% Very Rarely 25-49% Rarely 50-74% Sometimes 75-90% Often 91-100% Always Does Not Apply Comments D22 D23 D24 Delivery 8/14/2008DRAFT Security Statement Applicability Our Agency … D25 Documents the procedures for tracking security issues and has an escalation process when issues require additional support. D26 Has identified and controls potential backdoors to the infrastructure. D27 Routinely updates anti-virus software in accordance with OA/OIT direction. D28 Monitors and controls threats pertaining to the network. D29 Conducts annual network penetration tests and security assessments and implements corrective action plans. 0-24% Very Rarely 25-49% Rarely 50-74% Sometimes 75-90% Often 91-100% Always Does Not Apply Comments Signature Date Delivery 8/14/2008DRAFT To be completed by: Program Areas and IT Managers IT Performance Measures Project and Contract Management Project Management Statement Applicability Our Agency … PM1 Manages IT projects through a project management methodology that is clearly defined, understood, and executed. PM2 Shares best practices, documentation, and experiences with the IT team and business partners. PM3 Defines project charters, scope, work plans, and schedules for new initiatives and enhancements regardless of size. PM4 Develops contingency plans to avoid time or cost overruns when issues or risks are identified. PM5 Provides communication plans to disseminate information on project goals and progress, and to generate support for the project. PM6 Uses a formal change management process to address changes in scope. 0-24% Very Rarely 25-49% Rarely 50-74% Sometimes 75-90% Usually 91-100% Always Does Not Apply Comments PM7 Monitors and controls projects or enhancements throughout all project phases. Collaboration Statement Applicability Our Agency … PM8 Identifies collaborative multi-agency projects and builds partnerships with other agencies. PM9 Reviews existing solutions within the Commonwealth prior to submitting a new project or enhancement and works effectively when interdependencies exists with other agencies. PM10 Shares lessons learned with other agencies. 0-24% Very Rarely 25-49% Rarely 50-74% Sometimes 75-90% Usually 91-100% Always Does Not Apply Comments Project & Contract Mgmt 8/14/2008DRAFT Procurement and Contract Management Statement Applicability Our Agency … PM11 Engages DGS early and throughout the procurement process. PM12 Exercises control over the IT contracts and IT contractor personnel. 0-24% Very Rarely 25-49% Rarely 50-74% Sometimes 75-90% Usually 91-100% Always Does Not Apply Comments PM13 Maintains control of decision making and other management functions when contracting a project to external vendors. PM14 Uses a procurement approach that results in fair and open competition. PM15 Develops contracts that include a clear distinction between development and support/maintenance activities. PM16 Develops time and material contracts, or portions of contracts, that include clear and comprehensive service level standards tied to critical business functions. PM17 Formally evaluates project change requests and includes the participation by fiscal staff when costs are effected. PM18 Develops contracts that include specific requirements for transition and knowledge transfer. Effectively monitors contract compliance, evaluates service and deliverable quality, reconciles deficiencies, and reports results to stakeholders. PM19 Signature Date Project & Contract Mgmt 8/14/2008DRAFT To be completed by: Agency's Fiscal Team IT Performance Measures Financial Accountability Statement Applicability 0-24% 25-49% Rarely 50-74% Sometimes 75-90% Usually 91-100% Always Don't Know/ Does Not Apply Our Agency … FA1 Uses a formal process for reviewing proposed expenditures in technology that is linked with budget and includes program area participation. Very Rarely Comments FA2 Has an expenditure review process that includes the documentation of a business case*. FA3 Considers the integrity of our enterprise architecture as part of our expenditure review process. FA4 Considers proposed expenditures in the context of the current inventory of IT assets (hardware, software, applications, etc.) to see if they can be eliminated or reduced. FA5 Has a formal process to identify and address possible IT expenditures and proposals that are conflicting, redundant, and/or of limited business value. FA6 Has a formal process for overseeing projects as funds are being spent to ascertain if stated objectives are being met and costs and schedules are on target. FA7 Conducts post implementation reviews to determine if expected benefits have been achieved. FA8 Has and uses a formal process for comparing actual to original estimates for both development and support costs. Submits budget requirements in compliance with established target dates, allowing sufficient time for required activities and reviews. FA9 Financial Accountability 8/14/2008DRAFT Statement Applicability 0-24% 25-49% Rarely 50-74% Sometimes 75-90% Usually 91-100% Always Don't Know/ Does Not Apply Our Agency … FA10 Proactively engages the Budget Office/Comptroller Operations when there is a change to planned expenditures. Very Rarely Comments FA11 Our agency manages its part of the procurement process in compliance with established target dates, allowing sufficient time for required activities and reviews. Signature Date * The documented business case should explain: a) how an expenditure will support the agency's mission and is economically beneficial b) whether the function or project should be or is being performed by another agency c) whether the opportunity exists to collaborate with another agency with a shared or similar mission d) the other alternatives that were considered Financial Accountability 8/14/2008DRAFT To be completed by: Agency's HR and IT Management Teams IT Performance Measures Employee Development Statement Applicability 0-24% Very Rarely 25-49% Rarely 50-74% Sometimes 75-90% Usually 91-100% Always Don't Know/ Does Not Apply Comments ED1 In our IT Organization, voluntary turnover is 7% or less annually (not including retirements). ED2 Our agency has contingency plans that address the projected turnover rate within the IT area. ED3 IT staff members have a job description that accurately describes the work they perform. ED4 Employees in our IT Organization have individual development plans, progress towards which is reviewed at least once annually. ED5 Employees in our IT Organization have been made aware of performance expectations and goals. Performance and success in goal achievement are reviewed at least once annually. ED6 Our IT Organization deliberately weighs alternative approaches (e.g., contractor replacement, intern program, staff augmentation) to achieve the best fit/value when additional staff is required. PM7 Personnel decisions in our IT Organization are based upon merit factors. Signature Date Employee Development 8/14/2008DRAFT