Best Practices in Business Continuity Planning

Best Practices in Business Continuity Planning Roberta J. Witty, Research Director Information Security Strategies Big Ah Ha 9-11 Forever Changed Business Continuity Planning These slides are for internal use only. External use of Gartner copyrighted material must be approved in writing by Gartner Vendor Relations. Please e-mail your usage request to quote.requests@gartner.com for approval. Conclusions • An increase in e-commerce-related risk broadens the scope of business continuity planning. • E-commerce drives increased requirements for continuous availability and shorter recovery times and points. • E-commerce is transforming the market for business continuity services. These slides are for internal use only. External use of Gartner copyrighted material must be approved in writing by Gartner Vendor Relations. Please e-mail your usage request to quote.requests@gartner.com for approval. New E-Commerce Risks Performance/Capacity Human Error/ Operations Risk Planned/Unplanned Downtime Outsourced Service Providers Security Incidents Content/Application Links to Third Parties These slides are for internal use only. External use of Gartner copyrighted material must be approved in writing by Gartner Vendor Relations. Please e-mail your usage request to quote.requests@gartner.com for approval. E-Commerce BC: New Rules/New Realities • IT and business process management are integrated — no longer solo views • Production costs increase — no separate budget for BCP • Risk identification and management take on a matrix management focus, e.g., technology, financial, trading, operations • Problems are public — IT and business problem management must be integrated; root cause analysis • Only as strong as your weakest link — good application/bad operations • Contingency plans become critical when automation isn’t there — every component of the business process now must have a plan These slides are for internal use only. External use of Gartner copyrighted material must be approved in writing by Gartner Vendor Relations. Please e-mail your usage request to quote.requests@gartner.com for approval. BC Components Disaster Recovery Objective Mission-critical applications Business Recovery Mission-critical business processing (workspace) Site outage (external) Business recovery plan Electrical outage in the building Recovery site in a different power grid Business Resumption Business process workarounds Contingency Planning External event Focus Deliverable Site or component outage (external) Disaster recovery plan Fire at the data center; critical server failure Recovery site in a different location Application outage (internal) Alternate processing plan Credit authorization system down Manual procedure External behavior forcing change to internal Business contingency plan Main supplier cannot ship due to its own problem 25% backup of vital products; backup supplier Sample Event(s) Sample Solution Crisis Management These slides are for internal use only. External use of Gartner copyrighted material must be approved in writing by Gartner Vendor Relations. Please e-mail your usage request to quote.requests@gartner.com for approval. Creating Business Continuity Plans PROCESS Change Management Education Testing Group Plans and Procedures Risk Reduction Implement Standby Facilities Project Testing Review Ongoing Process Create Planning Organization Recovery Strategy Risk Analysis Business Impact Analysis Policy Organization Resources Scope Business Continuity Planning Initiation These slides are for internal use only. External use of Gartner copyrighted material must be approved in writing by Gartner Vendor Relations. Please e-mail your usage request to quote.requests@gartner.com for approval. E-Commerce BC — Integrated Processes E-Bus. Recovery Team E-Bus. Project Manager Business Manager Rules and tools Risk Manager Risk Management (Financial, Technology, Operations) OSPs/ Business Partners Architecture and Standards Application and Tech. Design Business Continuity Mgr. Audit IT Information Security Business Operations Legal/Compliance HR/Public Relations Business Process Owner Business continuity strategy/design Business Continuity Recovery/continuity strategy/design Security incident identification/response design Operations Architecture and Design IT Recovery management Information Security IT Operations Problem, Change, Performance, DR Audit — Financial and EDP These slides are for internal use only. External use of Gartner copyrighted material must be approved in writing by Gartner Vendor Relations. Please e-mail your usage request to quote.requests@gartner.com for approval. Too Much Testing and Reporting Is Never Enough Gartner Estimates There Is No Such Thing as a Failed Test 25% Plan Tested (+ One Year) Plan Tested (One Year) 25% Management Reporting is Critical 50% BCP Phase Plan Tested (< One Year) Order Prod. Eng. Fulfillment Location, Business Process or Department Acct. Payable Acct. Rec. Cash Mgt. R&D Impact Analysis Risk Analysis Strategy Resources Committed Last Tested Change Mgmt. Last Major Review Workable Solution Audit These slides are for internal use only. External use of Gartner copyrighted material must be approved in writing by Gartner Vendor Relations. Please e-mail your usage request to quote.requests@gartner.com for approval. What Is Your Cost of Downtime? Productivity • Number of employees impacted X hours out X burdened hourly rate Damaged Reputation • Customers • Suppliers • Financial markets • Banks • Business partners • ... Revenue • Direct loss • Compensatory payments • Lost future revenue • Billing losses • Investment losses Financial Performance • Revenue recognition • Cash flow • Lost discounts (A/P) • Payment guarantees • Credit rating • Stock price Other Expenses Temporary employees, equipment rental, overtime costs, extra shipping costs, travel expenses ... These slides are for internal use only. External use of Gartner copyrighted material must be approved in writing by Gartner Vendor Relations. Please e-mail your usage request to quote.requests@gartner.com for approval. Know your downtime costs per-hour, -day, -two days ... Applying High Availability to Disaster Recovery Assumes mirroring or shadowing plus Hot Standby or a complete application environment Load-Balanced Database and/or file and/or object replication Mirroring Log/journal transfer (continuous or periodic) net $$$+ Shadowing host $$$+ Cost Database and/or file and/or object backup disk $$$$+ Electronic appl. $+ Elec. Journaling Standard Recovery Vaulting net $ tape $ net $ host $ disk $ tape $ net $-$$+ net $$$+ host $$+ host $$+ disk $$$$+ disk $$$$+ 72 48 24 12 hrs. hours hours hours Disaster Recovery Times These slides are for internal use only. External use of Gartner copyrighted material must be approved in writing by Gartner Vendor Relations. Please e-mail your usage request to quote.requests@gartner.com for approval. Minutes Designing E-Commerce Applications for No Single-Point-of-Failure Site Load Balancer Geographic Load Balancer Web Server Clusters Site Load Balancer Application Server Clusters Transaction Replication Database Clusters Database Replication Database Clusters Standby or Active These slides are for internal use only. External use of Gartner copyrighted material must be approved in writing by Gartner Vendor Relations. Please e-mail your usage request to quote.requests@gartner.com for approval. Data Replication for Continuous Availability Database Clusters Host-based Disk-based Database Clusters Replication Methods Disk-to-Disk mirroring Log-based DBMS replication Examples EMC SRDF, Compaq DRM, IBM PPRC and XRC, HDS HARC and HRC Quest Shareplex, Oracle Standby Database, ENET RRDF, SQL Server 2000 Server-based block or file replication Application-based replication Legato Octopus, NSI Doubletake, Veritas SRVM Typically implemented with messagequeuing middleware These slides are for internal use only. External use of Gartner copyrighted material must be approved in writing by Gartner Vendor Relations. Please e-mail your usage request to quote.requests@gartner.com for approval. Emerging Technologies/Services • Capacity on demand/emergency backup • Wide-area clusters – HP Continental Clusters – IBM Geographically Dispersed Parallel Sysplex • Cascading data replication Host High Bandwidth (fiber) Disks Operational Site Disks Metropolitan/Regional Recovery Facility Disks Primary Recovery Site Host Tape Backup/Archival Host These slides are for internal use only. External use of Gartner copyrighted material must be approved in writing by Gartner Vendor Relations. Please e-mail your usage request to quote.requests@gartner.com for approval. Disaster Recovery: Market Dynamics Load-Balanced (2+Sites) HighAvailabilityBased Service Warm Site and Mobile Recovery Quick Ship 2000 Warm Site and Mobile Recovery Quick Ship 2004 These slides are for internal use only. External use of Gartner copyrighted material must be approved in writing by Gartner Vendor Relations. Please e-mail your usage request to quote.requests@gartner.com for approval. Resource Internally or Externally Internal •You have an alternative facility (50 km distant) •BC vendors have insufficient capacity •BC is a recognized and respected discipline •You cannot economically benefit from syndication External (shared) External (dedicated) •You do not have an alternate facility •You desire multisite continuous availability or hot standby support •RTOs/RPOs are very short • You want to focus on core competencies • Getting management sign-off for dedicated capital is difficult • Experience of supporting an invocation is important • Your planning scenarios include loss of technical staff These slides are for internal use only. External use of Gartner copyrighted material must be approved in writing by Gartner Vendor Relations. Please e-mail your usage request to quote.requests@gartner.com for approval. North American Business Continuity Market Full-Service Providers • Comdisco Recovery Services and Web Availability Services • IBM Business Continuity Recovery Services and Outsourcing Services • SunGard Recovery Services and E-Sourcing Business Continuity and Internet Services • • • • • • Professional services Planning software Hot/warm/cold standby Mobile/static facilities Mainframe/midrange/desktop Quick ship • • • • Peripherals Networks Work area Specialized ancillary services such as check processing and data recovery What’s new: Full-service Web-hosting with BC ―designed in,‖ multisite infrastructures for continuous availability, Web site and network ―throttling‖ for performance These slides are for internal use only. External use of Gartner copyrighted material must be approved in writing by Gartner Vendor Relations. Please e-mail your usage request to quote.requests@gartner.com for approval. Negotiating a Favorable BC Contract — Balance Risk With Economies of Scale Cost Always use competitive tendering, even at renewal Keep contracts to three years Unbundle contract costs Understand upgrade costs Specify test time and additional fees Specify occupancy/comm. fees Declaration fees are negotiable For unsyndicated equipment, check cost of self-acquisition Annual cap fees Contract Terms Include early-termination conditions Agree to a buy-out schedule Miscellaneous Understand the right of access: ―first come, first served‖ or shared Check syndication levels, risk exposures and exclusion zones Touch the equipment. Visit the recovery center These slides are for internal use only. External use of Gartner copyrighted material must be approved in writing by Gartner Vendor Relations. Please e-mail your usage request to quote.requests@gartner.com for approval. Recommendations • Applications architecture/design must include new continuous-availability requirements, including planning for physical site outages, regardless of cause. • An end-to-end analysis of the information flow through internal and external processing environments is required to successfully provide for recovery options for all potential scenarios. • Integrate business continuity planning into the enterprise project life cycle to ensure that recovery needs are identified in the initial phases of new projects, including ―project creep‖ and major upgrades. • Institute an information classification process to ensure that information critical to the enterprise is not disclosed during the problem management life cycle. • Testing e-business recovery plans requires an integrated effort of all parties involved with the business transaction. The participation of all outside service providers is critical to the success of the recovery process. When it is not possible to conduct a live test of a BC plan, or a component plan, conduct a tabletop testing to ensure that external dependencies are addressed. The contracts with outside service providers must address BC needs. • Enterprises should plan for multiple sites upfront when designing the applications architecture to increase the flexibility of options and better meet multisite needs. • Evaluate Web site and all integrated content/application availability and recovery strategies to ensure that they meet business requirements. • For RTOs under 24 hours, evaluate transaction and data replication techniques. • Select a service provider with local presence and resources. • When considering e-business service providers, evaluate their BC/DR experience. • For high-end Web sites, evaluate Comdisco, IBM and SunGard for hosting and continuity services. These slides are for internal use only. External use of Gartner copyrighted material must be approved in writing by Gartner Vendor Relations. Please e-mail your usage request to quote.requests@gartner.com for approval.