FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
www.aesrm.org
Incident Management An Exchange of Practices and Experiences
2008 Annual Meeting – Sonoma California
June 18, 2008 8:15 to 10:15
Andrew McCruden, Citigroup Randall Till, MasterCard Worldwide
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
Agenda
I. II. III. IV. V. VI.
Opening comments Methodologies for managing incidents Building and managing external relationships Conducting effective exercises of incident mgmt plans Communication strategies and case studies/experiences Wrap-up
2
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
www.aesrm.org
Methodologies for Managing Incidents
Incident Command Systems (ICS)
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
Why Incident Command System (ICS)?
• • • • •
Global events (e.g. Pandemic) Promote emergency management plan Management awareness Reputation and shareholder value US Presidential Directive (PD #5) - mandatory for:
– – –
A US federal agencies for federal funding US State governments All hazardous material incidents
–
US law enforcement, government, and the military
4
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
What is Incident Command System (ICS)?
ICS is a well organized team approach for managing critical incidents
• •
It is modular and scalable – only use teams that you need Provides for consistent and reliable communications using common terminology Ensures coordinated response among teams and locations (horizontal & vertical)
–
•
Especially helpful for companies with multiple locations
•
Employs standard and proven practices
Source: Emergency Management & Safety Solutions
5
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
ICS Organizational Structure
Command (manages)
Operations (does)
Planning & Intelligence (plans)
Logistics (care/gets)
Financial (pays/records)
6
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
ICS Team Types
IAT = Initial Assessment Team
•
Team for Small Regional Offices and sub-team of the CIRT/LIRT
LIRT = Local Incident Response Team
•
Regional Headquarters and Select Offices
CIRT = Corporate Incident Response Team
•
Corporate Headquarters ONLY
7
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
IAT Members
Initial Assessment Team (IAT) City name
Commander Alternate
Operations
Group Lead Alternate
Planning & Intelligence
Group Lead Alternate
Logistics
Group Lead Alternate
Financial
Group Lead Alternate
Facilities/ Real Estate
Primary Alternate
Security
Primary Alternate
Technology/ Operations
Primary Alternate
Last Revised
8
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
CIRT/LIRT Members
Business Continuity
Corporate Incident Response Team (CIRT)
Operations
Facilities/ Real Estate
Planning & Intelligence
Key Lines of Business Legal/ Regulatory Investor Relations Communications
Logistics
HR Business Partner Benefits
Financial
Global Finance
Security
Insurance
Technology/ Operations
Travel
Payroll
Meetings
Information Security
Accounting/ Accounts Payable
Technical Recovery Business Recovery
Purchasing
= Initial Assessment Team
9
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
ICS Structure (example)
Country IAT Country IAT
Corporate HQ CIRT
Country IAT Country IAT Country IAT Country IAT
Country IAT
Country IAT
Country IAT
Asia Pacific LIRT
Country IAT
Middle East & Africa LIRT
Country IAT
Country IAT
Country IAT
Country IAT Country IAT
Country IAT
Country IAT
Country IAT
Country IAT
Country IAT
10
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
EVENT
ICS Escalation Flow
Notify GSCC
Global Security Control Center (GSCC) Process
First Response process
Normal operating procedures ?
No
Yes
STOP
IAT only - Regional Offices
IAT activated
Assess (use Initial Assessment Form)
LIRT – Regional HQs & Select Offices
Security (if any), Incident Commander & Business Continuity discusses IAT Activation?
CIRT – Corporate Headquarters
Security, Incident Commander & Business Continuity discusses IAT Activation and Cross Office Notification?
No
Monitoring Continues
No
Cross Office Notification Process
Yes
Incident Commander: IC notifies Regional LIRT Incident Commander
IAT activated
Assess (use Initial Assessment Form)
Yes
IAT activated Assess (use Initial Assessment Form) Cross Office Notification Process Cross Office Notification Process
Activate LIRT? (appropriate components)
No
IAT continues monitoring Activate CIRT? (appropriate components)
No
IAT continues monitoring
Yes
LIRT activated
Conduct Action Planning Process IC notifies local Executive Management IC notifies CIRT Incident Commander
Yes
CIRT activated
•Conduct Action Planning Process •IC notifies the Policy Committee
11
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
www.aesrm.org
External Relationships
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
Building and Managing External Relationships – Taking Incident Management “Beyond Your Four Walls”
The major events of this decade support the premise that an organization’s incident management planning should be externally as well as internally focused. Pre-Event Coordination Strategies with:
Financial Services Firms and Industry Associations
Key Suppliers
Public Sector – Governmental and Non Governmental Organizations Regulators
Discuss as a group what’s working, where more attention is needed, and what’s being done to close the gaps.
13
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
www.aesrm.org
Conducting Effective Exercises of Emergency Management Plans
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
Integration of ICS with existing Business Continuity Program
•
Business Recovery and Technical Recovery activation
–
Planning & Intelligence on CIRT/LIRT
–
Problem Resolution Team (PRT) process
•
Business Recovery Plan
–
Activation Flow
• •
Pandemic planning scenario Business Continuity Manuals
ICS is an integral part of the Business Continuity Program
15
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
ICS Process
•
Event occurs beyond normal operations
•
Initial Assessment Team (IAT) meets to determine impacts, incident level, and necessity of LIRT activation
LIRT activated -- Incident Commander (IC) and Group Leaders hold action planning meeting to determine objectives and operational period (OP) Group Leaders share objectives on Action Plan and functional areas begin work LIRT members of the functional areas complete Action Plan Objectives and provide status to Group Leader Incident Commander and Group Leaders meet to share status and if needed determine new objectives and new operational period
•
•
•
•
16
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
IAT Assessment
• •
Assess impacts of the incident Determine incident level
Incident Levels: Level 1: Compartmentalized or Minor • An emergency that is limited in scope Level 2: Local or Minimum • An emergency that is moderate to severe in scope Level 3: Regional or Major • A catastrophic disaster that has severely damaged a mission critical facility requiring relocation of staff and business processes and/or severe disruption of services at that facility
•
Based on incident level, take appropriate action
Offices with IAT only, continue to address the event
•
17
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
CIRT/LIRT Process
• •
Decision is made to activate virtually or physically An action planning meeting by the Incident Commander (IC) and the Group Leaders is held as soon as the decision is made to activate the CIRT/LIRT The IC coordinates the Action Plan to share with CIRT/LIRT members CIRT/LIRT members take steps to complete Action Plan Objectives Report status updates to the Group Leader If needed, Action Planning begins again
• • • •
18
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
Steps to Complete
•
Incident Commander and Group Leaders conduct an Action Planning Meeting
– – –
Determine strategic objectives
Assign objectives to Groups Set Operational Period (OP)
•
– – –
LIRT group members receives objectives and begin taking action
Work across all Groups if necessary Record findings Update Group Leader
19
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
Emergency Management Planning Deliverables
Deliverables Core Offices CIRT/LIRTs
C1, C2, C3
LIRT’s
K1, K2, K3, K4
IAT’s
Remaining offices
Due Date
CIRT/LIRT Notification Test (conducted by BC)
2
Same as Exercise C1 = Mar. & Sep. C2 = Mar. & Sep. C3 = Mar. & Oct. C1 = Aug. C2 = Aug. C3 = Apr. C1 = Nov. C2 = Nov. C3 = May 1 Same as Exercise K1 = May K2 = Mar. K3 = Jul. K4 = Jul. K1 = Jun. K2 = May K3 = Oct. K4 = Oct. 1 1 1 29-Aug. Dates through out year 29-Aug.
IAT Training (conducted by BC)
2
CIRT/LIRT Functional Group Training (conducted by BC)
1
CIRT/LIRT Scenario Based Exercise (conducted by BC)
1
LIRT Notification Test (conducted by BC)
IAT Training (conducted by BC)
1
LIRT Scenario Based Exercise (conducted by BC)
1
IAT Notification Test (conducted by BC) IAT Training (conducted by BC) IAT Self Exercise (conducted by your team)
20
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
IAT Notification Tests and Self Exercise
Notification Test
• •
Test SMS message on work mobile phones and devices Execute Emergency Notification Tool sending a voice message to Work Phone and Mobile, text message to Work Email, and SMS to Mobile.
–
Respond to each message as requested.
Self Exercise
•
Conduct an IAT emergency table top exercise led by Incident Commander
Use the IAT Self Exercise Guidelines and ICS forms and tools Complete BC survey to validate successful completion
21
• •
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
Comprehensive ICS Exercise
Objectives
•
Practice the use of ICS processes under simulated emergency conditions and identify any processes or policies that need improvement Practice the LIRT’s ability to coordinate their response and decision making under simulated emergency conditions Provide a learning environment to allow LIRT members to increase proficiency in executing their roles and responsibilities
•
•
22
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
Exercise Structure
• •
Exercise conducted in a physical command center. Business Continuity staff will facilitate and provide assistance with ICS processes when needed. A simulation (sim) team will act as the “outside world” for this exercise. All issues requiring the outside world must be solved by contacting the simulation team. Such as; gathering information, order equipment, etc. Distribute messages with questions and concerns throughout the exercise from numerous entities (internal employees, media, etc…).
•
•
23
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
ICS Team Member Commitment and Empowerment
Effective emergency response is dependent on qualified staff being trained to execute with proper authority
ICS team members must be:
•
Trained to clearly understand their roles and responsibilities
Committed to fulfilling their responsibility Engaged by participating in meetings and exercises Empowered to perform their roles in accordance with practiced guidelines
• •
•
24
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
www.aesrm.org
Communications and Case Studies
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
Communications – Strategies Before, During and After
Focus Areas for Incident Communications:
Awareness (before)
Response (during and after)
What are Some Practical Challenges We Face?
What are the benefits and limitations of various communication tools and media?
How to manage multiple threads of internal and external communications, many of which are spontaneous during an incident? How do you (or should you) look to establish a “sole source of truth?” How should plans factor in the unavailability of various media during an incident?
26
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
Case Studies
Communication and Coordination Strategies – Putting It All Together:
9/11 Atlantic Storms of 2005: Katrina, Rita, Wilma
London Underground Bombings
What Experiences Can We Apply to the Incident Management Challenges Likely to Occur with Events of Uncertain or Lengthy Duration (e.g., Pandemic)?
27
FINANCIAL SERVICES TECHNOLOGY CONSORTIUM
www.aesrm.org