3. Agenda
• Introduction
• Disaster Recovery Approach
• DR Team Organization
• Case Study
• Example Disaster Recovery Services
• Open discussion
4. Introduction
What is a Disaster?
Any unplanned event that requires immediate
redeployment of limited resources
Sample Disasters
Natural Forces Technical Failure Human Interference
Fire Power Outage Criminal Act
Environmental Equipment Failure Human Error
Hazards Network Failure Loss of Users
Flood / Water Software Failure Explosions
Damage
Extreme Weather
5. Introduction
Some Examples of Disasters
The Chicago Flood
The underground flood of Chicago on Monday April 13, 1992 proved to be one of
the worst business disasters ever. 230 buildings lost power because water
threatened their underground power sources.
The World Trade Center Explosion
Businesses were forced to evacuate the World Trade Center in February 26, 1993.
When a bomb exploded in the underground parking garage. Companies that
were effected by the disruption were unable to remove critical equipment and
documents.
The San Francisco Earthquake
The Oct 18, 1989 quake measured 7.0 on the Richter Scale. The Bay bridge had
collapsed. The city had lost the main business section due to the collapse of
buildings and electricity.
6. Introduction
Some Examples of Disasters (Cont’d)
Hurricane Andrew
August 22, 1992, Hurricane Andrew hit the South Florida area. Many businesses
suffered physical and financial losses from the hurricane, the valuation of
destroyed property was the largest in US history.
The Kobe Quake
The devastation on January 17, 1995 was the worst in the port city of Kobe where
the 7.2 magnitude quake toppled roadways, wrecked docks, severed
communication lines and kept the city in flames into the next day.
Oklahoma City Bombing
On April 19, 1995, a terrorist bomb exploded in front of the nine-story Alfred P.
Murrah Federal Building in downtown Oklahoma City. The blast destroyed one-
third of the building from roof to ground, leaving a crater eight feet deep, and 30
feet wide.
7. Introduction
What is a Disaster Recovery Plan?
A management document for how and when to utilize
resources needed to maintain selected functions when
disrupted by agreed upon incidents
Other names commonly used:
Business Continuity Plan
Contingency Plans
Continuity Plans
Emergency Response Plans
Business Recovery Plans
Recovery Plans
8. Introduction
When an incident occurs, the Disaster Recovery response
activities are likely to be the following (at a high level).
Incident Assess Confirm Transfer to Execute
Damage Response Alternate Required
Strategy Location Functions
Prepare Transfer &
New Site Execute at
New Site
Restore Transfer &
Primary Execute at
Site Primary Site
Return to
Normal Operations Generate Assess DRP
Change Effectiveness
Requests
9. Introduction
What is the magnitude of an incident?
Regional Area
Local Area
Within Blocks
To The Building
Within Floors
On The Floor
Within The Room
Depending upon the magnitude of an incident, possible
alternative sites include:
Within The Room
Within the Building
Within the Region
Outside the Region
10. Introduction
Types of Controls
Integrity Controls Confidentiality Controls Availability Controls
Policy Proprietary Information Asset Identification
Methodology Policy Interruption Analysis
Staffing Ethics Statement Controls Review
Education “Need to Know”, “Need to Impact Analysis
Division of Withhold” Data Backup
Responsibility Classification Scheme Off-site Storage
Audit Records Management Avoidance Strategies
Error and Change Handling Procedures Mitigation Strategies
Control Physical & Electronic Early Detection &
Reporting and Security Measures Notification
Resolution Recovery Strategies
Test Alternate Locations
Quality Assurance Plans and Procedures
Vendor Relationships
Training
Testing
11. Introduction
Types of Strategies
Avoidance Strategy Mitigation Strategy Recovery Strategy
Redundant Early warning detection High level recovery plan
configuration to avoid Contractual agreements Off-site data storage
incidents with vendors Very responsive vendor
Site harden facilities to Mirrored data and relationships
resist incidents documents Very knowledgeable
Redundant utilities Detailed migration employees
and hardware recovery plan
Automated operation
recovery plan Types of Strategy Options
Hot site
Cold site
Self Backup
Service Bureau
Reciprocal Agreement
12. Introduction
What is a Critical Business Function?
A specific entity management has decided is so significant to the
business mission, that without it, the organization cannot successfully
operate after an identified time period.
Types of Impact
Financial Loss Extra Expense
Lost Revenue Labor Cost
Lost Sales — Recreate Lost
Lost Market Share Business
Lost Opportunity — Recreate Lost Data
— Use Manual Process
Human Interference Equipment Cost
Management Control — Hardware /
Employee Relations software
Stockholder Relations — Telephones
Public Image Money Cost
Legal Exposure — Delayed Receivable
Contractual Liability — Delayed Orders
Competitive Advantage — New Interest
— New Investments
13. Introduction
Criteria for a Critical Business Function
Timing Requirements
Cost of Control vs. Impact Minutes
Hours
Cost of Days
Impact $ Weeks
Cost of Quarters
Control $ Special Situations
Impact
Interdependencies
Inputs and Outputs
Cost
14. Introduction
Implementing Recovery Plans is not an easy task!
Recovery prevention techniques are inadequate
Increase the level of user security awareness and education
No recovery plan at all
Plan is stored on the “ultimate” computer (in IT directors’ head)
Establish short-term alternate processing procedures
Removal of systems running on obsolete machines
Recovery plans are too theoretical and not geared to the organization’s
needs
Plans are unwieldy
Recovery plans are in a written format and/or are not updated
Backup not tested
Plans not tested
Plans are located in the computer room or the building
Plans are too grandiose (EXPENSIVE)
Plan does not address PCs / workstations
“People Factors” are not taken into account
15. Disaster Recovery Approach
The following Life Cycle model is useful when
thinking about Disaster Recovery.
Planning Activities
Normal
Operations
Changes
Maintenance
Activities
Changes
Up-to-Date
from
tests DRP
Changes
from Recovery Activities
event
16. Disaster Recovery Approach
Planning Implementation
Scoping & Recovery Disaster Training
Risk Strategy Recovery & Approval
Assessment Development Plan Testing
Planning
The primary objective for the Planning Phase is to gain management
consensus on the focus areas and scope of a Disaster Recovery Plan that
will address major business risks
Implementation
The primary objective for the Implementation Phase is to develop, test,
and rollout a Disaster Recovery plan. The implementation phase could be
longer or shorter, depending upon scope, approach, and staffing defined
during the Scoping and Risk Assessment phase
17. Disaster Recovery Approach
Determine the focus areas and scope Scoping &
Risk
Assessment
Recovery
Strategy
Disaster
Recovery
Development Plan
Training
&
Testing
Approval
for the Disaster Recovery Plan
implementation phase
Activities Key Deliverables
• Management Briefing • Scoping and Risk Assessment
• Questionnaires Report
• Interviews • Requirements Summary
• Focus Groups • Current Capability Summary
• Workshop • Critical Business Functions
Matrix
• Critical Systems Matrix
18. Disaster Recovery Approach
Develop strategies for each of the Scoping &
Risk
Assessment
Recovery
Strategy
Disaster
Recovery
Development Plan
Training
&
Testing
Approval
most critical systems based upon the
outcome of the Scoping and Risk
Assessment phase
Activities Key Deliverables
• Develop Strategies • The Recovery Strategy Report
• Select Spinoff Projects • Alternatives and
recommendations
19. Disaster Recovery Approach
Develop detailed plans for business Scoping &
Risk
Assessment
Recovery
Strategy
Disaster
Recovery
Development Plan
Training
&
Testing
Approval
continuity based upon the specific
strategy identified for each critical
system
Activities Key Deliverable
• Develop Recovery Plan • Recovery plan includes
• Assessment Plan & Procedures
• Notification Procedure
• Recovery center Procedure
• Migration Plan (facilities, data,
people)
• Team Organization ( Roles &
Responsibilities)
20. Disaster Recovery Approach
Develop detailed plans for business Scoping &
Risk
Assessment
Recovery
Strategy
Disaster
Recovery
Development Plan
Training
&
Testing
Approval
continuity based upon the specific
strategy identified for each critical
system (continue)
Activities Key Deliverable
• Develop Maintenance • Maintenance Procedures include
Procedures
• Responsibility matrix for
maintenance
• Testing strategy
• How to update the Recovery
Procedure
• Ongoing Center recovery training
schedule
• Prepare facilities and • Recovery Center Location, facilities
Infrastructure and required component
21. Disaster Recovery Approach
Provide training to the recovery team Scoping &
Risk
Assessment
Recovery
Strategy
Disaster
Recovery
Development Plan
Training
&
Testing
Approval
and conduct the testing based upon the
testing approach documented in the
Maintenance procedure
Activities Key Deliverables
• Prepare training materials • Training material
• Conduct & Evaluate • Trained staff
Training
22. Disaster Recovery Approach
Get the Disaster Recovery Plan Scoping &
Risk
Assessment
Recovery
Strategy
Disaster
Recovery
Development Plan
Training
&
Testing
Approval
approved and rollout to the
organization
Activities Key Deliverable
• Revise plan (if necessary) • Management Sign-off
• Approve the Disaster • Publication & Distribution of the
Recovery Plan disaster recovery
23. DR Team Organization
An Example of Disaster Recovery Team
DRP Management
Team
Disaster Recovery
Director
Customer Production Disaster
Administrative
Application Recovery Site Restoration
Support Liaison
Support Coordinator
System Software
and Database Security
Administration
Computer
Network
Operation and
Delivery
Off-site Storage
Application Services
Support Delivery
24. DR Team Organization
Examples of Data Center Roles & Responsibilities
Title Roles Responsibilities
DR management Act as the steering committee • Provide overall management support to DR
Team of the DR Team team
• Responsible for strategic decision and key
requirements or changes on DRP
• Make key decisions according to DRP
Disaster Recovery Act as an advisor to the • Oversee the activities of the DR team
Director DR management team. • Budget for future DR requirements
• Communicate with other management to
deal with the business process and recovery
procedures
Administrative Provide administration • Provide the DR team with administrative
Support support to the DR team resources and facilities
• Co-ordinate with lawyers for court cases and
handle legal documents
• Responsible for accounting matters on DR’s
expenses
• Investigate the amount of damaged resources
and insurance claims
25. DR Team Organization
Examples of Data Center Roles & Responsibilities
Title Roles Responsibilities
Customer Liaison Coordinate and coordinate •Notify users and clients of the disaster
with users and customers •Issue updates of recovery progress and
on any recovery issue expected time of recovery
•Help on data center migration issues and
work re-allocation
Disaster Recovery Centralized coordination •Declare a disaster for each critical system
Coordinator for the entire DR team component or for an entire site
•Inform the DR team of the decision
•Execute DR procedures and recovery
strategies
•Ensure that the DRP is updated and test on a
regular basis
Site Restoration Co-ordinate the recovery •Organize security control for the disaster site
operations should a site be and alternate processing site as required
destroyed
System Software Prepare recovery and •Responsible for the restoration of Hosts,
and Database restoration of software Servers, DB, synchronize data, etc.
Administration and databases
26. DR Team Organization
Examples of Data Center Roles & Responsibilities
Title Roles Responsibilities
Computer Manage storage of the •Provide ready access to the required backups
Operations and off backups •Ensure the backups are stored in a secure
site storage environment
Application Manage applications with •Manage application changes to ensure they
Support regard to DRP are compliant with the DRP and vice versa
Security Review and monitor DR •Ensure the DR procedures comply with the
procedures firm security and audit policies
Network Delivery Manage and monitor •Oversee the recovery of the communication
voice and data network environment
•Switch users to use the alternate network
•Co-ordinate with the communication service
providers for WAN service recovery
27. DR Team Organization
Examples of Data Center Roles & Responsibilities
Title Roles Responsibilities
Service Delivery Manage IT service •Oversee the service management recovery
delivery •Provide helpdesk and end-user support as in
DRP
•Work closely with Customer Liaison and
Disaster Recovery Coordinator to ensure
synchronization of communication channel to
the users and the DR team activities.
28. Case Study
The Chicago Flood : Impact
• One of the worst business disasters
• 230 buildings lost power for a couple of days
• Valuable government records were in jeopardy
• Extensive impact on electrical and computing systems
• The greatest financial impact on the CBOT, losing 25 billion in trading of
36 products
29. Case Study
The Chicago Flood : Disaster Recovery
• Using Alternate Site Services approach
• Providing the alternate site nearly identical to the customer’s damaged
site
• Implemented by Comdisco Continuity Service
The Chicago Flood : Recovery Result
• Helped 2 Chicago banks resume operation within hours of evacuation
• 17 customers from the financial, brokerage, government and service/
distribution industries, were supported at their hot sites within half a day
30. Case Study
The World Trade Center Explosion : Impact
• Building-wide power outage
• Structural damaged and employee trauma, Businesses were down
• Water problem due to pipes were severed
• Injured and Dead reports, the building was considered a crime scene
The World Trade Center Explosion : Recovery
• Fiduciary Trust, a banking and financial institute’s Recovery Plan
• The data center switched automatically to their secondary power system
• Moved the operation to their alternate site in NJ which equipped with a
computer network nearly identical to that of the bank
31. Case Study
The World Trade Center Explosion : Recovery Result
• System was down for Friday afternoon and was up and running by
Monday morning as if nothing had happened
• Employees retained their usual telephone numbers
• Transactions went through the same as always
• Customers couldn’t even detect that the bank was no longer operating
from the World Trade Center
32. Example Disaster Recovery
Services
Examples of Disaster Recovery Services
Alternate Sites
Provide alternate site nearly identical to the customer’s damaged site
Business Impact Analysis
Provide services such as defining disaster plans and addressing
exposures to business and recovery administrators
Certification
Provide services such as certifying qualified individuals in the discipline
and promoting the credibility and professionalism of certified
individuals
33. Example Disaster Recovery
Services
Examples of Disaster Recovery Services
Education Classes
Creating a base of common knowledge for the business
continuity/disaster recovery planning industry through education,
assistance, and the promotion of international standards
On-Site Recovery Facilities
Manage the mobilization of an on-call response team, prepare pre-
designated site, erect temporary pre-engineered structures, install
mechanical and electrical systems and coordinate move-in activities
Satellite Communication
Provide satellite telecommunications products and services
34. Example Disaster Recovery
Services
Service Providers : Consulting Services
Andersen Consulting
www.ac.com
Bell Atlantic Federal CommGuard
www.commguard.com
Comdisco
www.comdisco.com
Computer Security Consultants, Inc.
www.crciweb.com
GSA Disaster and Business Recovery
www.gsa-gsa.com
Intessera Technologies Group
www. intessera.com
35. Example Disaster Recovery
Services
Service Providers : Alternate Site Services
ARC Disaster Recovery Services
www.arcdrs.com
Comdisco
www.comdisco.com
HP Business Recovery Services
www.hp.com
IBM Business Recovery Services
www.brs.ibm.com
SunGard Recovery Services, Inc. recovery.sungard.com
36. Example Disaster Recovery
Services
Providers : Computer Quick-ship , Hardware
Replacement
El Camino
www.elcamino.com