Backup and Disaster Recovery

Technical recovery design covering backup types, the 3-2-1 rule, immutable copies, RPO/RTO/MTD, recovery sites, failover, DR testing and ransomware resilience.

A backup is the last line of defence: when prevention and detection have failed, the ability to restore is what decides whether an incident is an inconvenience or a company-ending event. This page is the technical recovery complement to Business Continuity, which covers the BCP/contingency process (priorities, roles, planning flow). Here we cover how data and systems are actually copied, stored, restored and tested.

Backup protects availability in the CIA Triad, and also integrity (restoring a known-good state).


Backup Types

TypeWhat is copiedBackup speed / sizeRestore
FullEverything selectedSlowest, largestFastest: one set
IncrementalOnly changes since the last backup of any typeFastest, smallestSlowest: full + every incremental in order
DifferentialChanges since the last fullMedium, grows dailyMedium: full + latest differential
SnapshotPoint-in-time view of a volume/VM (usually copy-on-write)Near instantFast, but often stored on the same system
Continuous (CDP)Every write is journalledConstantRoll back to any moment

text
Sun     Mon     Tue     Wed     Thu
FULL    inc     inc     inc     inc        restore Thu = Full + Mon + Tue + Wed + Thu
FULL    diff    diff    diff    diff       restore Thu = Full + Thu

  • A snapshot is not a backup on its own: if it lives on the same storage array, the same failure or attacker takes both.
  • Database backups need application consistency (quiesce or use native dump/log shipping), otherwise the copy may be corrupt.
  • Choose the strategy from RPO/RTO (below), not habit: frequent incrementals lower data loss; periodic synthetic fulls shorten restore chains.

3-2-1 and Beyond

text
3 copies of the data   (production + 2 backups)
2 different media/storage types
1 copy off-site

Modern extension, 3-2-1-1-0:

  • +1 copy that is offline, air-gapped or immutable
  • 0 errors after restore verification
Copy typeMeaningDefeats
Off-siteDifferent location or regionFire, flood, site loss
Offline / air-gappedDisconnected from the network (tape, rotated disk, logically isolated vault)Network-borne malware, ransomware
Immutable (WORM)Cannot be altered or deleted for a retention period, even by admins (object lock, retention lock)Attacker with stolen admin credentials

Separate the backup administration identity from the normal domain admin identity and require MFA; attackers routinely hunt for backup consoles first (see IAM).


Protecting the Backups Themselves

  • Encrypt backups in transit and at rest; keep keys separate from the backup data and back up the keys too (see Cryptographic Solutions). A lost key means a lost backup.
  • Integrity: hash or sign backup sets and verify them, so tampering or silent corruption is detected.
  • Access control: least privilege, separate credentials, no standing delete rights, restricted network path to the backup server.
  • Classification-aware retention: backups contain the same sensitive data as production, so they inherit its classification, legal retention and secure-destruction obligations (see Data Lifecycle and Data Maintenance).
  • Media handling: label, track, transport securely and sanitise or destroy media at end of life.

RPO, RTO, MTD

TermQuestion it answersSet by
RPO (Recovery Point Objective)How much data can we afford to lose, measured in time?Business impact; drives backup frequency
RTO (Recovery Time Objective)How long can the system be down before recovery must complete?Business impact; drives recovery architecture
MTD / MTPD (Maximum Tolerable Downtime)How long can the business function be unavailable before damage is unacceptable?Business impact analysis
WRT (Work Recovery Time)Time to verify data and return to normal work after the system is backOperations

text
 last good        disaster           system         business fully
 backup                              restored       back to normal
    |<-- RPO -->|     |<----- RTO ----->|<--- WRT --->|
                      |<--------------- MTD ---------->|
Relationship:  RTO + WRT  <=  MTD

Lower RPO/RTO means higher cost: continuous replication and hot standby cost far more than nightly backups and cold rebuilds. Values come from the business impact analysis in the BCP; this page turns them into design.

Worked example

Order system: BIA says losing more than 15 minutes of orders is unacceptable (RPO 15 min), the shop must be back within 1 hour (RTO 1 h), and the business tolerates at most 4 hours of outage (MTD 4 h).

RequirementDesign consequence
RPO 15 minNightly backup is not enough. Use database log shipping/replication every few minutes plus hourly incrementals.
RTO 1 hRestoring a multi-hour full backup from tape is too slow. Keep a warm standby with replicated data and scripted failover.
MTD 4 hLeaves 3 h for WRT: reconciliation of in-flight orders, testing, comms. Check WRT actually fits.
RansomwareAdd an immutable daily copy; accept that this tier restores to a 24 h RPO and a longer RTO, and document it.

Recovery Site Models

ModelStateRTOCost
Hot siteFully equipped, data continuously replicated, ready to runMinutes to hoursHighest
Warm siteHardware and connectivity present, data restored from recent backupsHours to daysMedium
Cold siteSpace, power, network only; everything brought inDays to weeksLowest
Mobile / reciprocalTrailer or agreement with a partnerVariesVaries, with caveats on capacity and trust
Cloud / DRaaSReplicated VMs or infrastructure-as-code in another region, pay for compute only when failing overMinutes to hoursPay-as-you-go

Cloud recovery is attractive because capacity is elastic and standby is cheap, but it needs tested networking, identity and DNS cutover (see Cloud Security). Keep the recovery environment in a separate account/subscription with separate credentials.

Site topologies: active-passive (standby takes over) versus active-active (both serve traffic; best RTO, hardest consistency).


Failover and Failback

  • Failover: switching production to the recovery environment (DNS/load balancer change, promote replica, start standby systems).
  • Failback: returning to the primary once repaired. It is a second migration: data written during the outage must be synchronised back, and it should be planned, not rushed.
  • Automate where safe, but require a human decision for declaring a disaster to avoid flapping or split-brain (two nodes both believing they are primary).

Disaster Recovery Plan Contents

  1. Scope, assumptions and the declared RPO/RTO per system (tiered by criticality)
  2. Activation criteria and who can declare a disaster
  3. Contact lists and escalation (team, vendors, regulators), including out-of-band communication
  4. Inventory of systems, dependencies and recovery order (identity, DNS, network, then databases, then applications)
  5. Step-by-step recovery runbooks, written so someone unfamiliar can follow them
  6. Location of backups, credentials, encryption keys and licences (stored where an outage will not lock you out)
  7. Alternate site details and failover/failback procedures
  8. Validation: how to confirm data and service are healthy before declaring recovery complete
  9. Review and update schedule; change triggers (see Change Management)

Testing

An untested backup is a hope, not a control.

TestWhat happensDisruption
Read-through / checklistOwners review the plan for accuracyNone
TabletopTeam talks through a scenario around a tableNone
WalkthroughTeams step through procedures in detail against real systems' documentationMinimal
SimulationScenario played out with real tools, but no production impactLow
ParallelRecovery systems brought up and run alongside productionLow to medium
Full interruption / failoverProduction really moves to the recovery siteHigh, highest assurance

Record measured restore times and compare them with RTO; fix gaps; repeat at least annually and after major changes.


Ransomware Resilience

Ransomware turns backups into the primary target: attackers delete shadow copies, wipe backup catalogues and encrypt network shares first.

  • Immutable and offline copies out of reach of domain credentials (see 3-2-1-1-0).
  • Keep backup infrastructure off the production domain or tightly segmented.
  • Restore into a clean environment: rebuild from known-good images, rotate all credentials, and confirm the entry point is closed first, otherwise you restore into re-infection. Coordinate with the Incident Response Process (eradication before recovery) and preserve evidence for forensics.
  • Scan restore points for malware and find the last clean point (the infection may predate the latest backups).
  • Pre-define the "restore everything" runbook and measure how long it really takes: a petabyte over a 1 Gbps link is days.

Monitoring and Restore Verification

  • Alert on failed, skipped or missing jobs, sudden changes in backup size (mass encryption or deletion), changes to retention settings, and backup-admin logins. Feed these to the SIEM.
  • Run automated restore tests (restore a sample, compare checksums, boot the VM, run an application smoke test). This is the "0 errors" in 3-2-1-1-0.
  • Track metrics: success rate, restore time vs RTO, recovery point age vs RPO, capacity, and number of untested systems.

Key Points

  • Backups protect availability and are the last control that works after compromise; design them to survive an attacker with admin rights.
  • Follow 3-2-1-1-0: multiple copies, different media, off-site, one immutable/offline, zero restore errors.
  • RPO sets backup frequency, RTO sets recovery architecture, and RTO + WRT must fit within MTD.
  • Backups inherit data classification: encrypt, control access and destroy on schedule.
  • Test restores regularly; only measured recovery times count.