Backup, Geo-Redundancy & Disaster Recovery

Business continuity with backup strategies, geo-redundancy, and disaster recovery for data engineering

DR Architecture

Architecture Diagram

┌─────────────────────────────────────────────────────────────────────┐
│                    DISASTER RECOVERY ARCHITECTURE                    │
├─────────────────────────────────────────────────────────────────────┤
│                                                                     │
│  PRIMARY REGION: EAST US 2       SECONDARY REGION: WEST US 2       │
│  ┌──────────────────────┐        ┌──────────────────────┐         │
│  │ ┌──────────────────┐ │        │ ┌──────────────────┐ │         │
│  │ │ ADLS Gen2        │ │ Geo-Rep│ │ ADLS Gen2        │ │         │
│  │ │ (RA-GRS)         │─┼───────>│ │ (Secondary)      │ │         │
│  │ └──────────────────┘ │        │ └──────────────────┘ │         │
│  │                      │        │                      │         │
│  │ ┌──────────────────┐ │        │ ┌──────────────────┐ │         │
│  │ │ Synapse Pool     │ │ Geo-Rep│ │ Synapse Pool     │ │         │
│  │ │ (Active)         │─┼───────>│ │ (Standby)        │ │         │
│  │ └──────────────────┘ │        │ └──────────────────┘ │         │
│  │                      │        │                      │         │
│  │ ┌──────────────────┐ │        │ ┌──────────────────┐ │         │
│  │ │ Cosmos DB        │ │ Multi- │ │ Cosmos DB        │ │         │
│  │ │ (Multi-region    │─┼───────>│ │ (Replica)        │ │         │
│  │ │  writes)         │ │ region │ │                  │ │         │
│  │ └──────────────────┘ │        │ └──────────────────┘ │         │
│  └──────────────────────┘        └──────────────────────┘         │
│                                                                     │
│  RPO/RTO TARGETS:                                                   │
│  ┌─────────────────────────────────────────────────────────────┐   │
│  │ Service          │ RPO          │ RTO           │ SLA       │   │
│  │ ─────────────────────────────────────────────────────────── │   │
│  │ ADLS (RA-GRS)    │ <15 min      │ <30 min       │ 99.99%   │   │
│  │ Synapse (Geo)    │ <1 hour      │ <4 hours      │ 99.9%    │   │
│  │ Cosmos DB        │ 0 (multi-    │ 0 (automatic  │ 99.999%  │   │
│  │                  │  region)     │  failover)    │          │   │
│  │ Event Hubs       │ 0 (capture)  │ Minutes       │ 99.95%   │   │
│  │ Databricks       │ Varies       │ Minutes-Hours │ 99.9%    │   │
│  └─────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────┘

Backup Configuration

# Point-in-time restore for Synapse
# Azure CLI command
# az synapse sql pool restore \
#   --name SQLPool01 \
#   --workspace-name syn-prod-workspace \
#   --resource-group rg-dataengineering-prod \
#   --restore-point "$(date -d '2 hours ago' +%Y-%m-%dT%H:%M:%S)"

# Cosmos DB continuous backup
import requests
token = credential.get_token("https://management.azure.com/.default")

# Enable continuous backup
response = requests.patch(
    f"https://management.azure.com/subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.DocumentDB/databaseAccounts/{account}",
    headers={"Authorization": f"Bearer {token.token}", "Content-Type": "application/json"},
    json={
        "properties": {
            "backupPolicy": {
                "type": "Continuous",
                "continuousModeProperties": {
                    "tier": "Continuous7Days"
                }
            }
        }
    }
)

Geo-Redundancy Configuration

resource storageAccount 'Microsoft.Storage/storageAccounts@2023-01-01' = {
  name: 'stdatalake001'
  location: location
  sku: {
    name: 'Standard_RAGRS'  // Read-Access Geo-Redundant
  }
  kind: 'StorageV2'
  properties: {
    isHnsEnabled: true
    replication: {
      geoReplication: {
        enabled: true
        destinationAccountName: 'stdatalake001-geo'
        destinationRegion: 'westus2'
      }
    }
  }
}

ℹ️

Pro Tip: Use RA-GRS for ADLS Gen2 to provide read access during regional outages. Use Cosmos DB multi-region writes for automatic failover with zero data loss.

Interview Questions

Q1: Explain the difference between RPO and RTO. A: RPO (Recovery Point Objective) is the maximum acceptable data loss (e.g., 15 minutes). RTO (Recovery Time Objective) is the maximum acceptable downtime (e.g., 1 hour). Both drive DR strategy design.

Q2: How do you test disaster recovery in Azure? A: 1) Simulate regional outage, 2) Initiate failover to secondary region, 3) Verify data integrity, 4) Test application functionality, 5) Measure actual RPO/RTO vs targets, 6) Document findings and improve.

Q3: What is the cost impact of geo-redundancy? A: Geo-redundancy doubles storage costs (primary + secondary). However, the cost of downtime (lost revenue, reputation) often far exceeds the additional storage cost. Use RA-GRS for critical data.