加固备份恢复本地基线(07-26-internal-ops-recovery-baseline)
这个提交包含在:
@@ -516,3 +516,95 @@ echo LOCAL_CANDIDATE_PASS
|
||||
bash scripts/qa/run-local-candidate.sh full
|
||||
# Only this successful branch emits LOCAL_CANDIDATE_PASS.
|
||||
```
|
||||
|
||||
## Scenario: Paired Backup And Shadow-Only Restore
|
||||
|
||||
### 1. Scope / Trigger
|
||||
|
||||
- Trigger: changing or operating database/file backups, restore scripts,
|
||||
retention instructions or recovery-readiness evidence.
|
||||
- Purpose: prevent a database-only recovery point, a mixed attachment restore or
|
||||
an accidental destructive restore into the active internal instance.
|
||||
|
||||
### 2. Signatures
|
||||
|
||||
- Paired backup: `DB_TOOL_MODE=docker DB_CONTAINER=<name> ./scripts/backup-all.sh`.
|
||||
- Optional safe batch identifier: `BACKUP_RUN_ID=<safe-id>`.
|
||||
- DB shadow restore requires `RESTORE_SCOPE=shadow`,
|
||||
`RESTORE_CONFIRM_DATABASE=<actual-current-database>` and `DATABASE_URL`.
|
||||
- Storage shadow restore:
|
||||
`RESTORE_SCOPE=shadow ./scripts/restore-storage.sh <tar> <empty-target>`.
|
||||
- Local synthetic drill: `./scripts/qa/run-backup-restore-drill.sh`.
|
||||
- `DB_TOOL_MODE` is `auto`, `host` or `docker`; use `docker` with Compose to
|
||||
avoid host/server PostgreSQL tool-version drift.
|
||||
|
||||
### 3. Contracts
|
||||
|
||||
- A complete recovery point is one batch directory containing `database.dump`,
|
||||
`storage.tar.gz` and a final `manifest.json`. The manifest contains only safe
|
||||
relative names, byte sizes, SHA-256 hashes, batch time and optional build SHA.
|
||||
- The DB dump must pass `pg_restore --list` and the storage archive must pass
|
||||
`tar -tzf` before the manifest may say `complete`.
|
||||
- DB restore queries `current_database()` and compares it exactly with
|
||||
`RESTORE_CONFIRM_DATABASE` before `pg_restore --clean`.
|
||||
- The actual DB name must also contain a separate `shadow`, `drill` or
|
||||
`restore_test` marker; setting `RESTORE_SCOPE=shadow` alone is not trust.
|
||||
- Storage restore requires an explicit empty target and rejects absolute paths,
|
||||
`..` path segments, symbolic links and hard links before extraction.
|
||||
- Current scripts intentionally reject production restore. A real incident
|
||||
requires an approved Trellis task, stop window, paired batch, two responsible
|
||||
people, rollback point and post-restore `/ready`/permission checks.
|
||||
- Copy the whole completed batch to approved off-host/off-disk storage; a batch
|
||||
left only on the application host is not accepted recovery evidence.
|
||||
|
||||
### 4. Validation & Error Matrix
|
||||
|
||||
- One backup component fails or integrity listing fails -> remove the temporary
|
||||
batch; do not write a complete manifest.
|
||||
- Existing batch identifier -> fail without overwriting.
|
||||
- Missing/non-shadow restore scope -> fail before any restore write.
|
||||
- Actual DB name differs from confirmation -> fail before `pg_restore --clean`.
|
||||
- Confirmed DB name lacks a shadow/drill/restore-test marker -> fail before
|
||||
`pg_restore --clean`.
|
||||
- Storage target is implicit, root/repository root or non-empty -> fail before
|
||||
extraction.
|
||||
- Corrupt dump/tar or dangerous/link archive member -> fail before restore.
|
||||
- Host PostgreSQL tools mismatch in `auto` mode -> docker fallback is allowed
|
||||
only when the explicit target container and required DB variables exist.
|
||||
|
||||
### 5. Good/Base/Bad Cases
|
||||
|
||||
- Good: one command creates a validated pair, the complete batch is copied off
|
||||
host, and a disposable shadow drill restores the DB marker and attachment hash.
|
||||
- Base: backup succeeds locally but off-host copy and deployed `/ready` evidence
|
||||
remain explicitly pending; do not claim formal recovery readiness.
|
||||
- Bad: source the active trial `.env` and run a cleaning restore directly, or
|
||||
restore attachments into a non-empty live directory.
|
||||
|
||||
### 6. Tests Required
|
||||
|
||||
- Shell syntax checks for all backup/restore scripts.
|
||||
- Synthetic Docker/PostgreSQL and temporary-storage round trip.
|
||||
- Negative assertions for wrong DB confirmation, corrupt DB/tar, non-empty
|
||||
target and dangerous tar path.
|
||||
- Validate manifest relative paths, absence of connection secrets and matching
|
||||
SHA-256 values.
|
||||
- Real-instance restore and `/ready` remain separately approved evidence and
|
||||
must never be fabricated from the local drill.
|
||||
|
||||
### 7. Wrong vs Correct
|
||||
|
||||
#### Wrong
|
||||
|
||||
```bash
|
||||
source deploy/trialrun/.env
|
||||
./scripts/restore-db.sh some.dump
|
||||
./scripts/restore-storage.sh some.tar.gz deploy/trialrun/data/storage
|
||||
```
|
||||
|
||||
#### Correct
|
||||
|
||||
```bash
|
||||
./scripts/qa/run-backup-restore-drill.sh
|
||||
# Real restore remains an approved incident operation after shadow evidence.
|
||||
```
|
||||
|
||||
在新工单中引用
屏蔽一个用户