建立每日自动异机备份基线(07-27-automatic-offhost-backup)

这个提交包含在:
2026-07-27 10:50:01 +08:00
父节点 325c592aba
当前提交 b78f2a11ce
共修改 19 个文件,包含 2070 行新增和 10 行删除
@@ -912,3 +912,102 @@ SQL
test "$(printf '%s\n' "$actual" | LC_ALL=C sort)" = \
"$(printf '%s\n' "$expected" | LC_ALL=C sort)"
```
## Scenario: Daily Restricted Off-Host Backup
### 1. Scope / Trigger
- Trigger: installing or changing automatic paired backups, off-host transfer,
retention, freshness alerts or quarterly restore reminders.
- Purpose: keep a 24-hour recovery point without giving the production host
ordinary shell or delete access to the ARM archive.
### 2. Signatures
- Production runner: `scripts/run-automatic-backup.sh` with `run`,
`check-stale` and `remind-drill` modes.
- ARM forced receiver: `scripts/receive-offhost-backup.sh`.
- Installers: `install-automatic-backup-timer.sh` and
`install-offhost-backup-receiver.sh`.
- Local contract: `bash scripts/qa/run-automatic-backup-contract.sh`.
- Production required environment: `REMOTE_DIR`, `REMOTE_ENV_FILE`,
`COMPOSE_PROJECT_NAME`, `POSTGRES_DATA_DIR`, `STORAGE_HOST_DIR`, `BUILD_SHA`,
`VITE_BUILD_SHA`, `AUTOMATIC_BACKUP_DIR`, `AUTOMATIC_BACKUP_STATE_DIR`,
`OFFHOST_TARGET`, `OFFHOST_SSH_KEY` and `OFFHOST_KNOWN_HOSTS`.
- Receiver config: independent absolute `BACKUP_INBOX_DIR`,
`BACKUP_ARCHIVE_DIR`, `BACKUP_WEEKLY_DIR`, `RRSYNC_BIN`, `PYTHON_BIN`,
`DAILY_KEEP` and `WEEKLY_KEEP`.
### 3. Contracts
- Resolve one active Compose server/database pair and verify its build, PGDATA,
storage mount and database identity before backup. Do not use fixed container
names or repository-relative runtime data.
- Transfer only a complete `auto-<UTC timestamp>` three-file batch to ARM
inbox. The forced key allows write-only `rrsync` or an exact finalize command;
it rejects shell, sender/read and destructive rsync options.
- ARM revalidates manifest identity, exact regular-file set, sizes and SHA-256,
then atomically moves the batch to archive. Archive and weekly directories are
outside the rsync root; an archived name is never overwritten.
- Write production `last-success.json` and prune source copies only after ARM
finalize succeeds. Keep seven automatic source/ARM daily points and four ARM
weekly checkpoints; unrelated manual batches are outside automatic retention.
- Daily, freshness and quarterly reminder timers are separate. Alert summaries
contain categories only. AUTO_BACKUP sends the first failure, worsening/new
findings and the first quarterly reminder, while suppressing an identical
repeated failure or reminder.
- Installer `install` is disabled-by-default; `enable`, real transfer, real
WeCom alert and all server/account/key changes remain `EXTERNAL_GATED`.
### 4. Validation & Error Matrix
- Source mount/build/project drift, lock conflict or incomplete backup -> fail
before success state and source retention.
- Transfer/finalize failure, wrong/extra/link artifact, bad hash or second batch
for one UTC day -> leave no off-host success state and preserve source batch.
- Missing, stale or project/build-mismatched success record -> watchdog finding.
- Ordinary shell, rsync sender/read/delete or archive overwrite -> receiver
rejection without changing archive.
- Host/network/WeCom all unavailable at once -> self-hosted alert cannot prove
its own outage; this is a documented physical boundary, not a green state.
### 5. Good/Base/Bad Cases
- Good: one verified active source creates a complete batch, write-only rsync
resumes inside inbox, ARM finalize verifies and archives it, then production
writes success and prunes old automatic source points.
- Base: local implementation and synthetic contracts pass while both server
installers remain disabled and explicitly externally gated.
- Bad: give production a normal ARM login key, put archive below inbox, accept
a new host fingerprint automatically, or prune local recovery points after
rsync but before target-side finalize.
### 6. Tests Required
- Shell syntax plus `run-automatic-backup-contract.sh` using temporary files and
mocked Docker/SSH boundaries; it must never contact real hosts.
- Focused WeCom alert tests for stdin path redaction, preserved AUTO_BACKUP
issue identity, first/repeated failure and quarterly reminder behavior.
- Environment contract assertions for all four scripts, three timers, explicit
project/data/build/SSH binding and disabled-by-default installation.
- Real ARM setup, first paired batch, failure alert and quarterly shadow restore
are separately approved operational evidence.
### 7. Wrong vs Correct
#### Wrong
```bash
rsync -a <batch>/ backup-host:<archive>/
rm -rf <old-local-batches>
```
#### Correct
```bash
# Install only with explicit active identity; default remains disabled.
<explicit-project-data-build-and-ssh-env> \
bash deploy/trialrun/install-automatic-backup-timer.sh install
# The runner writes success and prunes only after restricted target finalize.
bash scripts/run-automatic-backup.sh run
```