记录异机影子恢复证据(07-26-internal-ops-recovery-baseline)
ci-e2e / e2e (push) Waiting to run
ci-minimal / check-minimal (push) Waiting to run

这个提交包含在:
2026-07-27 07:30:37 +08:00
父节点 64ed844bfe
当前提交 3c4086555a
共修改 3 个文件,包含 51 行新增和 2 行删除
@@ -757,6 +757,13 @@ DB_TOOL_MODE=docker DB_CONTAINER="${DB_CONTAINERS[0]}" \
set of business-table counts with the pre-backup baseline, compares extracted
regular-file count with the active storage source, and runs `/ready` using a
schema-compatible image. `/ready` alone is not recovery evidence.
- When shadow SQL is supplied through stdin or a heredoc, `docker exec` must use
`-i` and `psql` must use `ON_ERROR_STOP=1`; otherwise an empty result can be
mistaken for a data mismatch even though no verification query ran.
- Compare fixed table counts as a keyed set, not by database-returned row order.
Require the expected unique key count, then normalize both `table=count`
collections with `LC_ALL=C sort` (or an equivalent map comparison) before
equality checking; database collation order is not an evidence contract.
### 4. Validation & Error Matrix
@@ -782,6 +789,9 @@ DB_TOOL_MODE=docker DB_CONTAINER="${DB_CONTAINERS[0]}" \
- Shadow migration count, any fixed table count, attachment regular-file count
or compatible-image `/ready` differs from baseline -> reject the recovery
point and do not enter production cutover.
- Empty, duplicate or missing table-count output -> classify as a verification
execution failure first; do not report production-data drift until stdin was
delivered and every expected key was returned exactly once.
### 5. Good/Base/Bad Cases
@@ -791,6 +801,8 @@ DB_TOOL_MODE=docker DB_CONTAINER="${DB_CONTAINERS[0]}" \
after the active application connection is matched to the database container.
- Good: a legacy container without `/ready.data.buildSha` records its inspected
`sha256:` image ID and leaves `buildSha` null rather than inventing provenance.
- Good: a heredoc query uses `docker exec -i ... psql -v ON_ERROR_STOP=1`, then
compares the complete keyed count set under deterministic `C` sorting.
- Base: backup succeeds locally but off-host copy and deployed `/ready` evidence
remain explicitly pending; do not claim formal recovery readiness.
- Bad: source a stale deployment `.env` and treat a failed dump as a prompt to
@@ -808,6 +820,9 @@ DB_TOOL_MODE=docker DB_CONTAINER="${DB_CONTAINERS[0]}" \
SHA-256 values, explicit build-SHA precedence and image-digest preservation.
- Run the synthetic drill with Node available and with a `PATH` that excludes
Node but includes Python 3; both paths must complete the same round trip.
- For real-format count checks, assert that stdin queries return the full unique
key set, and prove that a different database collation/order does not change
the pass result while a changed count still fails.
- Real-instance restore and `/ready` remain separately approved evidence and
must never be fabricated from the local drill.
@@ -833,3 +848,22 @@ DB_TOOL_MODE=docker DB_CONTAINER=<verified-container> \
SOURCE_IMAGE_DIGEST=<verified-sha256-image-id> ./scripts/backup-all.sh
# Real restore remains an approved incident operation after shadow evidence.
```
For a containerized count check, the input and comparison contract is:
```bash
# Wrong: stdin is not attached, and raw row order becomes part of the gate.
actual="$(docker exec shadow-db psql -At -F = <<'SQL'
SELECT table_key, row_count FROM shadow_counts ORDER BY table_key;
SQL
)"
test "$actual" = "$expected"
# Correct: deliver stdin, fail on SQL errors and compare keyed sets deterministically.
actual="$(docker exec -i shadow-db psql -v ON_ERROR_STOP=1 -At -F = <<'SQL'
SELECT table_key, row_count FROM shadow_counts;
SQL
)"
test "$(printf '%s\n' "$actual" | LC_ALL=C sort)" = \
"$(printf '%s\n' "$expected" | LC_ALL=C sort)"
```