完善异机备份告警与同日恢复点取代（07-27-automatic-offhost-backup）

.trellis/spec/hyb-oa/deployment-trialrun.md

@@ -927,13 +927,18 @@ test "$(printf '%s\n' "$actual" | LC_ALL=C sort)" = \
 - Production runner: `scripts/run-automatic-backup.sh` with `run`,
   `check-stale` and `remind-drill` modes.
 - ARM forced receiver: `scripts/receive-offhost-backup.sh`.
+- Shared host alert helper: `scripts/send-wecom-group-alert.py` with explicit
+  `--webhook-file`, `--summary-file` and `--state-file` paths.
+- ARM watchdog: `scripts/check-offhost-backup-deadman.py` with archive root,
+  state root, alert helper, Webhook file and a default `93600` second threshold.
 - Installers: `install-automatic-backup-timer.sh` and
   `install-offhost-backup-receiver.sh`.
 - Local contract: `bash scripts/qa/run-automatic-backup-contract.sh`.
 - Production required environment: `REMOTE_DIR`, `REMOTE_ENV_FILE`,
   `COMPOSE_PROJECT_NAME`, `POSTGRES_DATA_DIR`, `STORAGE_HOST_DIR`, `BUILD_SHA`,
   `VITE_BUILD_SHA`, `AUTOMATIC_BACKUP_DIR`, `AUTOMATIC_BACKUP_STATE_DIR`,
-  `OFFHOST_TARGET`, `OFFHOST_SSH_KEY` and `OFFHOST_KNOWN_HOSTS`.
+  `OFFHOST_TARGET`, `OFFHOST_SSH_KEY`, `OFFHOST_KNOWN_HOSTS`,
+  `ALERT_WEBHOOK_FILE` and `ALERT_HELPER`.
 - Receiver config: independent absolute `BACKUP_INBOX_DIR`,
   `BACKUP_ARCHIVE_DIR`, `BACKUP_WEEKLY_DIR`, `RRSYNC_BIN`, `PYTHON_BIN`,
   `DAILY_KEEP` and `WEEKLY_KEEP`.
@@ -952,14 +957,32 @@ test "$(printf '%s\n' "$actual" | LC_ALL=C sort)" = \
   it rejects shell, sender/read and destructive rsync options.
 - ARM revalidates manifest identity, exact regular-file set, sizes and SHA-256,
   then atomically moves the batch to archive. Archive and weekly directories are
-  outside the rsync root; an archived name is never overwritten.
+  outside the rsync root. A different run ID for the same UTC day may replace
+  that day's archive only after full candidate validation: rename the old daily
+  point to a same-filesystem rollback name, publish the candidate, rebuild that
+  ISO week's hard-link checkpoint, then remove the rollback. Any publication or
+  weekly-checkpoint failure restores the prior daily point.
 - Write production `last-success.json` and prune source copies only after ARM
   finalize succeeds. Keep seven automatic source/ARM daily points and four ARM
   weekly checkpoints; unrelated manual batches are outside automatic retention.
 - Daily, freshness and quarterly reminder timers are separate. Alert summaries
-  contain categories only. AUTO_BACKUP sends the first failure, worsening/new
-  findings and the first quarterly reminder, while suppressing an identical
-  repeated failure or reminder.
+  contain categories only. A shared Python helper reads the complete WeCom group
+  robot URL only from a regular, executing-account-owned `0600` file and never
+  prints it. It sends the first failure, worsening/new findings, one recovery
+  notice and the first quarterly reminder, while suppressing an identical
+  repeated state. It does not call the OA server container or resolve an employee.
+- ARM runs its own 26-hour dead-man as root. It verifies the newest archive's
+  exact file set, manifest identity, sizes, checksums and timestamp, so a stopped
+  production host/server can still cause a group alert. Missing, corrupt, stale
+  or future-dated archives fail; recovery sends once.
+- Production command diagnostics for paired backup, rsync and finalize stay only
+  in the state directory as `last-error.log` mode `0600`. The summary and group
+  message contain only the safe phase category. A later successful off-host run
+  removes the retained diagnostic.
+- The production freshness check hard-fails missing, malformed, future, stale or
+  Compose-project-mismatched state. A fresh recovery point with a different
+  build SHA is `PASS_WITH_FINDINGS`, because deploy-time code identity and
+  recoverable data freshness are separate facts.
 - Installer `install` is disabled-by-default; `enable`, real transfer, real
   WeCom alert and all server/account/key changes remain `EXTERNAL_GATED`.
 
@@ -967,24 +990,32 @@ test "$(printf '%s\n' "$actual" | LC_ALL=C sort)" = \
 
 - Source mount/build/project drift, lock conflict or incomplete backup -> fail
   before success state and source retention.
-- Transfer/finalize failure, wrong/extra/link artifact, bad hash or second batch
-  for one UTC day -> leave no off-host success state and preserve source batch.
-- Missing, stale or project/build-mismatched success record -> watchdog finding.
+- Transfer/finalize failure, wrong/extra/link artifact or bad hash -> leave no
+  off-host success state, preserve the source batch and retain a root-only local
+  diagnostic. A corrupt same-day candidate leaves the previous archive intact.
+- Valid same-day retry -> replace only that UTC day's point and rebuild its ISO
+  weekly checkpoint; concurrent finalize -> `flock` rejection.
+- Missing, corrupt, stale, future or project-mismatched success record -> hard
+  watchdog failure. Fresh build mismatch -> one finding without a failing exit.
+- Missing/non-regular/symlink/non-`0600` Webhook file -> fail closed without
+  printing the file value. Identical alert state -> update/evaluate without send.
 - Ordinary shell, rsync sender/read/delete or archive overwrite -> receiver
   rejection without changing archive.
-- Host/network/WeCom all unavailable at once -> self-hosted alert cannot prove
-  its own outage; this is a documented physical boundary, not a green state.
+- ARM host/network/WeCom all unavailable at once -> the group robot path cannot
+  prove its own outage; this physical boundary is not a green state.
 
 ### 5. Good/Base/Bad Cases
 
 - Good: one verified active source creates a complete batch, write-only rsync
   resumes inside inbox, ARM finalize verifies and archives it, then production
-  writes success and prunes old automatic source points.
+  writes success and prunes old automatic source points. If production later
+  stops, ARM independently raises one dead-man alert and one recovery notice.
 - Base: local implementation and synthetic contracts pass while both server
   installers remain disabled and explicitly externally gated.
-- Bad: give production a normal ARM login key, put archive below inbox, accept
-  a new host fingerprint automatically, or prune local recovery points after
-  rsync but before target-side finalize.
+- Bad: send through `compose exec server`, hard-code a personal employee number,
+  persist a robot URL in `.env`/unit/repository, give production a normal ARM
+  login key, put archive below inbox, accept a new host fingerprint automatically,
+  or prune local points before target-side finalize.
 
 ### 6. Tests Required
 
@@ -993,28 +1024,37 @@ test "$(printf '%s\n' "$actual" | LC_ALL=C sort)" = \
 - Docker mocks must return the real `{{json .Mounts}}` shape consumed by the
   runner. A simplified tab-delimited mock is not valid evidence for real mount
   parsing.
-- Focused WeCom alert tests for stdin path redaction, preserved AUTO_BACKUP
-  issue identity, first/repeated failure and quarterly reminder behavior.
-- Environment contract assertions for all four scripts, three timers, explicit
-  project/data/build/SSH binding and disabled-by-default installation.
-- Real ARM setup, first paired batch, failure alert and quarterly shadow restore
-  are separately approved operational evidence.
+- Contract assertions for same-day replacement, corrupt-candidate preservation,
+  weekly hard-link rebuild, source diagnostic mode, build mismatch finding,
+  Webhook mode/symlink rejection and group alert behavior without a server
+  container. Test transport must be explicitly gated and local-only.
+- Environment contract assertions for both helpers, both installers, production
+  timers, ARM dead-man timer, explicit project/data/build/SSH/Webhook-file
+  binding, disabled-by-default installation and absence of a literal complete
+  robot URL in tracked or untracked repository files.
+- Real ARM setup, first paired batch, deliberate source-timer stop or transfer
+  failure, dead-man first/repeated/recovery behavior and quarterly shadow restore
+  are separately approved operational evidence. A direct robot send test alone
+  is not dead-man evidence.
 
 ### 7. Wrong vs Correct
 
 #### Wrong
 
 ```bash
+compose exec server send-alert --recipient=<personal-employee-number>
 rsync -a <batch>/ backup-host:<archive>/
-rm -rf <old-local-batches>
 ```
 
 #### Correct
 
 ```bash
-# Install only with explicit active identity; default remains disabled.
-<explicit-project-data-build-and-ssh-env> \
+# Install only with explicit active identity and a root-only Webhook file path;
+# default remains disabled.
+<explicit-project-data-build-ssh-and-webhook-file-env> \
   bash deploy/trialrun/install-automatic-backup-timer.sh install
+# ARM separately verifies archive freshness even when production is unavailable.
+python3 scripts/check-offhost-backup-deadman.py <explicit-safe-path-arguments>
 # The runner writes success and prunes only after restricted target finalize.
 bash scripts/run-automatic-backup.sh run
 ```