修复(08-08-production-deployment-execution): 持久化远端部署阶段日志

2026-08-08 21:29:55 +08:00
@@ -1460,6 +1460,9 @@ bash scripts/qa/run-automatic-backup-contract.sh
 - Aggregate read-only preflight: `deploy/trialrun/check-existing-deployment.sh`.
 - Root-owned forced-command source/installer: `scripts/refresh-deployment-contract.sh` and `deploy/trialrun/install-deployment-contract-refresh.sh`.
 - Remote preflight: `deploy/trialrun/preflight-remote.sh` executed over SSH before rsync.
+- Persistent long-stage evidence: repo-relative
+  `artifacts/deployment-runtime/<stage>-<12-char-sha>.log` for `build-init`,
+  `run-init` and `build-switch-server`.
 - Canonical parameter source: allowlisted `Environment=` values from `hyb-oa-automatic-backup-daily.service`.
 - Stable image stages: `base`, `client_deps`, `client_build`, `server_deps`, `server_build`.

@@ -1479,6 +1482,20 @@ bash scripts/qa/run-automatic-backup-contract.sh
 - After that partial-success state, repair the classified root/program/unit/timer problem, invoke the forced-command `refresh` directly, then rerun aggregate preflight. The application already passed release smoke; do not redeploy it merely to repair the backup build contract, and do not bypass the expected `backup_build` failure while it is stale.
 - `apk add` and both lockfile `npm ci` steps precede `BUILD_SHA` / `VITE_BUILD_SHA`. Build identity may invalidate application output, never stable dependency installation.
 - Seed and smoke output reports readiness, counts and booleans; it does not print full login identifiers, contact values, WeCom identifiers, redirect URI or generated credentials.
+- The three long remote mutation stages write stdout/stderr directly to a
+  remote file instead of relying on the SSH stream as the only copy. The log
+  directory is mode `0700`; each fresh log is mode `0600`. Stage names are a
+  fixed allowlist at call sites, and the filename uses the provenance-verified
+  12-character candidate prefix.
+- A stage success/failure emits only stage, exact child exit code and a
+  repository-relative `logRef`. Full raw logs remain remote and uncommitted;
+  after failure, retrieve only the needed tail read-only and redact any
+  sensitive content before writing task/chat evidence. Do not automatically
+  dump init/seed logs into ordinary output.
+- Logs are created after the verified-source rsync. A later rsync may remove
+  old runtime artifacts, so retrieve a failed stage log before any approved
+  retry. Keeping an interactive execution handle is not a substitute for this
+  second copy.

 ### 4. Validation & Error Matrix

@@ -1495,16 +1512,28 @@ bash scripts/qa/run-automatic-backup-contract.sh
 - Post-deploy installer still embeds the previous build SHA, any managed timer is inactive, or the real backup/watchdog fails runtime identity or off-host freshness -> deployment recovery remains incomplete even when `/ready` is green.
 - A later deployment attempt is blocked by `backup_build` after `DEPLOYED_BUT_CONTRACT_REFRESH_FAILED` -> expected fail-closed behavior; repair with direct forced-command `refresh`, not another application release.
 - Lockfile unchanged but a new SHA reruns dependency installation -> Dockerfile contract test fails.
+- Long stage exits non-zero -> preserve its original status and relative log
+  reference, stop under `set -e`, and do not enter the next stage.
+- SSH stream disappears -> use the already-written remote log for diagnosis;
+  do not repeat the deployment merely to recreate console output.
+- Runtime log directory/file mode is broader than `0700`/`0600`, stage output
+  has no log reference, or a bare build/init/server-switch call bypasses the
+  wrapper -> deployment evidence contract fails locally.

 ### 5. Good/Base/Bad Cases

 - Good: wrapper reads the frozen unit contract, preflight passes, Git archive syncs and unchanged dependency layers are cached.
+- Good: a long stage writes its complete remote log, returns a stable relative
+  reference and preserves the exact non-zero child status even if the
+  interactive stream is incomplete.
 - Good: ordinary and forced-command identities pass separately, all aggregate items are reported, deployment passes, root helper updates only fixed build drop-ins, and four timers remain enabled/active.
 - Good: a broken-root recovery creates a clean published-commit source tree, proves one environment candidate against running containers without exposing values, copies it with mode `0600`, deploys with `SYNC_ENV=0`, then updates the backup contracts and completes a real off-host backup.
 - Base: first initialization uses reviewed explicit parameters and `DEPLOY_DATA_MODE=initialize`; it does not pretend an existing unit is available.
 - Bad: inspect a running container's `working_dir`, append another `deploy/trialrun`, then discover the mistake only after rsync.
 - Bad: choose one of several stale `.env` files by path or modification time, use `SYNC_ENV=1`, or stop after `/ready` while the backup unit still expects the previous build.
 - Bad: grant NOPASSWD to a repository installer, point forced-command at a deploy-user-writable script, reuse an unrestricted root key, stop aggregation after its first failure, or downgrade refresh failure to a warning.
+- Bad: run `docker compose build/run/up` as bare SSH-stream-only calls, lose the
+  terminal tail, then retry production solely to recover diagnostics.

 ### 6. Tests Required

@@ -1513,6 +1542,9 @@ bash scripts/qa/run-automatic-backup-contract.sh
 - `node deploy/trialrun/validate-env-contract.js`.
 - `cd server && npm test -- tests/unit/scripts/deployment-safety-contract.test.js`.
 - Synthetic fixtures must prove wrong root, missing/symlink env and invalid data fail; static ordering must prove preflight precedes rsync and dependency stages precede build args.
+- Static deployment tests must prove all three long stages use the persistent
+  wrapper, output is redirected to the mode-`0600` file, the error branch
+  captures `$?` immediately, and the former bare calls are absent.
 - A controlled recovery must additionally record: exact published source commit, per-key PASS set/date without values, byte-copy/owner/mode PASS, new-SHA service/timer contract PASS, a successful complete backup, and a successful off-host freshness check. Old candidates remain untouched until a separately approved cleanup.

 ### 7. Wrong vs Correct
@@ -1538,6 +1570,14 @@ DEPLOY_SERVER=<approved-ssh-target> \
 # recovery runbook to prove one complete backup and off-host freshness.
 ```

+```bash
+# Wrong: the SSH stream is the only build evidence.
+"${compose_cmd[@]}" build init
+
+# Correct: the remote file is authoritative and the stream carries a safe ref.
+run_logged_deploy_stage "build-init" "${compose_cmd[@]}" build init
+```
+
 ## Scenario: Production Migration-First Target Gate

 ### 1. Scope / Trigger
@@ -80,6 +80,8 @@ bash deploy/trialrun/deploy-existing.sh

 包装器固定 `SYNC_ENV=0`、`DEPLOY_DATA_MODE=existing` 和 `DEPLOY_STATUS_OUTPUT_MODE=quiet`。`deploy.sh` 内部的 `prisma migrate deploy` 是本 task 唯一允许的 schema 写入；不允许 seed、业务 API 写入、reset、resolve 或手工 DDL。

+远端 `build init`、`run init` 和 `up --build server` 三个长阶段必须把完整 stdout/stderr 写入受控 `artifacts/deployment-runtime/`：目录 `0700`、日志 `0600`，文件名由固定阶段名和已验证候选 12 位 SHA 组成。普通输出只返回 `stage`、真实退出码和仓库相对 `logRef`；SSH 通道中断时已写入内容仍留在远端，后续以只读方式取尾部。原始日志不进 Git/task/普通对话，审查摘要必须脱敏。
+
 部署后 helper 以空 pending allowlist重新执行 target/status/reconciliation，要求 schema up to date；随后才执行 refresh 和只读 readiness。refresh 失败分类为 `DEPLOYED_BUT_CONTRACT_REFRESH_FAILED`，不重复发布应用。

 ## 6. 回滚窗口
@@ -102,6 +104,7 @@ bash deploy/trialrun/deploy-existing.sh
 | 历史身份/refresh/bootstrap | 历史 commit 引用、exact probe 计数、contract/origin 匹配和 DB 引导状态码 | 任一不唯一/不一致即停止，不进入 Prisma status |
 | 来源/13 项体检 | commit、published/remoteVerified、13 项 PASS 计数 | 停止，不执行部署写入 |
 | target/pre-status/窗口 | 原始受控输出引用、脱敏副本、pending/异常计数、窗口状态 | `PAUSED`，不进入 deploy |
+| build/init/server switch | 固定阶段 PASS、相对 `logRef`、候选短 SHA | 远端 `0600` 日志保留，读取尾部后再定性；不得仅凭 SSH 流或盲重试 |
 | migrate deploy | deploy wrapper 返回、候选 migration ID | 保留现场，不 reset/resolve/drop/重复部署 |
 | post-status/readiness | applied、schema up to date、`/ready`/SHA/smoke | 按窗口状态决定回退或 roll-forward |
 | refresh | exact forced-command PASS | `DEPLOYED_BUT_CONTRACT_REFRESH_FAILED`，只修复合同 |
@@ -165,3 +165,17 @@
 - termination migration 尚未应用时，终止能力不可能由本轮应用生效；helper 现以 `_prisma_migrations` applied set 走显式分支，不发送 `contract_registry` 状态查询，固定输出 `rollbackWindow=OPEN terminatedRows=0 basis=migration_not_applied`。
 - migration 已应用时继续执行真实 `status = 'terminated'` 计数，输出 `basis=terminated_status_count`；查询失败或结果不可解析分别返回固定错误，不使用 catch/default 把未知状态误报为 OPEN。
 - 聚焦 fixture 覆盖：未应用且状态查询被设置为必失败仍应 OPEN（证明未查询）、已应用 0 行 OPEN、已应用 2 行 CLOSED，以及已应用查询失败必须 `PROD_TERMINATED_COUNT_UNAVAILABLE`。聚焦测试 19/19 PASS。
+
+## First Production Deployment Attempt Evidence (2026-08-08)
+
+- 窗口依据修复提交 `b94afcf` 已进入 `origin/main`；发布前 preflight 再次通过：精确身份组、13 项聚合体检、数据库三方身份、唯一 pending allowlist 和 `rollbackWindow=OPEN ... basis=migration_not_applied` 全部 PASS。
+- 唯一生产入口随后完成已验证源码同步并进入远端 `build init`；编排进程在该长阶段退出，未到 `run init`。代码顺序固定为先 `build init`、后 `prisma:migrate:deploy`，只读复核同时证明 termination migration 仍 pending、公开 `/ready` 仍为旧 build、新 init 镜像仍为旧 build，故 schema 写入和应用切换均为 0。
+- 同一候选在本地 `client` typecheck/build PASS；远端当时仍有可用磁盘和内存，未取得明确 OOM 证据，不能把环境猜测写成根因。原裸调用只把输出写入 SSH 流，通道证据不完整，未能恢复真实末尾错误。
+- 失败后未重试部署、未运行 reset/resolve/drop、未改生产配置。额外只读检查确认现有 13 项合同仍健康、回滚窗口仍 OPEN；用户随后只批准一次在远端持久化日志机制落地后的受控重试。
+
+## Persistent Remote Stage Log Repair Evidence (2026-08-08)
+
+- `deploy.sh` 为 `build-init`、`run-init` 和 `build-switch-server` 三个长阶段统一增加远端日志包装器；完整 stdout/stderr 直接写入仓库相对 `artifacts/deployment-runtime/`，目录 `0700`、文件 `0600`，文件名只含固定阶段名和已验证候选 12 位 SHA。
+- 普通输出仅记录 stage、真实 child exit code 和相对 `logRef`。失败状态在 Bash `else` 分支中立即捕获并返回，随后由 `set -e` 停止，不会因 `fi` 后读取 `$?` 而丢失原码。
+- 原始 build/init 日志不写入 Git、task 或普通对话；若受控重试失败，先只读取对应远端日志尾部并脱敏定性，再决定后续动作。任何下一次 rsync 可能清除旧 runtime artifacts，因此读取必须发生在重试之前。
+- 聚焦静态回归证明三个裸调用均已替换、重定向和权限合同存在、失败分支立即捕获退出码；shell syntax、env contract 与部署安全测试 PASS。