|
|
|
@@ -2085,7 +2085,7 @@ node scripts/qa/payment-batch-flag-snapshot.mjs restore --snapshot-file "$REPORT
|
|
|
|
|
|
|
|
|
|
### 1. Scope / Trigger
|
|
|
|
|
|
|
|
|
|
- Trigger: an approved production runner changes feature flags or another versioned configuration through an API, then uses browser/network observation, response parsing or local counters to decide whether the write landed or whether rollback is needed.
|
|
|
|
|
- Trigger: an approved production runner changes feature flags or another versioned/runtime configuration through an API or environment file, then uses browser/network observation, response parsing, container health metadata or local counters to decide whether the write landed or whether rollback is needed.
|
|
|
|
|
- Purpose: prevent an observer defect from turning a known successful write into an unnecessary compensating write, while preserving zero retry and exact audit evidence.
|
|
|
|
|
|
|
|
|
|
### 2. Signatures
|
|
|
|
@@ -2094,6 +2094,7 @@ node scripts/qa/payment-batch-flag-snapshot.mjs restore --snapshot-file "$REPORT
|
|
|
|
|
- Authoritative readback: GET the changed resources and compare `version`, configured value and effective state with both the PRE snapshot and the expected POST snapshot.
|
|
|
|
|
- Independent evidence: action-specific audit total and per-target changelog deltas.
|
|
|
|
|
- Observer sources must be named separately: page requests, browser-context requests, `APIRequestContext`, server response and audit/readback.
|
|
|
|
|
- Container restart witness: inspect whether the service actually defines a Docker healthcheck, then combine running state, restart count, public `/ready` and the changed capability's read-only response. An absent healthcheck is `health=none`, not an unhealthy result.
|
|
|
|
|
|
|
|
|
|
### 3. Contracts
|
|
|
|
|
|
|
|
|
@@ -2107,6 +2108,7 @@ node scripts/qa/payment-batch-flag-snapshot.mjs restore --snapshot-file "$REPORT
|
|
|
|
|
- Make POST validation proportional to the changed dependency graph and failure cost. A reversible, zero-use feature flag with statically verified soft consumers does not justify probing unrelated domains merely because they are available.
|
|
|
|
|
- For a confirmed low-risk, reversible configuration state, a POST witness failure defaults to preserving the applied state for read-only diagnosis. Rollback requires evidence of a real product regression, not merely a runner assertion, stale error accumulator or probe-contract mismatch.
|
|
|
|
|
- Every HTTP probe must use the real route query contract. Page-error evidence must be sliced from immediately before one navigation until that navigation completes; errors accumulated during login, PRE or earlier pages cannot fail a later page witness.
|
|
|
|
|
- A restart waiter may require Docker healthy only when the resolved running service defines a healthcheck. If none is defined, it must not manufacture a failure; use running state, stable restart count, public `/ready` and dependency-specific read-only capability checks before deciding whether rollback is warranted.
|
|
|
|
|
|
|
|
|
|
### 4. Validation & Error Matrix
|
|
|
|
|
|
|
|
|
@@ -2117,14 +2119,18 @@ node scripts/qa/payment-batch-flag-snapshot.mjs restore --snapshot-file "$REPORT
|
|
|
|
|
- Some targets changed -> `STOP_FOR_MANUAL_DECISION`; stop, preserve evidence and request an explicit human decision. Neither continue nor rollback is automatically correct.
|
|
|
|
|
- Readback itself fails -> state remains unknown; stop with zero further mutation.
|
|
|
|
|
- Expected POST is confirmed and the only witness fails -> preserve the applied state, open a fresh read-only context and classify the witness. Confirmed product regression may enter an explicitly approved rollback; runner/probe defect must not.
|
|
|
|
|
- Container is running with `health=none`, restart count is stable, public `/ready` passes and the changed capability is ready -> classify the health waiter as incompatible with the runtime contract; preserve the applied state and fix the observer separately.
|
|
|
|
|
- Container exits, restart count increases, public `/ready` fails or the changed capability remains unavailable -> treat as a real POST acceptance failure and follow the approved rollback/stop contract.
|
|
|
|
|
|
|
|
|
|
### 5. Good/Base/Bad Cases
|
|
|
|
|
|
|
|
|
|
- Good: `APIRequestContext` writes a three-item batch while a page listener reports zero; GET shows all three at `PRE+1`, audit is `+3`, so the runner continues POST verification without rollback.
|
|
|
|
|
- Base: the batch response and observer agree, readback confirms the expected state, and the audit delta matches target count.
|
|
|
|
|
- Base: a configuration-only restart targets a service without Docker healthcheck; running state, restart count, public readiness and the changed capability all pass, so the applied state is retained.
|
|
|
|
|
- Bad: treat a page-only counter as authoritative for a context request and immediately restore the PRE values before checking current versions.
|
|
|
|
|
- Bad: call rollback because it is pre-authorized while the forward result is still unknown.
|
|
|
|
|
- Bad: add unrelated domain probes to a three-flag soft-disable, send unsupported query parameters, or let errors from an earlier page fail the current Portal witness.
|
|
|
|
|
- Bad: require `.State.Health.Status=healthy` without first proving a healthcheck exists, then rollback a healthy service because Docker reports no health object.
|
|
|
|
|
|
|
|
|
|
### 6. Tests Required
|
|
|
|
|
|
|
|
|
@@ -2134,6 +2140,7 @@ node scripts/qa/payment-batch-flag-snapshot.mjs restore --snapshot-file "$REPORT
|
|
|
|
|
- Assert final evidence reconciles response count, readback versions, global audit delta and per-target changelog delta; no one source may silently override contradictions.
|
|
|
|
|
- Derive every probe path/query from the actual route/client contract. For browser witnesses, reset or snapshot the page-error index before each navigation and assert only the new slice.
|
|
|
|
|
- For low-risk reversible changes, test that a witness failure preserves POST state for diagnosis and that rollback is unreachable until a fresh read-only check confirms product regression.
|
|
|
|
|
- Cover both resolved service shapes: healthcheck present must reach healthy; healthcheck absent must require running state, unchanged restart count, public `/ready` and the dependency-specific capability. Assert `health=none` alone cannot trigger rollback.
|
|
|
|
|
|
|
|
|
|
### 7. Wrong vs Correct
|
|
|
|
|
|
|
|
|
@@ -2156,6 +2163,23 @@ forward response/counter disagree -> APPLY_UNCERTAIN
|
|
|
|
|
-> rollback only after confirmed product regression and explicit approval
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
#### Wrong
|
|
|
|
|
|
|
|
|
|
```text
|
|
|
|
|
restart service -> always wait for Docker healthy
|
|
|
|
|
-> no healthcheck object -> declare failure -> rollback
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
#### Correct
|
|
|
|
|
|
|
|
|
|
```text
|
|
|
|
|
resolve service healthcheck contract
|
|
|
|
|
-> healthcheck present: require healthy
|
|
|
|
|
-> healthcheck absent: running + stable restart count + public /ready
|
|
|
|
|
+ dependency-specific capability
|
|
|
|
|
-> rollback only for a confirmed POST acceptance failure
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
## Scenario: Unique Production Administrator Break-Glass Reset
|
|
|
|
|
|
|
|
|
|
### 1. Scope / Trigger
|
|
|
|
|