增加铁路OCR金额字段安全诊断(07-28-travel-reimbursement-guide)
这个提交包含在:
@@ -273,7 +273,8 @@ await prisma.ocrResult.create({ data: { result_json: persisted } });
|
||||
imageBase64, sourceMimeType: "application/pdf" })`.
|
||||
- Output: one JSON object containing only `alias`, `provider`, `status`,
|
||||
`transportMode`, `elapsedMs`, safe document category/subtype/count,
|
||||
per-field hit booleans, missing field names and a bounded error code.
|
||||
per-field hit booleans, missing field names, an optional railway amount-field
|
||||
boolean diagnostic and a bounded error code.
|
||||
|
||||
### 3. Contracts
|
||||
|
||||
@@ -291,6 +292,12 @@ await prisma.ocrResult.create({ data: { result_json: persisted } });
|
||||
- Never output recognized identifiers, dates, amounts, buyer/seller/passenger
|
||||
names, provider messages, raw response or input bytes. Field presence is a
|
||||
boolean; errors are bounded stable codes.
|
||||
- When a railway result lacks `totalAmount`, the adapter may expose an
|
||||
ephemeral diagnostic for the documented Tencent `Total` and `Fare` fields.
|
||||
It contains only presence, string/number type, non-empty and current-parser
|
||||
acceptance booleans. The live runner re-allowlists those booleans; the
|
||||
persistence sanitizer must drop the complete diagnostic object before every
|
||||
cache or database write.
|
||||
- The JSON summary is the primary evidence even when the probe exits non-zero.
|
||||
Shell callers must capture and persist the bounded stdout summary before
|
||||
interpreting the exit status; do not let `set -e` discard a valid HOLD
|
||||
@@ -331,6 +338,9 @@ await prisma.ocrResult.create({ data: { result_json: persisted } });
|
||||
the provider mock is called.
|
||||
- Privacy tests seed raw provider and recognizable normalized markers, then
|
||||
assert serialized output contains only booleans and safe metadata.
|
||||
- Railway amount-diagnostic tests cover absent, present-but-unparseable and
|
||||
parseable fields, assert source values never reach stdout, and assert
|
||||
`sanitizeOcrResultForPersistence` removes diagnostics completely.
|
||||
- CLI test pipes one synthetic PDF over stdin with credentials absent and
|
||||
asserts one parseable redacted error object with empty stderr.
|
||||
- Caller capture test asserts a redacted HOLD JSON remains available when the
|
||||
|
||||
@@ -51,6 +51,8 @@
|
||||
- 每个进程只接收一份 PDF,验证文件签名和 7MiB 上限后最多调用一次 `RecognizeGeneralInvoice`。住宿与铁路样票分两次执行,任一失败立即停止,不自动遍历目录中的其他文件。
|
||||
- 输出只允许匿名别名、provider、票种、安全传输模式、耗时、字段命中布尔值、缺字段名和稳定错误码;不得输出文件名、原始响应、识别值、人员/企业名称、票号、金额或票面内容。
|
||||
- live runner 的代码、合成测试和生产部署先通过,再单独取得两次真实 API 调用批准;工具可用不等于真实样票已验证。
|
||||
- 当铁路票已被腾讯识别但金额缺失时,适配器可生成仅供 live runner 使用的临时字段形态诊断:只包含 `Total`/`Fare` 是否存在、是否为字符串或数字、是否非空及能否被当前金额解析器接受的布尔值。诊断不得携带字段值,必须在持久化白名单处被丢弃,普通 OCR job、缓存和数据库均不得保存。
|
||||
- 字段形态诊断先用合成 provider 响应证明不输出金额或原始 payload,再部署;真实调用仍受单次授权和“铁路未 PASS 即停止住宿”约束。
|
||||
|
||||
## 文档口径
|
||||
|
||||
|
||||
@@ -46,6 +46,8 @@
|
||||
- [x] 确认实际 provider 为 `tencent`,且只有一个 OCR job executor。
|
||||
- [x] 修正 live 样票脚本:复用 `RecognizeGeneralInvoice`、按票种必填字段和安全持久化白名单,支持单 PDF 标准输入且仅输出脱敏摘要。
|
||||
- [x] 使用合成 PDF/provider 响应覆盖住宿、铁路、非 PDF、超限、缺字段和原始响应不输出,并完成本地门禁、提交和生产部署。
|
||||
- [x] 为铁路 `Total`/`Fare` 增加只含存在、类型、非空和可解析布尔值的临时诊断;证明持久化白名单丢弃诊断且 live runner 不输出真实值。
|
||||
- [ ] 完成本地门禁、提交和生产部署后,使用剩余一次授权诊断铁路金额字段;铁路未 PASS 时仍不调用住宿样票。
|
||||
- [ ] 经单独样票批准后分别运行住宿电子普通发票和铁路电子客票 live OCR PDF 验收。
|
||||
- [ ] 只保存脱敏状态、耗时、provider 和字段命中摘要;确认不存在 mock 调用证据。
|
||||
- [ ] 按正常 API/业务生命周期清理或归档测试记录,并执行只读残留检查。
|
||||
@@ -85,6 +87,18 @@
|
||||
- 按“第一份未取得明确 PASS 即停止”约束,未调用住宿样票,也未重试铁路样票。
|
||||
- 空 stdin 捕获自检证明修正后的调用方式可在 provider 调用前保留 `HOLD/sample_empty` 摘要;该自检未消耗 OCR 次数。
|
||||
- 本次没有创建 OA 发票、附件或数据库记录;真实样票仍未验收通过,员工上传门禁保持关闭。
|
||||
- 经额外授权后,修正包装器重新验收铁路 PDF,明确得到 `HOLD/sample_pdf_invalid`;失败发生在 provider 调用前,本次未消耗腾讯 OCR 次数。
|
||||
- 匿名结构预检确认铁路 PDF 的 EOF 后存在非空白尾随数据,不符合现行严格 PDF 终止边界;不截断原件、不放宽安全门禁,需重新取得官方原始电子票据。
|
||||
- 住宿 PDF 的 PDF 头部、EOF 终止边界和 7 MiB 上限预检均通过,但因铁路票未 PASS,未调用住宿样票 provider。
|
||||
- 用户另行指定的铁路 PDF 通过同一结构预检,证明上一份 `sample_pdf_invalid` 是旧文件特有的 EOF 尾随数据问题,不是所有铁路 PDF 或校验器普遍失败。
|
||||
- 新铁路 PDF 经真实腾讯 OCR 识别为单张 `ElectronicTrainTicketFull`,票号、日期和购买方字段命中,但 `totalAmount` 未命中,终态为 `HOLD/missing_required_fields:totalAmount`;未输出或持久化任何识别值、原始响应或票面字段。
|
||||
- 当前适配器已按腾讯 SDK 合同读取铁路票 `Total` 与 `Fare` 两个金额字段;现有脱敏摘要不能区分 provider 未返回金额与返回值格式不可解析,需在另行批准后增加字段形态布尔诊断。铁路票未 PASS,因此仍未调用住宿样票 provider。
|
||||
|
||||
## 2026-07-29 铁路金额字段诊断实现验证
|
||||
|
||||
- 聚焦测试 19 项、完整后端 168 个文件/1075 项、Prisma validate、前端 typecheck/build、流程合同、Trellis governance、敏感内容和 diff 门禁通过;仅有既有依赖弃用提示与 22 个既有未注册文档 warning。
|
||||
- P2 聚合器的后端聚焦测试 94 项及前端门禁通过;其业务流因合成账号首次改密状态失败,live-readiness 因本地旧凭据和部署前构建标识不一致失败。进入 live OCR 步骤时已主动终止,生产最近 10 分钟新增发票和 OCR 尝试记录均为 0。
|
||||
- 诊断只存在于 provider 返回到 live runner 的进程内对象,持久化 sanitizer 丢弃整个 diagnostics;合成敏感标记未出现在序列化摘要中。
|
||||
|
||||
## 风险文件与操作
|
||||
|
||||
|
||||
@@ -131,6 +131,28 @@ function buildFieldHits(normalized) {
|
||||
);
|
||||
}
|
||||
|
||||
function buildTrainAmountDiagnostics(providerResult) {
|
||||
if (providerResult?.documentCategory !== "electronic_train_ticket") {
|
||||
return null;
|
||||
}
|
||||
|
||||
const source = providerResult?.diagnostics?.trainAmountFields;
|
||||
const booleanValue = (field, key) => source?.[field]?.[key] === true;
|
||||
|
||||
return {
|
||||
totalPresent: booleanValue("total", "present"),
|
||||
totalIsString: booleanValue("total", "isString"),
|
||||
totalIsNumber: booleanValue("total", "isNumber"),
|
||||
totalNonEmpty: booleanValue("total", "nonEmpty"),
|
||||
totalParseable: booleanValue("total", "parseable"),
|
||||
farePresent: booleanValue("fare", "present"),
|
||||
fareIsString: booleanValue("fare", "isString"),
|
||||
fareIsNumber: booleanValue("fare", "isNumber"),
|
||||
fareNonEmpty: booleanValue("fare", "nonEmpty"),
|
||||
fareParseable: booleanValue("fare", "parseable"),
|
||||
};
|
||||
}
|
||||
|
||||
function buildErrorSummary(alias, error) {
|
||||
return {
|
||||
alias: SAFE_ALIAS_PATTERN.test(alias) ? alias : "SAMPLE-PDF-UNKNOWN",
|
||||
@@ -174,6 +196,7 @@ async function runLiveSampleCheck({
|
||||
documentSubtype: safeToken(normalized.documentSubtype),
|
||||
documentCount: Number.isInteger(normalized.documentCount) ? normalized.documentCount : null,
|
||||
fieldHits: buildFieldHits(normalized),
|
||||
trainAmountDiagnostics: buildTrainAmountDiagnostics(providerResult),
|
||||
missingRequiredFields: qualityError?.startsWith("missing_required_fields:")
|
||||
? qualityError.slice("missing_required_fields:".length).split(",").filter(Boolean)
|
||||
: [],
|
||||
@@ -211,6 +234,7 @@ if (import.meta.url === invokedPath) {
|
||||
export {
|
||||
LiveOcrSampleError,
|
||||
buildErrorSummary,
|
||||
buildTrainAmountDiagnostics,
|
||||
parseArgs,
|
||||
readBoundedStdin,
|
||||
runLiveSampleCheck,
|
||||
|
||||
@@ -73,6 +73,20 @@ function toNumberString(value) {
|
||||
return num.toFixed(2);
|
||||
}
|
||||
|
||||
function buildNumericFieldDiagnostics(value) {
|
||||
const present = value != null;
|
||||
const isString = typeof value === "string";
|
||||
const isNumber = typeof value === "number" && Number.isFinite(value);
|
||||
|
||||
return {
|
||||
present,
|
||||
isString,
|
||||
isNumber,
|
||||
nonEmpty: isString ? value.trim().length > 0 : isNumber,
|
||||
parseable: toNumberString(value) != null,
|
||||
};
|
||||
}
|
||||
|
||||
function addNumberStrings(...values) {
|
||||
const normalized = values.map((value) => toNumberString(value));
|
||||
if (normalized.some((value) => value == null)) {
|
||||
@@ -221,6 +235,12 @@ function normalizeElectronicTrainTicket(info) {
|
||||
taxAmount,
|
||||
totalAmount,
|
||||
buyerName: normalizeString(info?.Buyer),
|
||||
diagnostics: {
|
||||
trainAmountFields: {
|
||||
total: buildNumericFieldDiagnostics(info?.Total),
|
||||
fare: buildNumericFieldDiagnostics(info?.Fare),
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -106,6 +106,60 @@ describe("check-ocr-provider-samples", () => {
|
||||
expect(summary.missingRequiredFields).toEqual(["buyerName"]);
|
||||
});
|
||||
|
||||
it("exposes only boolean railway amount diagnostics", async () => {
|
||||
const summary = await liveRunner.runLiveSampleCheck({
|
||||
alias: "SAMPLE-PDF-TRAIN-DIAGNOSTIC",
|
||||
buffer: syntheticPdf(),
|
||||
config: providerConfig(),
|
||||
runOcr: vi.fn().mockResolvedValue(
|
||||
normalizedResult({
|
||||
documentSubtype: "ElectronicTrainTicketFull",
|
||||
documentCategory: "electronic_train_ticket",
|
||||
invoiceCode: null,
|
||||
totalAmount: null,
|
||||
sellerName: null,
|
||||
diagnostics: {
|
||||
trainAmountFields: {
|
||||
total: {
|
||||
present: true,
|
||||
isString: true,
|
||||
isNumber: false,
|
||||
nonEmpty: true,
|
||||
parseable: false,
|
||||
value: "forbidden-total-marker",
|
||||
},
|
||||
fare: {
|
||||
present: false,
|
||||
isString: false,
|
||||
isNumber: false,
|
||||
nonEmpty: false,
|
||||
parseable: false,
|
||||
value: "forbidden-fare-marker",
|
||||
},
|
||||
},
|
||||
},
|
||||
}),
|
||||
),
|
||||
});
|
||||
const serialized = JSON.stringify(summary);
|
||||
|
||||
expect(summary.status).toBe("HOLD");
|
||||
expect(summary.trainAmountDiagnostics).toEqual({
|
||||
totalPresent: true,
|
||||
totalIsString: true,
|
||||
totalIsNumber: false,
|
||||
totalNonEmpty: true,
|
||||
totalParseable: false,
|
||||
farePresent: false,
|
||||
fareIsString: false,
|
||||
fareIsNumber: false,
|
||||
fareNonEmpty: false,
|
||||
fareParseable: false,
|
||||
});
|
||||
expect(serialized).not.toContain("forbidden-total-marker");
|
||||
expect(serialized).not.toContain("forbidden-fare-marker");
|
||||
});
|
||||
|
||||
it("never exposes provider raw fields or normalized business values", async () => {
|
||||
const summary = await liveRunner.runLiveSampleCheck({
|
||||
alias: "SAMPLE-PDF-PRIVACY",
|
||||
|
||||
@@ -139,12 +139,66 @@ describe("Tencent general invoice OCR", () => {
|
||||
totalAmount: "553.00",
|
||||
sellerName: null,
|
||||
buyerName: "合成公司",
|
||||
diagnostics: {
|
||||
trainAmountFields: {
|
||||
total: {
|
||||
present: true,
|
||||
isString: true,
|
||||
isNumber: false,
|
||||
nonEmpty: true,
|
||||
parseable: true,
|
||||
},
|
||||
fare: {
|
||||
present: false,
|
||||
isString: false,
|
||||
isNumber: false,
|
||||
nonEmpty: false,
|
||||
parseable: false,
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(buildMissingRequiredOcrError("vat", normalized)).toBeNull();
|
||||
expect(shouldDeduplicateByInvoiceNumberOnly(normalized)).toBe(true);
|
||||
expect(JSON.stringify(normalized)).not.toContain("SENSITIVE-");
|
||||
});
|
||||
|
||||
it("reports only boolean railway amount field diagnostics", () => {
|
||||
const normalized = normalizeGeneralInvoiceResult({
|
||||
MixedInvoiceItems: [
|
||||
buildInvoiceItem("ElectronicTrainTicketFull", "ElectronicTrainTicketFull", {
|
||||
Number: "251234",
|
||||
Date: "2026-07-26",
|
||||
Fare: "not-a-number-sensitive-marker",
|
||||
Buyer: "合成公司",
|
||||
}),
|
||||
],
|
||||
});
|
||||
|
||||
expect(normalized.diagnostics).toEqual({
|
||||
trainAmountFields: {
|
||||
total: {
|
||||
present: false,
|
||||
isString: false,
|
||||
isNumber: false,
|
||||
nonEmpty: false,
|
||||
parseable: false,
|
||||
},
|
||||
fare: {
|
||||
present: true,
|
||||
isString: true,
|
||||
isNumber: false,
|
||||
nonEmpty: true,
|
||||
parseable: false,
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(JSON.stringify(normalized.diagnostics)).not.toContain("sensitive-marker");
|
||||
expect(sanitizeOcrResultForPersistence("tencent", normalized)).not.toHaveProperty(
|
||||
"diagnostics",
|
||||
);
|
||||
});
|
||||
|
||||
it("returns safe terminal quality results for zero, multiple and unsupported invoices", () => {
|
||||
expect(normalizeGeneralInvoiceResult({ MixedInvoiceItems: [] })).toEqual(
|
||||
expect.objectContaining({ qualityError: "invoice_not_detected", documentCount: 0 }),
|
||||
|
||||
在新工单中引用
屏蔽一个用户