1. Every input is hostile
An audit report is an untrusted document that a stranger emailed you. It may be a scan, it may be nine hundred pages, and it may be a PDF whose object graph was built by a tool that stopped being maintained in 2014. The parser runs with no network, no filesystem and a hard wall-clock budget, and it is restarted rather than reused.
2. The budget is the SLA
Rather than promise “we parse everything”, the pipeline promises a decision within a fixed time. A document that cannot be parsed inside the budget is escalated to a human with the reason attached, which is a far more useful outcome than a job that silently retries all afternoon.
{
"artifact": "soc2_type_ii_2026.pdf",
"stage": "EXTRACT",
"budget_ms": 30000,
"on_timeout": "ESCALATE_TO_REVIEWER"
}3. Confidence, not certainty
Every extracted field carries a confidence score and the page it came from. Below the threshold, the field is presented as a suggestion rather than a fact, and the reviewer confirms it in one click. That single design decision is why the pipeline can be aggressive: being wrong is visible and cheap.
4. What broke
The first version normalised whitespace before locating headings, which quietly merged two control sections in reports produced by one large audit firm. It passed every test we had, because every test used a report from a different firm. The fix was to test against the shape of the input, not against a sample of it.
About David Kross
David is the lead security architect at Urengi, where he owns the evidence extraction pipeline. He writes about parsing untrusted documents at scale and the SLAs that make an automated control defensible in front of an auditor.
