A value produced by a model is a claim. It becomes operational state only when something with authority admits it. Consequential document extraction should therefore be treated as a state-admission and resource-allocation problem rather than a generation problem: candidate values remain non-authoritative until evidence, deterministic constraints, estimated residual risk, dependency structure, consequence, expected review benefit, and policy jointly support automatic commitment — and otherwise the affected decision group is routed to human review or held.
And the corollary this paper does not soften: almost none of the machinery here is new. Selective prediction, learning to defer, cost-sensitive abstention, and capacity-constrained routing are all established fields with substantial literatures. Section 3 states exactly what is borrowed and narrows the contribution to what survives. What survives is smaller than the framing would allow, and it is stated at that smaller size deliberately.
—Abstract
Organizations increasingly derive structured values from heterogeneous documents using probabilistic models, and then write those values into systems that other decisions depend on. The step between those two events is usually implicit. A confidence threshold, a validation pass, or a schema check stands in for a decision that nobody has specified: under what conditions may a model-derived value become the state of record?
This paper specifies that decision. It formalizes the boundary between extraction and authoritative operational state as an admission-control problem under finite human review capacity, with three commitments that distinguish it from a threshold.
The decision unit is a dependency group, not a field. Fields are partitioned into connected components of a loss-coupling graph, in which two fields are adjacent when the operational consequence of an error in one depends on the correctness of the other. Actions are taken over groups. This is what allows the loss function to be non-additive, which in turn is what allows identity-binding errors — a correct amount attached to the wrong account, contract, patient, or line item — to be represented at all. Additive per-field loss cannot express them, and every routing policy built on additive loss systematically under-prioritizes exactly the fields whose errors propagate.
The action set is commit, review, or abstain over state. Not accept-or-reject over a prediction. Abstention is a first-class terminal action with its own policy-specified cost, which changes the value of review: the counterfactual to reviewing a group is the best action available without review, which is frequently to hold rather than to commit.
Validation is reported as consistency, never as correctness. A candidate that satisfies every deterministic invariant may still be wrong, because internal consistency and external truth are different properties. The paper draws an explicit three-layer epistemic boundary — document fidelity, source authenticity, external truth — claims the first, surfaces the second, and disclaims the third.
On this foundation the paper derives: exact optimality of top-B selection under unit review cost, and the precise sense in which grouping is what buys that optimality (Lemma 1); a tight selection-regret bound showing that regret is controlled by error in estimating the value of intervention on a narrow boundary set, not by aggregate calibration, together with a four-group counterexample that kills the aggregate-calibration argument outright (Theorem 1); a decomposition of group review value under non-additive loss into a consequence-weighted shift in the distribution over outcome classes, with the corollary that review value is bounded by consequence times total-variation shift — so an arbitrarily high error probability yields near-zero review value when a reviewer cannot resolve the ambiguity (Proposition 2, Corollary 4); and an amplification result showing that field-level materiality ranking under-prioritizes identity fields by a factor that grows without bound in the binding-amplification ratio (Proposition 5).
The paper also specifies the substrate that produces the estimator's inputs — constrained decoding, symbolic cross-field validation, unsat-core-bounded repair, span-alignment provenance, self-consistency sampling, and a tamper-evident decision trace — and is explicit that all of it is prior art serving a decision layer, with a single boundary rule attached to each: a passed check is evidence of consistency, never of correctness.
Every empirical quantity in this framework is left UNFILLED. Machine residual risk, human residual error, consequence weights, review cost, abstention cost, capacity, and all effect sizes are named, given estimators, and left unmeasured. That is a deliberate constraint rather than an omission: the paper's companion argues at length that borrowed figures destroy business cases, and a sequel that quoted unmeasured numbers would retract that argument. Section 4 explains what the paper is therefore not entitled to claim.
—What this paper contributes
Six contributions, each developed in a specific section rather than asserted in passing. Each is stated at the size it survived a prior-art audit, which in two cases is considerably smaller than the first draft claimed.
1. A specification of the admission boundary itself. The transition from model output to system of record is treated as a governed decision with a named action set, a loss model, a capacity constraint, and a policy surface — rather than as an implementation detail buried in a threshold. → Sections 5, 6, 31
2. Dependency groups as the unit of decision, with non-additive loss. Deferral at sub-prediction granularity is established prior art. What is specified here is the derivation of the unit from a loss-coupling graph and the consequences that follow: exact top-B optimality via modularity, and a provable failure mode for field-level ranking. → Sections 6, 10, 15
3. A tight selection-regret bound, and the demolition of aggregate calibration as a sufficient condition. Regret is controlled by \(\|v-\hat v\|_\infty\) on the symmetric difference between the chosen and optimal review sets. Population-level calibration constrains an average and says nothing about that set. → Sections 11, 12, 30
4. Identity-key amplification. Under key-binding loss, group review value exceeds the field-level review value of the key by a ratio that grows without bound in the binding-amplification factor. This converts a hypothesis about dependency-aware routing into a statement with a mechanism and a falsification condition. → Section 15
5. An evaluation contract that refuses the two standard shortcuts. Deterministic validation pass is reported as consistency yield rather than accuracy; reviewer residual error is measured or declared a policy parameter, never assumed to be zero. → Sections 35, 36, Appendix D
6. An explicit account of what cannot be claimed and what would falsify the framework. Including a finding that no public document-extraction corpus supplies reviewer outputs, which makes one of the paper's own hypotheses untestable on public data — stated rather than worked around. → Sections 36, 37, Appendix E
—Who this paper is for
Architects and engineering leaders building document-processing systems whose outputs enter systems of record; heads of data, risk, and control functions accountable for the accuracy of state that machines now write; operations leaders who own the review capacity this framework allocates; and researchers working on selective prediction, learning to defer, and document information extraction who want the operational version of those problems stated precisely.
The mathematics is elementary and self-contained: expectations, a partition argument, a bijection, one total-variation bound. No result requires more than an undergraduate probability course. The engineering content assumes familiarity with structured extraction pipelines but not with the decision-theory literature, which Section 3 introduces from its origins.
—Relationship to Paper 01
This is the second of two papers with a shared thesis and a divided subject.
Paper 01 controls action. From Copilots to Controlled Digital Operations argues that probabilistic reasoning must not acquire execution authority: agents that read untrusted content hold no write tools, execution flows through a typed gateway that authorizes on state and policy rather than on model-supplied rationale, and transitions gate on evidence and deterministic post-conditions rather than on self-reported confidence.
Paper 02 controls admission. That architecture has a gap at its own foundation. It governs what a system may do; it assumes the state the system reasons over is already true. But in any document-driven process, much of that state was itself produced by a model reading a document. If extraction is ungoverned, then the entire control architecture of Paper 01 sits on top of values that were admitted without a decision — and a governed action taken on inadmissible state is not governed.
A value is not operational state because a model emitted it.
Evidence constrains interpretation. Policy governs authority.
The two papers share their register, their refusal to route on self-reported confidence, and their insistence that a control must be computable from state rather than from prose. They differ in object: Paper 01's decision is may this action execute; Paper 02's is may this value become the state that actions depend on. Read together, they close a loop. Read separately, each has a hole where the other stands.
Paper 01 is not a prerequisite. Where a concept is inherited it is restated here in full.
—A note on numbers, and on what is deliberately missing
Paper 01 contained a section titled Avoid fake ROI, flagged its worked trace as modeled three separate times, and instructed readers not to borrow its figures. A sequel that quoted unmeasured metrics would retroactively invalidate that instruction. This paper therefore operates under a rule stricter than its companion's, and the rule is worth stating before the first number appears.
No number in this paper is a measurement. Every quantity is exactly one of four things, and is labeled as such where it appears:
Stipulated. A parameter chosen to make a worked example arithmetically concrete — "suppose the binding-error consequence is twelve times the payload consequence." Stipulated values demonstrate the structure of a calculation. They are not claims about any domain.
Derived. A value computed from stipulated parameters through the paper's own equations. Every figure plotted in this paper is derived in this sense: the curves are plots of closed-form expressions under stated parameters, not fits to data.
Policy-specified. A value an organization sets rather than discovers — abstention cost, review capacity, consequence weights, the escalation threshold. These are inputs to the framework, not outputs of it.
Unfilled. Marked UNFILLED throughout. Machine residual risk, human residual error, all effect sizes, and every quantity in the hypothesis register. These require experiments that this paper does not report.
The consequence is that this paper is a specification with a falsification programme, not an empirical result. Section 36 states the hypotheses, Appendix F lists every unfilled slot with the estimator that would fill it, and Section 37 gives the conditions under which the framework should be judged wrong. A reader looking for evidence that this approach improves outcomes will not find it here, and should be suspicious of any paper of this shape that claims otherwise without a data section.
One further absence is worth naming early, because it constrains what the empirical programme can ever demonstrate on public data. No public document-extraction benchmark supplies human reviewer outputs. Every candidate corpus gives documents and gold values; none gives what a reviewer produced when shown an ambiguous case. Human residual error is therefore not measurable on any of them at any budget, which makes one of this paper's six hypotheses untestable outside a reviewer study. This paper's response is to declare \(r_H\) a policy parameter and report a sensitivity surface over it, rather than to assume a value, borrow one, or drop the affected claim. Appendix E documents the corpus landscape and its licensing; Section 35.6 states the consequence and Appendix F records the decision.
—Contents
- 07Setup: fields, loss coupling, dependency groups
- 08The action set and the cost model
- 09Selection value, and why it is not the commit gap
- 10Assumptions, stated and discharged
- 11Lemma 1: exact optimality of top-B
- 12Theorem 1: selection regret, and its tightness
- 13Why aggregate calibration is not sufficient
- 14Proposition 2: review value under non-additive loss
- 15Corollary 4: consequence times distributional shift
- 16Proposition 5: identity-key amplification
- 17Proposition 3: hard gates as a limiting case
- 18Proposition 1: the degenerate corner
- 19Heterogeneous review cost: the knapsack case
- 20Review as information acquisition
- 21Misspecification of the dependency graph
- 22Reference pipeline and the evidence chain
- 23Constrained decoding: removing syntactic error
- 24Symbolic cross-field validation, and why SAT is not truth
- 25Unsat-core bounded repair
- 26Span alignment as measured anchoring
- 27Self-consistency as an uncertainty coordinate
- 28The tamper-evident decision trace
- 29Estimating the selection value
- 30Where calibration effort should go
- 31The admission controller as a runtime component
- 32Policy surface: consequence, capacity, abstention
- 33Worked example with stipulated parameters
- 34Failure modes and anti-patterns
- 35The evaluation contract
- 36Hypotheses and what would falsify them
- 37When not to build this
- 38Closing: authority is the scarce resource
- 01The admission boundary
- 02Field, group, document: three units of decision
- 03The three-action decision surface
- 04Selection regret and the boundary band
- 05Regret against estimator error
- 06Review value under distributional shift
- 07Identity-key amplification
- 08Risk–coverage under three routing policies
- 09The extraction substrate and evidence chain
- 10Consistency is not correctness
- 11Top-B against knapsack allocation
- 12The admission controller at runtime
What the admission boundary is, why the standard treatment of it is inadequate, what the relevant literatures already establish, and what the framework is entitled to claim about truth.
01The gap between extraction and state
Consider what actually happens when an organization processes a document.
A file arrives. Software parses it, a model reads it, and structured values come out: an amount, a date, an account identifier, a party name, a quantity, a code. Those values are written somewhere — a database row, a ledger entry, a case record, a claim, a schedule, a position. From that moment they are indistinguishable from values a human typed. Downstream systems query them. Reports aggregate them. Reconciliations balance against them. Other decisions, some of them automated and some of them consequential, take them as given.
Between the model emitting a value and the system treating that value as true, something happened. In most production systems, what happened is one of four things:
- Nothing. The value was written because the pipeline had nowhere else to put it. This is more common than practitioners admit, particularly in systems that began as prototypes.
- A confidence threshold. The model reported a number, the number exceeded a constant, and the value was committed. Section 2 is about why this is weaker than it appears.
- A validation pass. The value satisfied a schema, a regular expression, a range check, or a cross-field constraint, and was committed on that basis. Section 24 is about the precise sense in which this is necessary and insufficient.
- Blanket human review. Every extracted value was checked by a person, which is safe, expensive, and self-defeating — it converts an automation project into a data-entry project with extra steps.
Each of these is an answer to a question nobody wrote down. The question is:
Under what conditions may a value derived by a probabilistic process from a document become the authoritative state of record — and when it may not, what should happen to it instead?
Stated that way, several things become visible that a threshold conceals.
It is a decision, so it has a decision-theoretic structure. There are actions, there are consequences that depend on an unobserved truth, there is a prior, and there is a resource constraint. Decisions of this shape have a normative solution, and it is not "compare a number to a constant."
It is a resource-allocation problem, because review is finite. An organization has some number of analyst-hours per day. Every value routed to a human consumes them. The interesting question is never "is this value certain enough" in isolation; it is "given that I can check forty things today, is this one of the forty." Those are different questions with different answers, and only the second one is operationally meaningful.
It has more than two outcomes. The implicit action set — commit or review — omits the action an experienced operator takes most often with a genuinely ambiguous document: neither. Hold it. Do not commit it, and do not spend a reviewer on it either, because the reviewer cannot resolve it any better than the machine can and the organization can proceed without the value. Abstention is not failure. It is frequently the optimal action, and a framework without it will systematically mis-price review.
The unit is wrong. Fields are not independent, and treating them as though they were is not a simplification but a category error. If a line item's amount is correct and its contract identifier is wrong, the amount has not been "mostly right." It has been attached to the wrong obligation, which is a worse outcome than if both fields had been left blank. Section 7 develops this; Section 16 quantifies it.
1.1 Why this gets missed
The gap is invisible for a structural reason: the two disciplines that would notice it are looking at different things.
The extraction literature optimizes a scoring function against gold labels. F1, exact-match accuracy, edit distance, field-level precision and recall. These are the right metrics for the question that literature asks, which is how good is the extractor. They are silent on the question this paper asks, which is which of these four thousand extracted values must a person look at. A system with an F1 of 0.94 has told you nothing about which six percent are wrong, or about which of the wrong ones matter.
The systems and governance literature — including this paper's companion — specifies controls over actions. Who may execute, under what authority, with what approval, leaving what evidence. It generally treats the data those actions operate on as an input rather than as a control surface. Paper 01 is explicit that agents must not acquire execution authority; it is silent on whether the state they reason over was itself admitted under any policy.
The result is a seam. Extraction hands values across it; governance picks them up on the other side and asks what may be done with them. Nobody owns the crossing.
1.2 What goes wrong at the seam, structurally
Four failure modes recur, and each is a symptom of an unspecified admission policy rather than of a weak extractor.
| Failure | Mechanism | Why a better extractor does not fix it |
|---|---|---|
| Silent commit of consequential error | A high-consequence value passes a generic threshold because the threshold is calibrated to average difficulty, not to this value's consequence. | Reducing the error rate reduces the frequency, not the exposure. A one-in-a-thousand error on a value that binds an entire record is an unbounded loss at any error rate above zero. |
| Review capacity spent on the wrong cases | Routing on uncertainty sends reviewers to the cases the model finds hard, which correlate poorly with the cases where review changes an outcome. | A better extractor shifts which cases are hard. It does not make hardness a proxy for review value. Corollary 4 shows these are formally distinct quantities. |
| Consistency mistaken for correctness | A cross-field validator passes, so the record is committed. But an OCR digit substitution can produce an internally consistent, externally false record. | Validation constrains the relation among extracted values. It has no access to the document's actual content, so it cannot detect an error that preserves the relation. |
| Correct payload, wrong binding | Every field extracted correctly except the identifier, so a correct set of values attaches to the wrong object. | Field-level accuracy improves and the loss does not, because loss is not additive over fields. Proposition 5 quantifies exactly how badly field-level ranking mis-prices this. |
Notice that all four survive an arbitrarily good extractor. That is the argument for treating admission as a separate layer with its own specification: it addresses a class of failure that extraction quality does not reach.
1.3 The scope of "consequential"
This framework earns its complexity only where the consequence of a wrong committed value is materially heterogeneous across values and materially larger than the cost of checking one. Both conditions matter.
If every field carries identical consequence, materiality-aware routing degenerates to uncertainty-aware routing and the framework collapses to Proposition 1 — a known and unremarkable special case. If the cost of review approaches the consequence of error, review is never worth performing and the correct policy is to commit everything and absorb the loss. Section 37 states these and seven other conditions under which something simpler should be built.
Where the conditions do hold — and they hold in most settings where documents drive money, obligations, care, or compliance — the difference between a specified admission policy and a threshold is not a refinement. It is the difference between a control and the appearance of one.
02Why a confidence threshold is not an admission policy
The default admission mechanism in production systems is a comparison of a model-reported confidence against a constant. Paper 01 rejected this mechanism for routing workflow transitions. The argument transfers to admission, and it is worth making in full here rather than by reference, because the failure has a different shape when the object is a value rather than a transition.
2.1 The measurement objection
A self-reported confidence from a generative model is a produced token sequence or a softmax statistic, not a calibrated posterior over the correctness of the extracted value. Four properties make it unsuitable as a control input:
- It is poorly calibrated, and asymmetrically so. Overconfidence concentrates in exactly the regime that matters: fluent, plausible output over incomplete or ambiguous evidence. The cases where the number is least trustworthy are the cases where a control is most needed.
- It is prompt-sensitive. A wording change intended as cosmetic shifts the score distribution, and therefore silently re-tunes the operating point of a production control.
- It is non-stationary across model versions. A vendor upgrade re-tunes the control without a change-management event. The threshold constant survives; its meaning does not.
- It is compressed. Scores cluster in a narrow high band, which makes threshold selection arbitrary and makes the difference between 0.94 and 0.96 carry more operational weight than it can bear.
These are not only analytical objections. Recent work on confidence estimation for document field extraction reports that token log-probabilities, verbalized confidence, and multi-sample self-consistency all collapse toward all-positive behaviour at the thresholds where an automation decision would actually be taken [27, 28] — that is, they separate poorly in precisely the operating region where a control needs them to separate well.
The objection is nonetheless only half the story. Suppose it were entirely solved. Suppose a perfectly calibrated posterior over field correctness, stable across versions and invariant to phrasing. The threshold policy would still be wrong, for a reason that has nothing to do with measurement.
2.2 The decision-theoretic objection
A threshold on error probability answers the question how likely is this to be wrong. That question is not the one an admission policy must answer. The relevant question is what do I gain by having a person look at it, and the two come apart in three separate ways.
They come apart on consequence. Two values with identical error probability can differ by orders of magnitude in what a wrong commit costs. A misread description field and a misread counterparty identifier are not the same event. A policy that ranks on probability alone is indifferent between them.
They come apart on the reviewer. A value can be very likely wrong and still not worth reviewing, because the human cannot determine the right answer either — the document is illegible, the source is genuinely ambiguous, the required context is not in the document at all. Review of such a case consumes capacity and returns a residual error close to the machine's. Corollary 4 makes this precise: review value is bounded above by consequence multiplied by the total-variation distance between the outcome distributions under commit and under review. When review does not move that distribution, review value approaches the negative of its cost regardless of how high the error probability is.
They come apart on the alternative. If a value can simply be held — not committed, not reviewed, flagged as unresolved — then the counterfactual to reviewing is not committing. It is holding. A high-error-probability value whose absence is cheap should be abstained on, not reviewed. Section 9 formalizes this and shows that ranking on the commit-versus-review gap is internally inconsistent with a three-action model.
Error probability, expected consequence, and value of intervention are three distinct quantities. Every threshold policy conflates them into one. The whole formal apparatus in Part II exists to keep them apart, because the cases where they diverge are precisely the cases where routing decisions are non-obvious — and therefore precisely the cases where a policy is doing work.
2.3 What a threshold is actually good for
Nothing in this argument says confidence scores are worthless. They are a legitimate coordinate of the feature vector from which the selection value is estimated. What they cannot be is the decision rule itself.
The operational recommendation, inherited from Paper 01 and unchanged: record self-reported confidence on every extraction from day one, route on nothing, and bin the scores against adjudicated outcomes until a reliability curve exists. Calibration data is cheap to collect prospectively and impossible to reconstruct retroactively. Once a curve exists, the score becomes a usable input to the estimator described in Section 29. It does not thereby become a policy.
2.4 The threshold that survives
There is one place where a threshold is exactly right, and locating it clarifies the rest. A deterministic gate — a hard constraint whose violation means the record is malformed rather than uncertain — is properly a threshold, because it is not measuring anything. A currency code that is not in ISO 4217 is not "probably wrong"; it is invalid. Proposition 3 in Section 17 shows how such gates sit inside the residual-risk model as a limiting case, which lets an implementation express hard gates and soft evidence in one parameterization without pretending they are the same kind of object.
03What this borrows, and what narrowly survives
An informed reader will recognize most of this paper's vocabulary within a few pages: risk and coverage, deferral, abstention cost, capacity constraints, selective prediction. That recognition is correct. Stating the inheritance precisely is not a weakness in the argument; withholding it until a reviewer discovers it is the weakness.
This section was written after a prior-art audit that removed two claims the earlier drafts of this work made. Both removals are recorded here rather than quietly dropped, because the record of what a framework stopped claiming is part of its evidence.
3.1 Selective prediction and the reject option
The idea that a predictor may decline to predict is roughly six decades old. Chow's formulation of the optimum error–reject tradeoff [1] established the decision-theoretic version: with a fixed rejection cost, the Bayes-optimal rule rejects when the posterior probability of the most likely class falls below a threshold determined by that cost. El-Yaniv and Wiener [2] developed the modern framework around the risk–coverage curve, deliberately avoiding explicit cost models on the grounds that rejection cost is hard to quantify in many settings, and characterizing instead the trade-off between the fraction of instances answered and the risk incurred on them. Geifman and El-Yaniv [3] brought the framework to deep networks, constructing a selective classifier over a trained model that guarantees a user-specified risk level with high probability; SelectiveNet [4] later trained prediction and rejection jointly.
Two points of contact with this paper deserve stating.
First, an inheritance correction. Early drafts of this work credited the risk–coverage framework to Geifman and El-Yaniv. That is wrong: the framework is El-Yaniv and Wiener, and the reject option is Chow. Geifman and El-Yaniv is the deep-network instantiation. The error is small and the kind that costs credibility cheaply.
Second, a genuine divergence. El-Yaniv and Wiener explicitly set aside cost models because rejection costs are hard to specify. This paper takes the opposite position: in an operational admission setting, costs are not merely specifiable but already specified — somewhere in the organization there is a policy about what a wrong committed value costs, even if it is implicit in an escalation matrix rather than written as a loss function. Making it explicit is part of the contribution. This is a difference of setting, not a claim of superiority; in domains where consequence genuinely cannot be elicited, the coverage-based framing remains the right one.
3.2 Learning to defer
Madras, Pitassi and Zemel [5] introduced deferral as a learning problem: train a rejector alongside a classifier so the system can pass instances to a downstream decision-maker. Mozannar and Sontag [6] gave the setting its theoretical foundation, reducing expert deferral to cost-sensitive learning and constructing a consistent convex surrogate that upper-bounds the combined system loss. The line has since developed substantially: one-versus-all consistent surrogates [7], post-hoc estimators over fixed pretrained models [8], exact algorithms [9], and multi-expert settings [10].
The post-hoc branch [8] is structurally closest to this paper's setting, because it assumes a fixed predictor and learns a deferral rule over its outputs — which is what an admission controller sitting downstream of a document extractor does.
3.3 Capacity-constrained deferral — a removed claim
Earlier drafts of this work claimed, as a contribution, the allocation of authority under finite review capacity. That claim does not survive contact with the literature and has been removed.
| Work | What it establishes |
|---|---|
| Alves et al., DeCCaF [11] | Names the same three gaps this framework names — cost-sensitive scenarios where error types carry different costs, restrictive requirements on expert-prediction data, and human work-capacity limits — and solves assignment with constraint programming that globally minimizes error cost subject to per-expert capacity, batched over operating periods. |
| De, Okati, Zarezade and Gomez-Rodriguez [12] | Selects which instances humans handle under an automation-level budget; proves the problem NP-hard, and for margin-based classifiers decomposes the objective into a monotone \(\gamma\)-weakly submodular part minus a modular part, admitting greedy algorithms with approximation guarantees. |
| De, Koley, Ganguly and Gomez-Rodriguez [13] | The regression counterpart, with the same budget structure and hardness result. |
| Okati, De and Gomez-Rodriguez [14] | Differentiable formulation of the triage problem. |
| Reid, Sühr, Vernade and Samadi [15] | Online deferral under explicit budget constraints as a contextual bandit with partial feedback. |
| Raghu et al. [16] | The triage framing: prediction, triage, and the allocation of human effort as a joint problem. |
Budgeted allocation of scarce human review is therefore a setting this framework shares with an active literature, not a contribution it makes. It is cited as such.
One technical distinction does survive, and it is worth stating precisely because it makes this paper's problem easier rather than harder. In De et al. [12], the assignment changes what the model learns — the classifier is retrained on the machine-handled subset — which makes the objective non-modular and forces the weak-submodularity machinery and approximation guarantees. In the admission setting the extractor is fixed and allocation is post-hoc. The objective is therefore modular, and top-B selection is exactly optimal rather than approximately so (Lemma 1, Section 11). An honest reading is that this framework operates in a more tractable corner of a harder problem. Borrowing submodular-optimization apparatus it does not need would be rhetorical weight, not rigor.
3.4 Structured and partial deferral — a second removed claim
The second removed claim concerns granularity. Deferring on part of a structured output rather than on the whole prediction is established.
Garcia, Essid, Clavel and d'Alché-Buc [17] formalize structured output learning with abstention, in which a model abstains on components of a structured output at a user-chosen cost, via a class of asymmetric abstention-aware losses with excess-risk guarantees. Rayan and Tewari [18] argue explicitly that rejecting an entire prediction is inappropriate for long structured outputs and construct rejectors that defer specific outputs within a sequence, showing that granular deferral improves the cost–accuracy trade-off relative to whole-output deferral. The question of what the right unit of abstention is has also been raised directly in dense-prediction settings, where the choice among pixel, patch and image is exactly the choice among field, group and document.
So: partial deferral is not novel, and this paper does not claim it. What remains unclaimed elsewhere, so far as the audit found, is the specific combination developed in Part II — deriving the deferral unit from a loss-coupling graph, defining the loss over that unit as non-additive by construction, and targeting the decision at admission to authoritative state rather than at a prediction. Section 3.6 states that surviving claim at its actual size.
3.5 The extraction substrate
Everything in Part III is prior art, adopted as engineering substrate. Grammar-constrained decoding, symbolic validation of extracted records, unsat-core-guided repair, sequence-alignment provenance, self-consistency sampling, and hash-chained audit records are all established techniques with their own literatures. They appear in this paper because they determine the quality of the inputs the decision layer consumes, and because each carries a boundary condition that is routinely dropped when the technique is deployed. Part III's contribution is the boundary conditions, not the techniques.
3.6 Concurrent work in document extraction specifically
The audit that produced Sections 3.3 and 3.4 searched the deferral and selective-prediction literatures. A second, narrower pass — for selective prediction and deferral applied specifically to document information extraction — was the one place a fatal collision could still have been hiding, since that is the exact intersection this paper occupies. It found no collision with the claim in Section 3.7, and it found that the intersection is considerably more crowded than the preceding sections imply. Both facts are recorded.
Per-field selective risk control. Recent work applies conformal risk control to document extraction at field granularity [26], establishing distribution-free risk guarantees over accepted fields and diagnosing the ways naive application of the machinery fails on documents. It converges independently on a point this paper makes in Section 35.1 — that document clustering breaks exchangeability, so field-level samples are not independent and marginal guarantees must be labeled as such. The setting is distinct from this paper's in two ways that matter: it controls a marginal risk level rather than allocating a finite review budget, and its unit is the field, so no non-additive loss arises. It is nonetheless the closest published neighbour on granularity, and Section 30.3's acceptance criteria should be read alongside it.
Confidence estimation for field-level deferral. A multi-signal confidence engine for LLM document extraction [27] reports that token log-probabilities, verbalized confidence, and multi-sample self-consistency each collapse toward all-positive behaviour at practical operating thresholds, and fuses cross-call disagreement, OCR quality, and layout signals instead. A companion benchmark [28] evaluates verbalized against log-probability confidence across models on nested line-item schemas.
These matter to this paper twice over. They supply empirical support for the argument of Section 2.1, which is otherwise made from first principles: the claim that self-reported confidence is unsuitable as a control input is no longer only an argument. And they supply a caution for Section 27, since self-consistency is reported there as an uncertainty coordinate and is reported in [27] as collapsing at exactly the thresholds where a control would operate. Section 27.2's insistence that the signal must be calibrated against adjudicated outcomes before entering the estimator should be read as the minimum, not as a formality.
Field interdependence, measured. Work on joint versus separate extraction of multi-field documents [29] quantifies dependency among numeric fields and shows that joint extraction outperforms separate extraction in proportion to the strength of that interdependence. This is not a deferral result, but it is the closest thing in the literature to independent evidence for the premise underlying this paper's decision unit: that fields in these schemas are not independent, and that treating them as though they were costs something measurable. It also demonstrates that the dependency structure is extractable from a public corpus, which is directly relevant to testing H2 (Appendix E.2).
Routing between extractors. Pre-inference routing selects among extraction models by predicted document difficulty [30]. The object is compute cost rather than review capacity, so it is a different problem — but its central finding rhymes with Section 37: routing helps only when the cheap option fails often enough and those failures are predictable from observable features, and it does not help otherwise. That is the same shape of scoping discipline this paper applies to itself, arrived at independently.
Deferral granularity outside documents. The question of the right unit of deferral is live in dense prediction as well, where confidence-aware deferral at sub-image granularity substantially outperforms global thresholding at the low review budgets that characterize real workflows [31]. The finding is consistent with this paper's Observation 11.1 without bearing on it directly, since the losses there remain additive.
It does not change the claim in Section 3.7. None of [26]–[31] derives a decision unit from a loss-coupling graph, admits a non-additive loss over that unit, or states anything resembling identity-key amplification — the quantity \(L - W^M\) does not exist in a field-additive formalism.
It does change the framing. Field-granular selective prediction in document extraction is an active area with 2026 publications, not an open space. A version of this paper that presented field-level abstention over extracted records as new would now be wrong, and the surviving claim narrows accordingly to the group formulation and its consequences.
3.7 The surviving claim, stated at its size
This paper does not introduce selective prediction, learning to defer, cost-sensitive abstention, capacity-constrained allocation, or partial deferral over structured outputs. Each is established, and each is cited.
The contribution is the instantiation of these at the boundary where document-derived values become authoritative operational state, under three structural commitments and their consequences:
(i) the decision unit is a dependency group derived from a loss-coupling graph, rather than a field or a whole document;
(ii) the loss over that unit is non-additive by construction, so identity-binding errors are representable;
(iii) the action set is commit / review / abstain over state, not accept / reject over a prediction.
The results that follow from those commitments — exact top-B optimality via modularity, the tight selection-regret bound, the total-variation bound on review value, and identity-key amplification — are derived in Part II and are the paper's technical content. The evaluation contract in Section 35 is offered as a methodological contribution in its own right.
That is a narrower claim than the framing of Section 1 might suggest. It is stated narrowly on purpose. A reader who knows the deferral literature will accept it; the same reader would have discounted the entire paper on encountering the wider version.
04Three quantities that are routinely conflated
Before any formalism, the conceptual core of the framework can be stated in a sentence: error probability, expected consequence, and value of intervention are three different numbers, and admission policy depends on the third. Systems fail at this boundary largely because they compute the first and act as though they had computed the third.
4.1 The three quantities
| Quantity | Question it answers | What it is sensitive to | What it is blind to |
|---|---|---|---|
| Error probability | How likely is this candidate to be wrong? | Model uncertainty, document quality, layout novelty, OCR noise | What a wrong value costs; whether anyone can fix it |
| Expected consequence | What does it cost if this is wrong and I commit it? | Materiality, dependency structure, downstream propagation, regulatory exposure | Whether review would change the outcome |
| Value of intervention | What do I gain by spending review capacity here, net of cost? | All of the above, plus reviewer capability on this error class, plus the best alternative action | Nothing relevant — this is the quantity policy should rank on |
Each pair comes apart in a way that has an operational instance.
4.2 High probability, low consequence
A free-text description field on a line item is misread at some meaningful rate — layout variance, abbreviation, handwriting. The error probability is the highest in the record. The consequence of an error is close to zero: no downstream calculation reads it, no obligation depends on it, and a subsequent human encountering the record can see the source. A probability-ranked policy sends reviewers to this field constantly. A consequence-aware policy never does.
4.3 Low probability, high consequence
A counterparty identifier is printed clearly and extracted correctly nearly always. Its error probability is the lowest in the record. But an error binds every other value in the record to the wrong object, and the resulting loss is not the loss of one field — it is the loss of the entire record plus whatever downstream processes acted on the mis-bound data before anyone noticed. A probability-ranked policy almost never reviews it. This is the case that motivates the entire group formulation, and Proposition 5 shows the mis-pricing is not a constant factor but a ratio that grows with the binding structure.
4.4 High probability, high consequence, low review value
This is the case practitioners find least intuitive and the one that most clearly separates the third quantity from the first two. A value is probably wrong, and a wrong value is expensive — but the document itself is ambiguous. The figure is obscured, the source contradicts itself, or the required disambiguating context is simply not present in the document. A human reviewer opening this case reaches the same impasse as the machine, spends fifteen minutes doing so, and either guesses or escalates.
Review value here is close to negative: the organization spends capacity and receives almost no reduction in expected loss. The correct action is abstention — hold the value, flag it as unresolved, and either obtain a better source document or proceed without it. Corollary 4 in Section 15 states this formally: review value is bounded above by the maximum consequence times the total-variation distance between outcome distributions under commit and under review, minus the review cost. When review does not move the distribution, the bound is negative and no error probability can rescue it.
4.5 Low probability, low consequence, high review value
Less common but worth stating for completeness, because it shows the ordering is not a lexicographic hierarchy. Suppose a value is usually right and individually unimportant, but it is the key on which a group's dependency structure hangs, and a reviewer can verify it in seconds with certainty. Cheap review, large distributional shift, and the consequence enters through the group rather than the field. The group-level treatment picks this up automatically; the field-level treatment cannot see it, because at field level both the probability and the weight are small.
4.6 Why the conflation is so persistent
Three reasons, all structural rather than careless.
Only the first quantity is produced by the model. Error probability, or a proxy for it, comes free with the extractor. Consequence requires a policy elicitation. Review value requires knowing something about reviewers. The cheapest quantity is the one that gets used.
The metrics reinforce it. A literature that reports F1 rewards work on the first quantity and has no place to report the third. An organization that adopts those metrics inherits the conflation with them.
The three coincide in the degenerate case. If all consequences are equal, review is perfect, and abstention is unavailable, then ranking by error probability is optimal. That is exactly Proposition 1 in Section 18, and it is why threshold policies appear to work in demonstrations: demonstrations are usually built on uniform-consequence data with an implicit oracle reviewer. The policy fails when it meets a real consequence distribution, which is heavy-tailed.
05The epistemic boundary: fidelity, authenticity, truth
Any framework that claims to govern the admission of information must be explicit about what kind of correctness it is capable of establishing. Overclaiming here is the most common way that document-processing systems mislead their operators, and it is usually done by omission rather than assertion — a system reports "validated" and lets the reader supply a meaning the system cannot support.
Three distinct questions are involved, and they are frequently collapsed into one.
| Layer | Question | What this framework claims | Mechanism and limit |
|---|---|---|---|
| L1 Document fidelity | Does the extracted value correctly represent what the document says? | Claimed. This is the property the framework is designed to govern. | Evidence binding to source spans, deterministic invariants, residual-risk estimation, human review of routed groups. Limit: probabilistic, with the residual explicitly estimated rather than assumed away. |
| L2 Source authenticity | Is the source document genuine and unaltered since receipt? | Surfaced, not solved. The framework makes tampering detectable and makes provenance visible to a reviewer. | Immutable source bytes, content hashing, hash-chained receipts (Section 28). Limit: detects alteration after receipt; cannot establish that the document was genuine when it arrived. |
| L3 External truth | Is what the document says true about the world? | Explicitly disclaimed. Out of scope, permanently. | No mechanism. A document is a claim by its author; nothing in an extraction pipeline can adjudicate that claim. |
A perfectly grounded extraction from a forged or factually false document is L1-correct and L3-silent. Every value was read accurately; every invariant held; the record is wrong about the world. This framework claims L1, surfaces L2, and disclaims L3 — and any deployment that reports its output as "verified" without that qualification is making a claim its architecture cannot support.
5.1 Why the distinction is load-bearing rather than pedantic
It determines what the residual-risk estimator is estimating. If \(r_M\) is defined as the probability that a value misrepresents the document, it is estimable from adjudicated extraction outcomes — a competent annotator with the document can settle it. If it were defined as the probability that a value misrepresents reality, it would require ground truth about the world, which no annotation process produces and no benchmark contains. Conflating the two produces an estimator whose target does not exist.
It also determines the correct scope of human review. A reviewer routed a group under this framework is being asked an L1 question: does the document say this? That question is answerable in seconds to minutes and has a determinate answer. If the reviewer is implicitly asked an L3 question — is this figure right? — the task becomes unbounded, review time becomes unpredictable, and the measured residual error r_H becomes a measurement of a different quantity than the one the model is charged with. Section 35.4 makes the reviewer's question explicit for this reason.
5.2 The L2 gap, stated honestly
Cryptographic receipts (Section 28) prove that a record has not been altered since it was committed. They prove nothing about the document's provenance before it entered the system. A supplier who submits a fabricated invoice, a claimant who alters a statement before sending it, or an upstream system that emits corrupted output all produce documents that this framework will process faithfully and commit confidently.
This is not a defect to be engineered away inside the admission layer. It is a different control problem — source authentication, supplier verification, out-of-band reconciliation — and it belongs to a different layer of the architecture. What the admission layer owes is visibility: a reviewer looking at a routed group should be able to see which source each value came from and what its trust class is, so that a decisive figure originating in an unverified third-party document is recognizable as such. Paper 01's trust classification carries over unchanged.
5.3 What consistency establishes
The single most consequential confusion in this area is between internal consistency and correctness, and it deserves its own statement because Part III depends on it.
A deterministic validator establishes that a set of extracted values satisfies a relation: totals reconcile, dates order correctly, identifiers match a format, percentages sum to one hundred. The validator has access only to the extracted values, not to the document. Consequently, any error that preserves the relation is invisible to it.
The canonical instance: an OCR process reads a digit 3 as an 8 in a subtotal, and the same substitution occurs consistently in the total, because both were rendered in the same degraded font on the same scan. Every arithmetic invariant passes. The record is internally coherent and externally false. No amount of solver expressiveness detects it, because the solver is checking the wrong thing.
The operational consequence, developed in Sections 24 and 35, is a reporting rule: validation pass rate is reported as consistency yield and never as accuracy. A field that satisfies every invariant can still be wrong, and the rate at which that happens must be measured on adjudicated data rather than defined away. In the formalism, this appears as the requirement that passing validation reduces \(r_M\) rather than setting it to zero.
06Operational state and the admission boundary
The framework needs one more piece of vocabulary before the formalism, and it is the piece that determines where the boundary sits: what counts as operational state.
6.1 Definition
A value is authoritative operational state when a process other than the one that produced it may read it and act on it without independently re-deriving it.
The definition is behavioral rather than architectural, and deliberately so. It does not matter whether the value sits in a table, a message, a cache, or a file. What matters is whether something downstream treats it as settled. A value in a staging table that no other process reads is not operational state. The same value, once a reconciliation job queries it, is.
Two corollaries follow that are useful in practice.
Admission is a property of the read, not the write. Systems that "write everything and filter later" have not avoided admission; they have relocated it to an unspecified consumer, usually a query that has no idea which rows were confident. This is the most common way an organization believes it has an admission policy when it does not.
There can be more than one admission boundary. A value may be admissible for reporting and inadmissible for payment. Section 32 treats this as a policy surface with per-consumer thresholds rather than as a single global gate, which is closer to how organizations actually operate and avoids forcing the strictest consumer's standard onto every use.
6.2 The three states of a candidate value
| State | Meaning | Who may read it | How it leaves this state |
|---|---|---|---|
| Candidate | Produced by the extraction substrate, bound to evidence, scored. Not authoritative. | The admission controller only. | Admission decision: commit, review, or abstain. |
| Committed | Admitted as authoritative operational state under a recorded policy. | Any authorized downstream consumer. | Correction through the normal change path for state of record — never silently. |
| Held | Not admitted. Explicitly unresolved, with a recorded reason. | Humans and processes that handle exceptions. Not the ordinary consumer path. | New evidence, a better source document, a policy change, or expiry. |
The held state is the one most systems lack, and its absence is what forces the false binary. Without it, an ambiguous value must either be committed — polluting the state of record — or routed to a reviewer who cannot resolve it, consuming capacity for nothing. With it, the system has a place to put values that are genuinely undetermined, and the policy can price that placement explicitly through the abstention cost \(\alpha_k\).
Note that held is not an error state and is not a queue of failures. A well-tuned admission policy holds a nonzero fraction of groups permanently, and that fraction is a legitimate operating parameter rather than a defect. Section 34 lists "driving the hold rate to zero" among the anti-patterns, because a system that never abstains has either a perfect extractor or an unexamined commit policy, and one of those is much more common than the other.
6.3 The boundary in the architecture
Plate 1 places the boundary relative to the components around it. Three features carry the meaning.
The extraction substrate has no path to the system of record. Its only output is a candidate group with attached evidence, scores, and check results, handed to the admission controller. This mirrors Paper 01's privilege split, where reasoning components hand a typed plan object to a gateway and never touch execution directly. The argument is the same in both papers: a component that produces probabilistic output should not also hold the authority to make that output binding.
The controller authorizes on state and policy, never on model-supplied text. Its inputs are the estimated selection value, the consequence model, the capacity ledger, and the policy — all computable quantities. A rationale string emitted by the extractor is display content for a reviewer, not an input to the decision. If it were an input, it would be an attack surface, for exactly the reasons Paper 01 develops around indirect prompt injection.
Review capacity is a metered resource with a ledger, not an unbounded escape hatch. The controller cannot route more work than capacity permits, which is what makes the allocation problem in Part II real rather than notional. A design in which "route to human" is always available is a design in which the hard question has been deferred to whoever is holding the queue.
6.4 What the boundary does not do
Three explicit non-responsibilities, stated because each has been assumed of systems in this shape:
- It does not improve extraction. The controller consumes candidates; it does not produce better ones. Everything in Part III that reduces error does so before the boundary. A framework that improves routing on a bad extractor still routes on a bad extractor.
- It does not adjudicate truth. Per Section 5, it governs L1 and disclaims L3. A committed value is a value the organization has decided to treat as the document's content, at a stated and estimated residual risk.
- It does not replace the system of record's own controls. Referential integrity, transaction boundaries, retention, and access control remain where they were. Admission decides whether a value may enter; it does not take over the store it enters.
The decision problem stated precisely, the assumptions it requires, and the results that follow. All mathematics is elementary and self-contained. Every result names the assumptions it uses; Appendix B gives the proofs in full.
07Setup: fields, loss coupling, dependency groups
7.1 Fields and records
A document instance produces a record over a schema of fields \(F = \{f_1,\dots,f_n\}\). For each field there is an unobserved gold value and a machine-produced candidate. The framework never observes the gold value at decision time; it observes only features.
Write \(Y\) for the gold record and \(\hat{Y}^M\) for the machine candidate record. Both are elements of the schema's value space. A human review of some portion of the record produces \(\hat{Y}^H\), which is not assumed equal to \(Y\) — the reviewer is a fallible process with its own error distribution, and treating it otherwise is the single most consequential modeling error available in this setting.
7.2 The loss-coupling graph
The central structural object is a graph over fields. Its edges are defined by consequence, not by schema syntax.
Fields \(f_i\) and \(f_j\) are loss-coupled, written \(f_i \sim f_j\), if the operational consequence of an error in \(f_i\) depends on the value or correctness of \(f_j\). The loss-coupling graph \(\mathcal{G}=(F,E)\) has an edge for each such pair.
This definition does real work later, so it is worth dwelling on why it is stated in terms of consequence rather than in terms of the more obvious alternatives.
Not statistical dependence. Two fields may have highly correlated errors — both misread because the same region of the scan is degraded — without being loss-coupled. Correlated errors matter for estimating risk; they do not change the structure of the loss. Conversely, two fields may have independent errors and be tightly loss-coupled: an identifier and an amount can fail for entirely unrelated reasons while the consequence of the amount error depends entirely on whether the identifier is right.
Not schema reference. A foreign key relationship in a schema is evidence of possible coupling but is neither necessary nor sufficient. Many schema references are inert with respect to consequence; many real couplings — an effective date that determines which rate table applies to an amount — are not expressed as references at all.
Consequence, because that is what the theory needs. Section 10 shows that the only separability assumption the results require is that total loss adds across components of this graph. Defining edges by loss coupling makes that assumption true by construction rather than by hope. It converts an assumption into a definition, and moves the remaining risk into a single named quantity — graph misspecification, treated in Section 21.
Constructing the graph in practice
The graph is elicited, not learned. A workable procedure, offered as an engineering recipe rather than a result:
- Seed from schema. Start with arithmetic relations, foreign keys, and validity constraints. These are cheap and mostly correct.
- Add binding edges. For every field that identifies an object to which other fields attach, connect it to everything that attaches. Identifiers, effective dates that select a regime, currency codes, and unit-of-measure fields are the usual instances.
- Interrogate with the counterfactual. For each candidate pair, ask a domain expert: if field \(i\) were wrong, would you care differently depending on whether field \(j\) were right? A yes is an edge.
- Test for over-merging. Because components are connected sets, a single spurious edge can merge two groups that should be separate, which coarsens the decision unit and wastes review capacity on unaffected fields. Prefer to omit a doubtful edge and record it as suspected coupling — Section 21 shows that omission carries a bounded, estimable penalty, whereas over-merging silently degrades the allocation.
7.3 Dependency groups
Let \(G_1,\dots,G_m\) be the connected components of \(\mathcal{G}\). These are the dependency groups, and they are the atoms of decision throughout this paper. Every field belongs to exactly one group; groups are disjoint and exhaust \(F\).
For each group \(k\), define:
| \(S_k\) | Feature vector observed for group \(k\) at decision time — check results, alignment scores, sampling divergence, layout descriptors, model statistics, and any other computable signal. Section 29 specifies its coordinates. |
| \(Y_k\) | Gold state of the group. Unobserved. |
| \(\hat{Y}^M_k\) | Machine candidate for the group. |
| \(\hat{Y}^H_k\) | State after human review of the group. A random variable, not equal to \(Y_k\). |
| \(\ell_k(\hat{Y}_k, Y_k)\) | Loss from committing state \(\hat{Y}_k\) when the truth is \(Y_k\). Non-negative. Not assumed additive over fields. |
| \(a_k\) | Action taken on group \(k\), an element of \(\{\textsf{COMMIT}, \textsf{REVIEW}, \textsf{ABSTAIN}\}\). |
7.4 Why the loss must be allowed to be non-additive
This is the modeling commitment on which most of Part II depends, so it is argued rather than asserted.
Suppose loss were additive: \(\ell_k(\hat{Y}_k,Y_k) = \sum_{i \in G_k} w_i \mathbf{1}[\hat{Y}_i \neq Y_i]\), with each field carrying a fixed weight. Then the loss of a record in which the identifier is wrong and everything else is right equals \(w_{\kappa}\), the identifier's own weight. But the actual operational event is that a complete, correct set of values has been attached to the wrong object. Downstream, this is worse than a record with several individually wrong fields, because a record with wrong fields is often visibly anomalous, whereas a mis-bound record is internally coherent and therefore propagates silently.
To represent that, the loss of the joint event must exceed the sum of the losses of its parts. Additive loss cannot do this at any setting of the weights: raising \(w_{\kappa}\) to reflect binding consequence makes the identifier expensive even when it is the only thing wrong in an otherwise-discarded record, and still fails to make the joint event superadditive.
Non-additivity is therefore not a generalization for its own sake. It is the minimum expressive power required to state the loss that motivates the framework. Section 16 shows what follows once it is admitted.
7.5 Three worked schema fragments
Groups are easiest to see in instances. The following are illustrative and interchangeable; none is privileged, and the framework is indifferent among domains.
| Setting | Fields | Coupling | Resulting groups |
|---|---|---|---|
| Transaction schedule | Counterparty ID, reference, effective date, currency, gross amount, net amount, rate | Amounts reconcile through rate; currency scales all amounts; counterparty and reference bind the row to an obligation | One group containing the identifier, the date, the currency and all amounts. The free-text description is a singleton. |
| Clinical order | Patient ID, encounter ID, order code, quantity, route, frequency, ordering clinician | Patient and encounter bind the order; code, quantity, route and frequency are jointly interpretable and individually meaningless | One group containing identifiers and the full order specification. Clinician is coupled if it determines authority, otherwise a singleton. |
| Contract clause set | Party A, Party B, effective date, term, jurisdiction, obligation text, cap amount | Parties bind every obligation; jurisdiction determines interpretation of the term; cap scales with obligation scope | One group of parties, dates and jurisdiction; obligation-and-cap forms a second group coupled to the first through the parties. |
Two patterns recur across all three, and they generalize:
Identifier fields pull everything into their component. Because an identifier binds every attached value, it is loss-coupled to all of them, and the component therefore contains the whole record fragment. This is correct and is the source of Proposition 5.
Descriptive fields tend to be singletons. Fields that nothing depends on form their own groups, and their decisions decouple entirely. This is also correct: they should be cheap to commit and should never consume review capacity.
7.6 Group size and the granularity trade-off
Groups are determined by the graph, but the graph is elicited, and elicitation has a knob. It is worth being explicit about what that knob trades.
| Granularity | Effect on the decision problem | Failure it produces |
|---|---|---|
| Too fine (edges omitted) | Loss is treated as separable where it is not. Assumption A1 fails; results acquire the misspecification term \(\kappa\) of Section 21. | Binding errors are under-priced. Reviewers are sent to high-probability, low-consequence fields. This is the failure mode of field-level routing. |
| Correct | A1 holds by construction. Top-\(B\) is exactly optimal. Non-additivity is fully absorbed into \(C_k(\cdot)\). | — |
| Too coarse (spurious edges) | Components merge. A single review action covers more fields than necessary, and \(c_k\) rises with group size. | Review capacity is wasted on fields that did not need it. In the limit — one group per document — the framework degenerates to whole-document deferral and loses its advantage over prior art. |
The asymmetry matters. Under-merging costs a bounded, estimable term; over-merging costs allocation efficiency silently, with no diagnostic that fires. The recommended default is therefore to omit doubtful edges and record them, then estimate \(\kappa\) from audit as described in Section 21.
08The action set and the cost model
8.1 Three actions
For each group the controller selects one action.
| Action | Semantics | Consumes review capacity? | Resulting state |
|---|---|---|---|
| \(\textsf{COMMIT}\) | Admit the machine candidate as authoritative operational state. | No | Committed |
| \(\textsf{REVIEW}\) | Route the group to a human, then admit the post-review state. | Yes | Committed (or held; see Section 20) |
| \(\textsf{ABSTAIN}\) | Do not admit. Record the group as unresolved with a reason. | No | Held |
The inclusion of abstention is not cosmetic. It changes the arithmetic of the whole problem, because it changes what reviewing is being compared against — see Section 9.
8.2 Conditional costs
Each action has an expected cost conditional on the observed features.
where \(c_k\) is the cost of reviewing group \(k\), \(H\) denotes the reviewing process, and \(\alpha_k\) is the policy-specified cost of holding the group unresolved.
Three features of this specification are deliberate and are frequently got wrong.
Review is imperfect
Equation (8.2) carries a residual loss term \(\mathbb{E}[\ell_k(\hat{Y}^H_k, Y_k)\,|\,S_k,H]\). Review does not set the loss to zero. Any formulation in which \(C_k(\textsf{REVIEW}) = c_k\) has assumed an oracle reviewer, and every downstream conclusion inherits that assumption — most damagingly, it makes review value a function of machine error alone, which is exactly the conflation Section 4 warns against.
The residual is written conditional on \(S_k\) because reviewer performance is not constant across error classes. A reviewer resolves a transposed digit in a clearly printed field almost always and resolves a genuinely ambiguous handwritten annotation almost never. Treating \(r_H\) as a scalar constant discards exactly the structure that makes routing decisions non-trivial.
Abstention has a cost, and it is a policy input
\(\alpha_k\) is set, not measured. It encodes what the organization loses by not having the value: a delayed settlement, an incomplete report, a manual downstream workaround, a customer contact. Some groups have low abstention cost and some have effectively infinite abstention cost — a mandatory regulatory field cannot be held indefinitely. Setting \(\alpha_k = \infty\) recovers a two-action model as a special case, which is the right way to express "this must be resolved."
Review cost is a measured resource, not a count
\(c_k\) is the loss-denominated cost of review, and it is distinct from \(c^{\mathrm{res}}_k\), the amount of the capacity resource that review consumes. In the simplest setting these coincide up to scale and every group consumes one unit. In reality review time varies substantially with group size and difficulty, which turns the allocation problem into a knapsack — Section 19.
8.3 The unconstrained optimum
Absent a capacity constraint, the optimal action per group is immediate:
This is the classical three-way decision under a known loss, and it is not the interesting case. It is worth writing down only to make the point that the interesting content of the problem is entirely in the constraint: an organization that could review everything worth reviewing would not need an allocation theory. Plate 3 shows the partition of feature space that (8.4) induces, and where the capacity constraint deforms it.
8.4 The constrained problem
With capacity \(B\) and per-group resource consumption \(c^{\mathrm{res}}_k\), the problem is
Everything in Sections 9 through 21 concerns the structure of this problem: what makes it tractable, what the optimal policy looks like, how much is lost when the quantities are estimated rather than known, and what happens when its assumptions are relaxed.
8.5 Observability of each quantity
A formalism whose variables cannot be obtained is ornamental. The following table states, for each quantity, exactly how it is to be obtained. This table is the framework's answer to the charge of ornamental mathematics, and every row is either an estimator, a policy elicitation, or an explicit admission that the value is unfilled.
| Quantity | Kind | How it is obtained |
|---|---|---|
| \(\mathcal{G}\), \(G_k\) | Elicited structure | Schema seed plus expert counterfactual interrogation (Section 7.2). Recorded as a versioned artifact; changes are governed events. |
| \(\ell_k\) | Policy-specified | Consequence model elicited from the accountable owner and expressed over outcome classes (Section 32.1). Parameterized transparently; sensitivity reported. |
| \(r_M\) | Estimable UNFILLED | Held-out adjudicated extraction outcomes. Requires L1 annotation only (Section 5.1), which is tractable. |
| \(r_H\) | Declared parameter UNFILLED | Reviewer study with independent adjudication, or — as adopted here — declared a policy parameter with a reported sensitivity surface. Never assumed zero, never borrowed. Section 35.6, Appendix E, Appendix F. |
| \(c_k\) | Measurable | Instrumented review time multiplied by a loaded rate. Available from any review tool with timestamps. |
| \(c^{\mathrm{res}}_k\) | Measurable | Same instrumentation, expressed in capacity units rather than currency. |
| \(\alpha_k\) | Policy-specified | Elicited per group class. \(\infty\) for mandatory fields. |
| \(B\) | Policy-specified | Staffing decision. Known exactly. |
| \(\theta_j\) | Fit or policy-set | Logistic fit of check outcomes against adjudicated correctness, or set to the hard-gate limit by policy (Section 17). |
| \(\kappa\) | Estimable UNFILLED | Audit of committed records for consequences spanning groups (Section 21.3). |
| \(v_k, \hat v_k\) | Derived | Computed from the above via (9.2). The estimator is specified in Section 29. |
09Selection value, and why it is not the commit gap
The quantity a capacity-constrained policy must rank on is the value of spending review capacity on a group. Getting its definition right is the difference between a coherent policy and one that fights itself, and the natural first definition is wrong.
9.1 The natural definition, and its defect
The obvious candidate is the reduction in expected loss that review produces relative to committing:
This is the quantity most naturally written down, and it is what earlier versions of this framework ranked on. In a two-action world it is correct. In a three-action world it is not, and the failure is not subtle.
The counterfactual to reviewing group \(k\) is not committing it. It is taking the best action available without spending review capacity, and when abstention is permitted that may be holding.
Consider a group where the candidate is very likely wrong, so \(C_k(\textsf{COMMIT})\) is large, but the value is discretionary and can be held cheaply, so \(\alpha_k\) is small. Then \(\Delta_k\) is large — review looks extremely valuable — while the truth is that the organization can simply not commit the value at zero capacity cost and lose almost nothing. Ranking on \(\Delta_k\) sends a reviewer to a group where the best action was already available for free.
Ranking on \(\Delta_k\) is internally inconsistent with a three-action model. It over-prices review for exactly those groups whose abstention cost is low, which is a systematic bias rather than noise: the groups it mis-ranks are a structured subset — discretionary, non-mandatory, low-downstream-dependency — and a policy that ranks on \(\Delta_k\) will spend a predictable fraction of its capacity on them permanently.
9.2 The correct definition
The no-review baseline and the selection value of group \(k\) are
\(b_k\) is what the group costs if no capacity is spent on it; \(v_k\) is what spending capacity buys, net of the review cost already embedded in \(C_k(\textsf{REVIEW})\).
The relationship to (9.1) is exact: \(v_k = \min\{\Delta_k,\ \alpha_k - C_k(\textsf{REVIEW})\}\), and in particular \(v_k = \Delta_k\) precisely when \(C_k(\textsf{COMMIT}) \le \alpha_k\), that is, when committing is already better than holding. So \(\Delta_k\) survives as the special case \(\alpha_k = \infty\), which is the mandatory-field regime.
9.3 Reading the three regimes
| Condition | Baseline is | Selection value | Interpretation |
|---|---|---|---|
| \(C_k(\textsf{C}) \le \alpha_k\) and \(v_k>0\) | Commit | \(v_k=\Delta_k>0\) | Review is worth capacity: it improves on an already-acceptable commit. |
| \(C_k(\textsf{C}) \le \alpha_k\) and \(v_k\le 0\) | Commit | \(v_k\le 0\) | Commit outright. Review costs more than it returns. |
| \(\alpha_k < C_k(\textsf{C})\) and \(v_k>0\) | Abstain | \(v_k = \alpha_k - C_k(\textsf{R})\) | Review is worth capacity, but it is competing against holding, not against committing. Its value is smaller than \(\Delta_k\) suggests. |
| \(\alpha_k < C_k(\textsf{C})\) and \(v_k\le 0\) | Abstain | \(v_k\le 0\) | Hold it. Spending a reviewer here is worse than simply not having the value — the case Section 4.4 describes. |
The third row is the one that repays attention. It describes groups that a \(\Delta_k\)-ranked policy will chase and a \(v_k\)-ranked policy will correctly de-prioritize, and it is common: discretionary enrichment fields on hard documents have precisely this profile.
9.4 Positivity and the candidate set
Only groups with \(v_k > 0\) are candidates for review. A group with \(v_k \le 0\) should never be reviewed at any capacity, because reviewing it increases expected total loss. Define the candidate set
If \(m^{+} \le B\), the capacity constraint is slack, every worthwhile review is performed, and the allocation problem disappears. That this can happen is worth noting, because it identifies a regime in which the framework's central machinery is unnecessary: a well-instrumented, low-consequence, high-quality pipeline with generous staffing does not need an allocation theory. It needs (8.4). The framework earns its complexity when \(m^{+} \gg B\), which is the operating regime of essentially every document process at scale.
10Assumptions, stated and discharged
Four assumptions are used in Sections 11 and 12. Each is stated here, and its status — construction, technical convenience, or genuine restriction — is given plainly. An earlier version of this framework conflated the first two, which produced a real gap in an earlier result; the resolution is recorded because it is instructive.
10.1 The assumptions
Total operational loss decomposes across groups:
\[ L\!\left(\hat{Y},Y\right) \;=\; \sum_{k=1}^{m} \ell_k\!\left(\hat{Y}_k, Y_k\right). \]Status: true by construction under Definition 7.1, subject to graph misspecification (Section 21).
Loss within a group decomposes over fields:
\[ \ell_k\!\left(\hat{Y}_k,Y_k\right) \;=\; \sum_{i \in G_k} w_i \,\mathbf{1}\!\left[\hat{Y}_i \neq Y_i\right]. \]Status: explicitly rejected as the primary regime. Retained only to derive corollaries that recover known special cases.
\(c^{\mathrm{res}}_k = 1\) for all \(k\); capacity \(B\) is an integer count of groups.
Status: a genuine restriction, relaxed in Section 19, where the problem becomes a knapsack and the regret result does not transfer.
Both the optimal and the selected review sets contain exactly \(B\) groups, with \(B \le m\). Where fewer than \(B\) groups have positive selection value, pad both sets with zero-value groups.
Status: technical, without loss of generality. The padding convention makes the symmetric-difference argument in Theorem 1 clean; it changes no conclusion.
10.2 The A1 / A2 distinction, and why it was the whole gap
An earlier statement of the selection-regret result flagged an unresolved assumption, described roughly as "top-\(B\) selection presumes separability of group review values even though losses are non-additive within a group." That description conflates two different properties, and the conflation is why the gap looked unresolvable.
Separating them dissolves it.
Top-\(B\) selection requires A1 only. It does not require A2, and this framework rejects A2.
The reason is structural rather than technical. The action is taken at group granularity. Whatever coupling exists among the fields inside \(G_k\) is entirely absorbed into the scalars \(C_k(\textsf{COMMIT})\), \(C_k(\textsf{REVIEW})\), \(C_k(\textsf{ABSTAIN})\) before any selection occurs. By the time the allocation problem sees group \(k\), it sees three numbers. Non-additivity inside a group is invisible to selection.
Non-additivity across groups would not be invisible — it would break the decomposition in Lemma 1 outright. That is exactly what A1 rules out, and Definition 7.1 is engineered so that A1 holds by construction: any coupling that would violate it is, by definition, an edge, and therefore lies inside a component rather than across components.
This is a small observation with a large consequence: it means the grouping is not a modeling convenience laid on top of the decision problem, it is what makes the decision problem tractable. Section 11.4 states the asymmetry that follows.
11Lemma 1: exact optimality of top-B
11.1 Statement
Assume A1, A3, A4. The policy that
(i) reviews the \(B\) groups with the largest selection value \(v_k\) among those with \(v_k > 0\), and
(ii) for every non-reviewed group takes \(\arg\min\{C_k(\textsf{COMMIT}),\ \alpha_k\}\),
minimizes total expected loss subject to the capacity constraint.
11.2 Proof
Fix a review set \(S\) with \(|S| \le B\). Groups outside \(S\) are unconstrained and, by A1, contribute additively and independently to the objective; each is therefore optimally assigned its baseline cost \(b_k\). Total expected cost is
\[ J(S) \;=\; \sum_{k \in S} C_k(\textsf{REVIEW}) \;+\; \sum_{k \notin S} b_k \;=\; \sum_{k=1}^{m} b_k \;-\; \sum_{k \in S} v_k , \]using \(v_k = b_k - C_k(\textsf{REVIEW})\) from (9.2). The first term does not depend on \(S\). Minimizing \(J\) is therefore equivalent to maximizing the set function \(V(S) = \sum_{k \in S} v_k\) subject to \(|S| \le B\).
\(V\) is modular: it is a sum of per-element terms with no interaction. The feasible sets form a uniform matroid. For a modular objective over a uniform matroid, greedy selection is exact. Concretely, by exchange: suppose \(S\) is feasible, \(j \in S\), \(k \notin S\), and \(v_k > v_j\). Then \(S' = (S \setminus \{j\}) \cup \{k\}\) is feasible and \(V(S') - V(S) = v_k - v_j > 0\), so \(S\) is not optimal. Hence no set other than a top-\(B\) set can be optimal. Groups with \(v_k \le 0\) are excluded because including one weakly decreases \(V\) while consuming capacity that a positive-value group could use. □
11.3 What is and is not being claimed
Lemma 1 is not a difficult result and is not presented as one. Sorting by value and taking the top \(B\) is the obvious algorithm, and the lemma confirms it is exactly right under its assumptions. The content is entirely in what makes those assumptions hold, and in the two observations that follow.
11.4 Modular, not submodular — and why that is a concession
The objective here is modular, not merely submodular. This is a stronger and more convenient property, and it is worth being explicit that it makes this paper's problem easier than the problems solved in the closest prior work.
In De et al. [12], the set of instances assigned to humans changes what the model learns, because the classifier is retrained on the machine-handled subset. The objective is consequently non-modular; the paper proves NP-hardness and obtains guarantees through a decomposition into a monotone \(\gamma\)-weakly submodular function minus a modular one, solved by distorted greedy with an approximation ratio.
None of that machinery is needed here, because the extractor is fixed and allocation is post-hoc: reviewing group \(k\) does not change \(v_j\) for any \(j \neq k\). The correct statement is therefore that this framework operates in a more tractable corner of a harder problem and obtains an exact solution rather than an approximation.
It would be easy, and would look more impressive, to import submodular-maximization apparatus into this section. It would also be dishonest, because the objective does not require it. Where a problem is easy, saying so is part of the specification: a reader who knows the harder literature should be able to see immediately which assumptions bought the simplification, and reverse them if their setting differs.
11.5 The asymmetry that grouping buys
The second observation is the one that makes the group formulation load-bearing rather than decorative.
Under field-level actions with non-additive loss, the objective is not modular, and top-\(B\) selection is not optimal.
The reason is immediate: if \(\ell_k\) is non-additive, the loss reduction from reviewing field \(i\) depends on whether field \(j\) in the same group is also reviewed. Reviewing an amount while leaving its binding identifier unreviewed produces a different — and in the key-binding case, much smaller — reduction than reviewing both. The per-element decomposition in the proof of Lemma 1 fails at the first line.
So the choice of decision unit is not a presentational matter. Grouping is what converts a set-function optimization with interaction terms into a sort. That asymmetry — modular at group level, non-modular at field level, under identical loss — is the formal content of the group formulation, and everything in Sections 14 through 16 is downstream of it.
12Theorem 1: selection regret, and its tightness
Lemma 1 assumes the selection values are known. They are not: \(v_k\) depends on \(r_M\), on \(r_H\), and on a consequence model, and all three are estimated. The operative question is how much is lost by ranking on estimates.
12.1 Setup
Let \(\hat v\) be any estimate of \(v\), with per-group error \(\varepsilon_k = |v_k - \hat v_k|\) and sup-norm error \(\varepsilon = \|v - \hat v\|_\infty\). Let \(S^{\star}\) be a top-\(B\) set under the true values and \(\hat S\) a top-\(B\) set under the estimates, ties broken arbitrarily. Define the selection regret
By Lemma 1 this is exactly the excess expected loss incurred by the estimated policy relative to the optimal one, so a bound on \(R\) is a bound on the operational cost of estimation error.
12.2 Statement
Assume A1, A3, A4. With \(d = |S^{\star} \setminus \hat S|\),
\[ R \;\le\; \sum_{k \in S^{\star} \setminus \hat S} \varepsilon_k \;+\; \sum_{k \in \hat S \setminus S^{\star}} \varepsilon_k \;\le\; 2\,d\,\varepsilon \;\le\; 2\,\min(B,\ m-B)\,\varepsilon . \]Let \(D = S^{\star} \setminus \hat S\) and \(E = \hat S \setminus S^{\star}\). By A4 both selections have exactly \(B\) elements, so \(|D| = |E| = d\). Elements of \(S^{\star} \cap \hat S\) appear in both sums of (12.1) and cancel, giving
\[ R \;=\; \sum_{k \in D} v_k \;-\; \sum_{j \in E} v_j . \]Fix any bijection \(\sigma : D \to E\). For \(k \in D\) with \(j = \sigma(k) \in E\): the estimated ranking selected \(j\) and rejected \(k\), so \(\hat v_j \ge \hat v_k\). Then
\[ v_k - v_j \;=\; \underbrace{(v_k - \hat v_k)}_{\le\, \varepsilon_k} \;+\; \underbrace{(\hat v_k - \hat v_j)}_{\le\, 0} \;+\; \underbrace{(\hat v_j - v_j)}_{\le\, \varepsilon_j} \;\le\; \varepsilon_k + \varepsilon_j . \]Summing over \(k \in D\), and using that \(\sigma\) is a bijection so each \(j \in E\) appears exactly once,
\[ R \;=\; \sum_{k \in D}\left(v_k - v_{\sigma(k)}\right) \;\le\; \sum_{k \in D}\varepsilon_k + \sum_{j \in E}\varepsilon_j \;\le\; 2d\varepsilon . \]Finally \(d \le B\) since \(D \subseteq S^{\star}\), and \(d \le m - B\) since \(E \subseteq \hat S\) and \(E\) is disjoint from \(S^\star\), whose complement has \(m-B\) elements. □
12.3 Tightness
The bound cannot be improved without further assumptions: both the constant 2 and the factor \(B\) are attained.
Take \(B=1\), \(m=2\), with true values \(v = (2\varepsilon,\, 0)\) and estimates \(\hat v = (\varepsilon,\, \varepsilon)\). Both estimates are within \(\varepsilon\) of truth, so \(\|v-\hat v\|_\infty = \varepsilon\). The estimated ranking is tied; resolving the tie toward group 2 gives \(\hat S = \{2\}\), \(S^{\star} = \{1\}\), and
\[ R \;=\; v_1 - v_2 \;=\; 2\varepsilon \;=\; 2B\varepsilon . \]Replicating this gadget in \(B\) disjoint blocks, each with its own unit of capacity, gives \(R = 2B\varepsilon\) exactly. □
The construction also shows the bound is not an artifact of adversarial estimation error: the estimator here is uniformly accurate to \(\varepsilon\) and unbiased in the ordinary sense. The loss arises purely from rank inversion at the capacity boundary.
12.4 A tightening that is free
Earlier statements of this result bounded regret by summing estimation error over the full sets \(S^{\star}\) and \(\hat S\). That version is valid but loose, and the looseness matters interpretively rather than just numerically.
Groups that both rankings agree about — selected by both, or rejected by both — contribute nothing to regret no matter how badly their values are estimated. The symmetric-difference form makes that visible; the full-set form conceals it behind a larger constant. Since the tighter form costs nothing to state, the looser one should not be used.
12.5 Corollary: boundary-localized calibration
Selection regret depends only on estimation error within \(S^{\star} \triangle \hat S\). Since \(\hat S\) is a top-\(B\) set, that symmetric difference lies entirely within the band of groups whose estimated values are close to \(\hat v_{(B)}\), the \(B\)-th largest estimate. Estimation error on groups ranked far above or far below that band contributes zero regret.
This has a direct and somewhat counterintuitive operational reading, and it is one of the more useful things the theory produces.
Do not calibrate everything. Calibrate near the capacity boundary.
Effort spent improving the estimator on groups that are obviously worth reviewing, or obviously not, buys nothing. Effort spent on the band around the \(B\)-th ranked value buys regret reduction proportionally. Annotation budget, active-learning acquisition, and reviewer-study sampling should all be concentrated there. Section 30 turns this into a concrete estimator-training protocol.
The width of the relevant band is a function of the value distribution rather than a constant, and it moves when capacity moves: a staffing change relocates the boundary and therefore relocates where estimator quality matters. This is worth building into the operating cadence, because a model that was well calibrated at \(B = 40\) is not thereby well calibrated at \(B = 120\).
12.6 Reading the bound in operational terms
Rewriting Theorem 1 in loss units makes its practical content explicit. If the estimator's sup-norm error over the boundary band is \(\varepsilon\), then excess expected loss per operating period is at most \(2B\varepsilon\). Two consequences follow.
Regret scales with capacity, not with volume. Doubling document volume does not double regret; doubling review capacity does. This is initially surprising and, on reflection, obvious: regret arises from swaps at the boundary, and there are \(B\) opportunities to swap. A system processing a million documents with forty reviewers has the same regret exposure as one processing ten thousand with forty reviewers, all else equal.
Halving estimator error halves regret exactly. The relationship is linear with no threshold effects, which makes estimator improvement easy to value against its cost. Plate 5 plots the bound against \(\varepsilon\) for several capacities.
13Why aggregate calibration is not sufficient
A natural conjecture — and one an earlier version of this framework asserted as a theorem — is that a well-calibrated risk estimator suffices to control selection regret. It does not. The conjecture is false, and its falsity is worth exhibiting concretely because the intuition behind it is so strong.
13.1 The false claim
The claim was, roughly: if the estimator is calibrated over the accepted region — meaning that among groups assigned estimated risk \(p\), a fraction \(p\) are in fact wrong — then the policy's excess loss is bounded by the calibration error. This has the shape of a true statement and is not one.
13.2 The counterexample
Four fields, unit consequence, perfect review, review capacity \(B=2\), so the selection value is proportional to the residual risk.
| Field | 1 | 2 | 3 | 4 |
|---|---|---|---|---|
| True risk \(r\) | 0.5 | 0.1 | 0.4 | 0.4 |
| Estimated risk \(\hat r\) | 0.3 | 0.3 | 0.4 | 0.4 |
Calibration. The estimator assigns 0.3 to fields 1 and 2, whose true risks average \((0.5+0.1)/2 = 0.3\). It assigns 0.4 to fields 3 and 4, whose true risks are exactly 0.4. Aggregate calibration error is zero in every bin.
Selection. Ranking by \(\hat r\) selects fields 3 and 4 (estimated 0.4 each) over fields 1 and 2 (estimated 0.3 each). The optimal selection under true risk is fields 1 and 3 (0.5 and 0.4).
Regret. \(R = (0.5 + 0.4) - (0.4 + 0.4) = 0.1 > 0\), despite perfect aggregate calibration. The riskiest field in the record is left unreviewed.
13.3 Why the intuition fails
The failure has a clean explanation once seen. Calibration is a statement about conditional averages; selection is a statement about order. A calibrated estimator may err arbitrarily on individual groups provided the errors cancel within bins. Selection does not average — it thresholds. Errors that cancel in expectation do not cancel in a ranking, and in the counterexample they conspire: the estimator compresses a high-risk and a low-risk field onto the same value, which is exactly the operation that destroys the ordering information selection depends on.
Theorem 1 identifies the property that actually controls regret: sup-norm accuracy of the selection value on the symmetric difference. Calibration matters only insofar as it sharpens \(\hat v\) there, and a calibration procedure that improves bin-level averages while compressing within-bin spread can make selection strictly worse.
Reliability diagrams and expected-calibration-error statistics are not sufficient acceptance criteria for an admission-control estimator. A model can post excellent ECE and route badly. The acceptance criterion must be stated in terms of ranking quality at the operating capacity — pairwise inversion rate within the boundary band, or realized selection regret against adjudicated values on a held-out set. Section 30.3 specifies both.
13.4 What calibration is still good for
Three legitimate uses survive, and it would be an overcorrection to discard the instrument:
- Constructing \(\hat v\). The selection value is built from probability estimates. Calibrated probabilities produce better-scaled value estimates, all else equal — calibration is an input, not a criterion.
- Communicating residual risk. A committed record's stated residual risk is a calibrated probability or it is meaningless. Downstream consumers reading a risk annotation need it to mean what it says.
- Detecting drift. A sudden deterioration in calibration is a strong signal that the input distribution has shifted, and is worth monitoring for that reason even though it does not bound regret.
The distinction to hold onto: calibration is a property worth having and an insufficient criterion for the decision this framework governs.
14Proposition 2: review value under non-additive loss
Sections 11 and 12 treat \(v_k\) as a scalar and say nothing about its internal structure. This section opens it. The object is to express \(C_k(\textsf{COMMIT})\) and \(C_k(\textsf{REVIEW})\) in a form that (i) makes no additivity assumption, (ii) exposes what review actually buys, and (iii) recovers the familiar field-level formula as a corollary rather than taking it as a premise.
14.1 Outcome classes
The device is a partition of outcomes on which the loss is constant.
Let \(\mathcal{E}_k\) be a countable partition of the joint outcome space of \(\left(\hat{Y}_k, Y_k\right)\) such that \(\ell_k\) is constant on each cell. Write \(L_k(e)\) for the loss on cell \(e \in \mathcal{E}_k\).
Such a partition always exists — in the worst case, the level sets of \(\ell_k\) themselves. In practice it is constructed directly from the domain, and it is usually small and interpretable. For a transaction group: all correct; amount wrong, binding correct; date wrong, binding correct; binding wrong; multiple payload fields wrong, binding correct. Five classes suffice to express a loss structure that no additive weighting can express at all.
Define the class probabilities under each action:
14.2 The decomposition
With the outcome-class partition of Definition 14.1,
\[ C_k(\textsf{COMMIT}) \;=\; \sum_{e \in \mathcal{E}_k} p^M_k(e)\, L_k(e), \qquad C_k(\textsf{REVIEW}) \;=\; c_k \;+\; \sum_{e \in \mathcal{E}_k} p^H_k(e)\, L_k(e), \]and therefore
\[ \Delta_k \;=\; \sum_{e \in \mathcal{E}_k} \left[\, p^M_k(e) - p^H_k(e) \,\right] L_k(e) \;-\; c_k , \]with the selection value \(v_k = \min\{C_k(\textsf{COMMIT}),\ \alpha_k\} - C_k(\textsf{REVIEW})\) as in (9.2).
Immediate from the definitions. \(\ell_k\) is constant on each cell of the partition, so the conditional expectation in (8.1) is the sum of \(L_k(e)\) weighted by the conditional class probabilities, and likewise for (8.2). Subtracting gives \(\Delta_k\). No additivity over fields is used anywhere. □
14.3 What the decomposition says
Read plainly, Proposition 2 says:
Not an error rate. Not an error rate multiplied by a weight. A shift in a distribution, weighted by what each outcome costs. Three things follow that the error-rate formulation cannot express.
Review can be worthless against a high error rate. If \(p^H_k \approx p^M_k\) — review does not change which outcome class obtains — then \(\Delta_k \approx -c_k\) regardless of how much probability mass sits on expensive classes. Section 15 makes this a bound.
Review can be harmful. Nothing in the formalism requires \(p^H_k\) to place less mass on expensive classes than \(p^M_k\). A reviewer who systematically "corrects" a correct machine output in some class makes that class worse. This is not hypothetical: it is the standard failure mode of review interfaces that present a machine answer as a default and reward throughput. The formalism represents it as \(\Delta_k < -c_k\), and the evaluation contract in Section 35 requires measuring it rather than assuming it away.
The relevant quantity is joint, not marginal. \(p^M_k\) and \(p^H_k\) are distributions over joint error patterns. Marginal per-field error rates are insufficient statistics for them whenever the loss is non-additive, because the loss depends on which fields are wrong together. Any estimator built on marginals alone is estimating the wrong object.
14.4 Recovering the field-level formula
Assume A2, so \(L_k(e) = \sum_{i \in e} w_i\) where \(e\) is identified with the set of fields in error. Then
\[ \sum_{e} p_k(e) L_k(e) \;=\; \sum_{i \in G_k} w_i \Pr\!\left(\text{field } i \text{ wrong}\right), \]and consequently
\[ \Delta_k \;=\; \sum_{i \in G_k} w_i\left(r^M_i - r^H_i\right) - c_k, \qquad\text{and for singleton groups}\qquad \Delta_i \;=\; w_i\left(r^M_i - r^H_i\right) - c_i . \]Exchange the order of summation: \(\sum_e p(e)\sum_{i\in e} w_i = \sum_i w_i \sum_{e \ni i} p(e) = \sum_i w_i \Pr(\text{field } i \text{ wrong})\). Apply to both \(p^M\) and \(p^H\) and subtract. □
This is the formula that appears throughout the practical literature on materiality-weighted review, and it is worth being precise about its status here. It is a corollary of the group formulation under an assumption this framework rejects, not an independent result and not a premise. Its scope is exactly the regime in which loss is additive — which is to say, the regime in which binding errors do not exist. Where they do exist, this formula is the wrong object, and Section 16 quantifies how wrong.
14.5 Estimating the class probabilities
Proposition 2 requires \(p^M_k(\cdot \mid S_k)\) and \(p^H_k(\cdot \mid S_k)\), which is more demanding than a per-field risk score. The practical position, stated so the requirement does not become a reason to quietly revert to marginals:
- \(p^M_k\) is estimable from adjudicated extraction outcomes by treating class membership as a multiclass label. This requires L1 annotation of full groups rather than isolated fields, which raises annotation cost per unit but not prohibitively — an annotator reading a document to check one field can check its whole group at marginal cost.
- \(p^H_k\) requires reviewer outcome data and is UNFILLED. It cannot be obtained from any public corpus (Appendix E). Either a reviewer study supplies it or it is declared a policy parameter with a reported sensitivity surface.
- Coarse partitions are permitted and often sufficient. With a binding-versus-payload partition of two or three classes, the estimation problem is small and the essential non-additivity is captured. Fine-grained partitions are a refinement, not a requirement.
15Corollary 4: consequence times distributional shift
Proposition 2 admits a bound that is the sharpest formal statement this framework makes about why error probability is not review priority.
15.1 Statement
Let \(L^{\max}_k = \max_{e} L_k(e)\) and let \(\mathrm{TV}\!\left(p^M_k, p^H_k\right)\) denote total variation distance. Then
\[ \Delta_k \;\le\; L^{\max}_k \cdot \mathrm{TV}\!\left(p^M_k,\, p^H_k\right) \;-\; c_k . \]From Proposition 2, and discarding cells where the difference is negative,
\[ \Delta_k + c_k \;=\; \sum_{e}\left[p^M_k(e)-p^H_k(e)\right]L_k(e) \;\le\; L^{\max}_k \sum_{e\,:\,p^M_k(e) > p^H_k(e)} \left[p^M_k(e)-p^H_k(e)\right] \;=\; L^{\max}_k \cdot \mathrm{TV}\!\left(p^M_k,p^H_k\right), \]using the standard identity that total variation equals the sum of positive parts of the difference. □
15.2 The operational reading
The bound factorizes review value into two independent quantities, and both must be large for review to be worth capacity.
| Factor | What it measures | What it depends on | What it does not depend on |
|---|---|---|---|
| \(L^{\max}_k\) | How much the worst outcome costs | Consequence policy, dependency structure, downstream propagation | Anything about the model or the reviewer |
| \(\mathrm{TV}(p^M_k, p^H_k)\) | How much review moves the outcome distribution | Reviewer capability on this error class, document legibility, whether the answer is present in the source at all | How much the error costs |
If review does not change the outcome distribution — the reviewer reaches the same impasse as the machine — then \(\mathrm{TV} \approx 0\), the bound gives \(\Delta_k \lesssim -c_k\), and review is worth less than nothing however high the error probability is. Error probability does not appear in the bound at all.
This is the formal content of the case in Section 4.4, and it discharges half of what earlier drafts of this work stated as a single empirical hypothesis. That the bound holds — that an arbitrarily high error probability can carry near-zero review value when a reviewer cannot move the outcome distribution — is a theorem, not a claim awaiting data. What remains empirical is the frequency question: whether \(\mathrm{TV}\) in fact varies materially across error classes in a given deployment. Section 36.1 states these separately as H3a and H3b, because conflating a proved mechanism with an unmeasured frequency understates the first and overstates the readiness of the second.
15.3 What the bound implies for reviewer instrumentation
If review value depends on \(\mathrm{TV}\), then a review process must be instrumented to estimate \(\mathrm{TV}\), which is a stronger requirement than measuring reviewer accuracy. Specifically it requires knowing, per error class, what the reviewer produced and what was actually correct — an independent adjudication, not a reviewer self-report. Appendix D specifies the protocol. This is the single largest instrumentation cost the framework imposes, and it is unavoidable: the alternative is assuming \(\mathrm{TV}\), which is assuming the answer.
15.4 A useful lower bound as well
The upper bound is the interesting direction, but a lower bound is available when review is known to be an improvement in a stochastic-dominance sense. If for every cell \(e\) with \(L_k(e) > \tau\) we have \(p^H_k(e) \le p^M_k(e)\) — review never increases the probability of any outcome costing more than \(\tau\) — then
where \(\mathrm{TV}^\tau\) restricts the total variation to cells above \(\tau\) and \(L^{\min,\tau}_k\) is the smallest loss among them. The condition is a monotonicity assumption about the reviewer and is testable; where it fails, the review process itself is the defect, not the routing policy.
16Proposition 5: identity-key amplification
This is the result that makes the group formulation load-bearing rather than decorative, and it converts the dependency-routing hypothesis from a conjecture into a claim with a derivable mechanism and a falsification condition.
16.1 The key-binding loss model
Consider a group \(G_k\) containing one identity field \(\kappa\) — an account, contract, patient, entity, or line-item identifier — and a set of payload fields \(P = G_k \setminus \{\kappa\}\). The loss:
The structure says: when the binding is right, payload errors cost what they cost; when the binding is wrong, the record is mis-attached and the payload's correctness is irrelevant to the consequence. This is a two-class outcome partition in the sense of Definition 14.1, and it is the simplest loss that is genuinely non-additive.
Write \(r^M_\kappa, r^H_\kappa\) for the identity field's residual error before and after review, and
for expected payload loss conditional on a correct key, before and after review.
16.2 Assumptions
(i) Binding dominance: \(L \ge W^M\). A mis-bound record costs at least as much as its expected payload errors. This is the empirical content of the model and is the condition whose failure falsifies it.
(ii) Monotone review: \(r^H_\kappa \le r^M_\kappa\) and \(W^H \le W^M\). Review does not make matters worse in expectation. Testable, and its failure indicts the review process rather than the theory.
16.3 Statement and proof
Under the key-binding loss (16.1) and assumptions (i)–(ii),
\[ \Delta_k \;\ge\; \left(r^M_\kappa - r^H_\kappa\right)\left(L - W^M\right) \;+\; \left(1 - r^M_\kappa\right)\left(W^M - W^H\right) \;-\; c_k \;\;\ge\;\; \left(r^M_\kappa - r^H_\kappa\right)\left(L - W^M\right) - c_k . \]From (16.1),
\[ C_k(\textsf{COMMIT}) = r^M_\kappa L + \left(1-r^M_\kappa\right)W^M, \qquad C_k(\textsf{REVIEW}) = c_k + r^H_\kappa L + \left(1-r^H_\kappa\right)W^H . \]Subtracting,
\[ \Delta_k + c_k \;=\; \left(r^M_\kappa - r^H_\kappa\right)L \;+\; \left(1-r^M_\kappa\right)W^M - \left(1-r^H_\kappa\right)W^H . \]Rewrite the trailing pair by adding and subtracting \((1-r^M_\kappa)W^H\):
\[ \left(1-r^M_\kappa\right)W^M - \left(1-r^H_\kappa\right)W^H = \left(1-r^M_\kappa\right)\left(W^M - W^H\right) - \left(r^M_\kappa - r^H_\kappa\right)W^H . \]Substituting and using \(W^H \le W^M\) from assumption (ii),
\[ \Delta_k + c_k = \left(r^M_\kappa - r^H_\kappa\right)\left(L - W^H\right) + \left(1-r^M_\kappa\right)\left(W^M-W^H\right) \;\ge\; \left(r^M_\kappa - r^H_\kappa\right)\left(L - W^M\right) + \left(1-r^M_\kappa\right)\left(W^M - W^H\right). \]Both terms are non-negative under (i) and (ii). □
16.4 The amplification corollary
The consequence for routing policy is the point of the whole exercise.
A field-level policy scores the identity field at its own field-level review value,
\[ \Delta_\kappa \;=\; w_\kappa\left(r^M_\kappa - r^H_\kappa\right) - c_\kappa , \]where \(w_\kappa\) is the identifier's own materiality weight. Comparing leading terms with Proposition 5,
\[ \frac{\Delta_k}{\Delta_\kappa} \;\asymp\; \frac{L - W^M}{w_\kappa} , \]which grows without bound in the binding amplification ratio \(A = L \big/ \sum_{i \in P} w_i\).
Two features of this ratio deserve emphasis.
The denominator is typically small. An account number, considered as an isolated field, carries little intrinsic consequence — it is a string, and a wrong string in a field nobody reads costs nothing. Its consequence is entirely relational. Field-level materiality elicitation, which asks "how much does an error in this field cost," reliably assigns it a low weight, because the honest answer to that question in isolation is low.
The mis-pricing grows with the structural property that makes the domain consequential. The more tightly a domain binds payload to identity — the more that a document's values only mean anything relative to what they attach to — the larger \(A\) becomes, and the worse field-level ranking performs. This is not a small-constant inefficiency to be tuned away. In a domain with \(A = 20\) and a low-weight identifier, field-level ranking can under-prioritize the identity field by more than an order of magnitude.
16.5 Status of the result, stated carefully
Proposition 5 is a statement about a loss model, not a finding about the world. It says: if a domain's loss has key-binding structure with amplification \(A\), then field-level ranking mis-prices the identity field by a ratio growing in \(A\). Whether any particular domain has that structure is an empirical question about consequence, and it is answerable — by eliciting the consequence model and by auditing realized losses from historical mis-binding incidents.
Proposition 5 is operationally falsified in a domain if measured binding-error consequence does not exceed payload-error consequence — that is, if assumption (i) fails. In such a domain the group formulation is unnecessary and field-level routing is adequate. This condition is listed among the kill criteria in Section 36.3, and it is checkable before any of the machinery in this paper is built.
16.6 Why this carries the novelty weight
Section 3 narrowed the framework's contribution considerably after a prior-art audit: capacity-constrained allocation and partial deferral both went to the borrowed column. What remains distinctive is the combination of a loss-coupling-derived decision unit with non-additive loss, and Proposition 5 is the sharpest consequence of that combination. It is a statement that cannot be made in a field-additive formalism at all, because the quantity \(L - W^M\) does not exist there.
An abstract for this work should therefore lead with amplification rather than with routing. Routing under capacity is the setting; amplification is the result.
17Proposition 3: hard gates as a limiting case
Real admission systems contain deterministic gates alongside probabilistic estimates: an invalid currency code, a date outside a permitted window, an identifier that fails a checksum. These are not uncertainty, they are invalidity. A specification needs to express both without pretending they are the same kind of object — and without maintaining two parallel decision paths that can disagree.
17.1 The parameterization
Let \(q\) be the estimated residual risk from the probabilistic model alone, and let \(b_1,\dots,b_J \in \{0,1\}\) be the outcomes of \(J\) deterministic checks, with \(b_j = 1\) denoting a pass. Model the combined residual risk in log-odds:
where \(g\) is a calibration link applied to the raw model score.
Under (17.1):
Soft penalty. Finite \(\theta_j > 0\) raises residual risk when check \(j\) fails, by an amount monotone in \(\theta_j\), while leaving risk unchanged when it passes.
Hard gate. As \(\theta_j \to +\infty\), a failed check drives \(r(s) \to 1\), forcing the group out of the commit region regardless of the model score.
If \(b_j = 1\) for all \(j\), the sum vanishes and \(r = g(q)\). If some \(b_j = 0\), the log-odds increase by \(\theta_j > 0\), and since the logistic function is strictly increasing, \(r\) increases. As \(\theta_j \to \infty\) the log-odds diverge and \(r \to 1\). □
17.2 Direction, and a correction worth recording
An earlier version of this parameterization had the sign reversed, with the effect that a failed check drove risk toward zero — the exact opposite of the intended behavior, and the kind of error that is invisible in prose and fatal in code. The formulation above is correct: a failed check adds to the log-odds of error, so risk increases.
The general lesson is worth stating because it recurs: in any additive-in-log-odds risk model, the sign convention on indicator variables must be checked against a limiting case rather than against intuition. Substituting \(b_j = 0\) and letting \(\theta_j\) grow is a two-second check that catches it.
17.3 What this proposition is not
Proposition 3 does not establish that deterministic gates and probabilistic scores are "the same thing," and it should not be quoted that way. It exhibits one parameterized family in which both appear as special cases. That is useful for implementation — a single risk pathway rather than two — and it is a modeling convenience, not an ontological claim.
In particular, the hard-gate limit is a mathematical device. An implementation should express a genuine hard gate as a hard gate, not as a large finite \(\theta_j\), because large finite penalties are numerically awkward and produce a false sense that the constraint is negotiable.
17.4 Setting the penalties
The \(\theta_j\) are either fit or set:
- Fit by logistic regression of adjudicated correctness on check outcomes and model score over a held-out set. This yields the empirical informativeness of each check — and frequently reveals that a check everyone believed was diagnostic carries almost no signal, which is a useful finding in itself.
- Set by policy where a check encodes a rule rather than evidence. A regulatory constraint that a record must satisfy is not an informativeness question, and fitting it would be a category error.
Where a check is fit and the estimated \(\theta_j\) is near zero, the honest conclusion is that the check is not evidence of correctness. It may still be worth running for other reasons — it may be evidence of consistency, which Section 24 shows is a different property — but it should not be presented to reviewers or auditors as a correctness signal.
18Proposition 1: the degenerate corner
For completeness, and to make explicit what the framework reduces to under maximally simplifying assumptions.
Assume: A2 (additive loss), A3 (unit cost), no abstention, perfect review (\(r^H = 0\)), known materiality weights, and that non-reviewed fields are committed. Then the optimal \(B\)-review set is the \(B\) fields with the largest \(w_i \cdot r^M_i\).
Under these assumptions each field is its own group, \(\alpha_i = \infty\) so \(b_i = C_i(\textsf{COMMIT}) = w_i r^M_i\), and \(C_i(\textsf{REVIEW}) = c\) with zero residual. So \(v_i = w_i r^M_i - c\), and ranking by \(v_i\) is ranking by \(w_i r^M_i\) since \(c\) is constant. Apply Lemma 1. □
18.1 Honest assessment
This result is unsurprising, and it is included at the size it deserves. It is the degenerate corner of the group model, obtained by assuming away every property that motivates the framework: non-additivity, reviewer fallibility, abstention, and consequence heterogeneity beyond a scalar weight.
It is worth stating for two reasons. First, it is what most deployed materiality-weighted review policies implicitly implement, so naming it locates existing practice inside the formalism. Second, its assumption list is a compact statement of what the rest of Part II is for: each of Sections 14 through 20 relaxes exactly one entry.
| Assumption | Relaxed in | What changes |
|---|---|---|
| Additive loss (A2) | §14, §16 | Outcome-class decomposition; identity-key amplification appears |
| Perfect review (\(r^H=0\)) | §14, §15 | Review value becomes a distributional shift, bounded by consequence × TV |
| No abstention | §9 | Selection value replaces the commit gap; the baseline becomes a minimum |
| Unit review cost (A3) | §19 | Knapsack; top-\(B\) no longer optimal; no regret bound claimed |
| Review is terminal | §20 | Sequential formulation; terminal form is a conservative lower bound |
| Known values | §12, §13 | Selection regret; aggregate calibration shown insufficient |
| Correct grouping | §21 | Misspecification term \(\kappa\) added to every bound |
19Heterogeneous review cost: the knapsack case
Assumption A3 — that reviewing any group consumes one unit of capacity — is the least realistic assumption in Part II. Review capacity is analyst-time, and groups differ substantially in how much of it they consume: a two-field group with a clear scan and a twelve-field group requiring a source-document comparison are not the same expenditure.
19.1 The problem without A3
Dropping A3, the allocation problem becomes
which is a 0/1 knapsack problem. Three consequences follow, and this section states them plainly rather than gliding over the change.
| Property | Under A3 (unit cost) | Without A3 (knapsack) |
|---|---|---|
| Optimal policy | Top-\(B\) by \(v_k\); exact | NP-hard in general; top-\(B\) by \(v_k\) is not optimal |
| Practical algorithm | Sort | Density-greedy by \(v_k / c^{\mathrm{res}}_k\), combined with the best-single-item solution: a standard \(1/2\)-approximation |
| LP relaxation | Integral | Fractional optimum exceeds the integer optimum by at most the value of one item |
| Regret bound | Theorem 1 applies, tight | Theorem 1 does not transfer. No regret bound is claimed here. |
19.2 The non-claim, stated explicitly
Theorem 1 is proved under A3 and does not extend to the heterogeneous-cost case without a separate argument. A density-ranking analogue is plausible — the natural conjecture is a bound in terms of estimation error in \(v_k/c^{\mathrm{res}}_k\) plus an approximation term — but it is not derived in this paper and is not asserted. An implementation operating under heterogeneous review costs is outside the regime where this paper's regret guarantee holds, and should say so in its own documentation.
This matters practically because the heterogeneous case is the common one. The recommended stance for a deployment: measure \(c^{\mathrm{res}}_k\); if its variance across groups is small, treat A3 as a working approximation and inherit Theorem 1 with a stated caveat. If its variance is large, use density-greedy, accept the \(1/2\)-approximation guarantee, and do not claim the regret bound.
19.3 Why the LP gap is usually benign
The integrality gap of one item is worth a sentence because it determines whether the knapsack complication matters in practice. If no single group consumes a substantial fraction of total capacity — the typical case, where \(B\) covers dozens or hundreds of reviews and each group takes minutes — then the gap is a rounding error and density-greedy is effectively optimal. The knapsack structure bites only when a few groups are very expensive relative to total capacity, which occurs in low-volume, high-complexity settings such as long-form contract review.
19.4 Multiple reviewer classes
A further generalization, noted for completeness and not developed: where reviewers differ in capability and cost — a junior checker and a senior specialist — capacity is a vector rather than a scalar, \(p^H_k\) becomes reviewer-dependent, and the problem becomes a generalized assignment. This is exactly the setting of the multi-expert deferral literature [10, 11], and an implementation needing it should adopt that machinery rather than extending this paper's. The framework's contributions — the decision unit, the loss structure, and the amplification result — carry over unchanged; only the allocation algorithm is replaced.
20Review as information acquisition
Equation (8.2) treats review as terminal: the group is reviewed, and the post-review state is committed. That is not what review is.
20.1 The sequential formulation
Review is an information-acquisition action. After it, the operator still faces a choice: commit the reviewed state, or hold it because review revealed the document to be genuinely undetermined. The correct specification is therefore
where \(\mathcal{F}^H_k\) is the information available after review — including whatever the reviewer learned about how determinable the group is.
\(C^{\mathrm{seq}}_k(\textsf{REVIEW}) \;\le\; C^{\mathrm{terminal}}_k(\textsf{REVIEW})\), with equality only when the post-review decision is never to abstain.
By Jensen's inequality applied to the concave function \(x \mapsto \min\{x, \alpha_k\}\):
\[ \mathbb{E}\left[\min\left\{\mathbb{E}\left[\ell_k \mid \mathcal{F}^H_k\right], \alpha_k\right\}\right] \;\le\; \min\left\{\mathbb{E}\left[\ell_k\right],\ \alpha_k\right\} \;\le\; \mathbb{E}\left[\ell_k\right], \]where the outer expectation is over the realization of \(\mathcal{F}^H_k\). The right-hand side is the residual term in (8.2). □
20.2 Consequence
The terminal specification understates review value. Every \(v_k\) computed the terminal way is a lower bound on the true selection value, and the gap is largest exactly where review is most informative about determinability — which is to say, on hard groups.
Two acceptable resolutions, and one unacceptable one:
- Adopt (20.1) as primary. More faithful, at the cost of requiring an estimate of the post-review distribution of the conditional expectation rather than just its mean. Feasible where a reviewer study captures reviewer-reported determinability.
- Keep the terminal form and label it. Report explicitly that computed review values are conservative lower bounds. Because the bias is one-directional, this is a defensible position: a policy that under-values review will under-review, which is a known and stated bias rather than an unknown one.
- Leave it unstated. Not acceptable. A quantity that is conservative by construction but presented as exact will read as an error to any reviewer who checks, and the error will be discovered rather than disclosed.
20.3 The implication for the action set
Proposition 6 also clarifies the state machine. A reviewed group does not necessarily become committed state — it may be held, on the reviewer's determination that the document does not settle the question. This is a legitimate and important outcome, and a review interface that offers only "confirm" and "correct" is missing the option that (20.1) prices. Table 6.1's transition column should therefore read: review leads to committed or held, and Section 34 lists the missing third button among the anti-patterns.
21Misspecification of the dependency graph
Assumption A1 holds by construction under Definition 7.1 — but the construction depends on eliciting a graph, and elicitation is fallible. This section states the cost of getting it wrong and how to bound it.
21.1 The misspecification term
Suppose the elicited graph omits a coupling: fields in different components are in fact loss-coupled. Then total loss does not decompose as A1 requires, and the decomposition in the proof of Lemma 1 acquires an error.
Let \(\kappa\) denote the largest expected loss attributable to cross-group coupling that the elicited graph omits:
\[ \kappa \;=\; \sup_{\text{policies}} \left| \; \mathbb{E}\!\left[L\!\left(\hat Y, Y\right)\right] \;-\; \sum_k \mathbb{E}\!\left[\ell_k\!\left(\hat Y_k, Y_k\right)\right] \; \right| . \]Every bound in Part II then carries an additive \(+\kappa\): Lemma 1's policy is optimal to within \(\kappa\), and Theorem 1 becomes \(R \le 2B\varepsilon + \kappa\).
21.2 Why omission is preferable to over-merging
The asymmetry noted in Section 7.6 can now be stated precisely.
Omitting an edge costs \(\kappa\), which is additive, bounded, and — critically — estimable, because a cross-group consequence is an observable event in an audit.
Adding a spurious edge merges two components. The resulting group has a larger \(c^{\mathrm{res}}_k\) and a review action that covers fields which did not need it. This degrades allocation efficiency, produces no diagnostic signal, and cannot be detected by auditing outcomes — the merged group's reviews all look successful, because they were. The loss is entirely in the reviews that were not performed elsewhere.
Since one error is measurable and the other is silent, elicitation should be biased toward omission, with doubtful edges recorded as suspected couplings for periodic re-examination.
21.3 Estimating the coupling defect
A concrete audit procedure, which is the only reason \(\kappa\) is worth defining:
- Sample committed records over an operating period.
- For each realized operational consequence traced back to an extraction error, record which fields were implicated.
- Flag any incident in which the consequence depended jointly on fields assigned to different groups.
- The expected loss of those incidents, per period, estimates \(\kappa\).
- Each flagged incident also names a missing edge. Add it, and the estimate declines.
This makes \(\kappa\) a decreasing, monitored quantity rather than an unbounded unknown, and it makes the graph an artifact that improves through operation. Its current value is UNFILLED; the procedure that fills it is above.
21.4 Versioning the graph
The dependency graph is a governed artifact, and changing it changes the decision unit — which changes what every stored selection value refers to. Following Paper 01's treatment of graph versioning:
- The graph is versioned, and every admission decision records the graph version under which it was made.
- Adding an edge is a minor version change requiring re-estimation of the affected groups' values; it does not invalidate decisions already committed under the prior version.
- An edge change never applies retroactively to committed state. Correcting historical admissions is a separate, explicit remediation exercise with its own authorization.
- Consequence-model changes and capacity changes are not graph changes and do not require re-versioning, though both relocate the capacity boundary and therefore trigger the calibration re-check of Section 12.5.
The machinery that produces the admission controller's inputs. Every technique in this part is prior art. What is specified here is the boundary condition each one carries, because those are what get dropped in deployment — and a dropped boundary condition is how a consistency check becomes a correctness claim.
22Reference pipeline and the evidence chain
The admission controller consumes candidate groups with attached evidence, check results, and scores. Something must produce them. This section specifies that something at the level of interfaces and guarantees, not implementation.
22.1 The governing principle
A model that emits both a value and a quotation purporting to support it has produced two model outputs, not a value and its evidence. The quotation is subject to the same failure modes as the value — it may be paraphrased, hallucinated, or drawn from the wrong region of the document — and treating it as verification is circular.
The alternative is to construct the evidence units outside the generative model and have the model select among them. A model-selected canonical evidence identifier is independently verifiable: the identifier resolves to a byte range that exists or does not, and its content can be compared to the extracted value by deterministic code. This is the same structural move Paper 01 makes with typed plan objects — constrain the model's output to references into a catalog the model did not create.
22.2 The chain
| # | Link | What it produces | What it guarantees, and what it does not |
|---|---|---|---|
| 1 | Immutable source capture | Original bytes, content hash, receipt timestamp, transport metadata | Guarantees the document has not changed since receipt. Guarantees nothing about its authenticity before receipt (L2 gap, §5.2). |
| 2 | Layout-preserving parse | Text with character offsets, page coordinates, table structure, reading order | Makes anchoring possible at all. Flattening layout here destroys information no later stage can recover. |
| 3 | Canonical evidence units | Deterministically segmented spans with stable identifiers | Created outside the model. This is the link that makes provenance verifiable rather than asserted. |
| 4 | Constrained candidate generation | Schema-valid candidate values with references to evidence unit IDs | Guarantees syntactic validity (§23). Guarantees nothing semantic. |
| 5 | Independent support verification | Alignment score per value against its cited evidence unit | Measures whether the value corresponds to the cited span (§26). Does not establish that the span is the right one. |
| 6 | Deterministic validation | Pass/fail per invariant, plus unsat cores on failure | Establishes internal consistency (§24). Never correctness. |
| 7 | Bounded repair | Revised candidates, capped retry count, escalation on exhaustion | Reduces \(r_M\) before routing (§25). Does not remove the routing decision. |
| 8 | Uncertainty measurement | Sampling divergence, model statistics, layout novelty (§27) | Supplies coordinates of \(S_k\). Requires calibration; is not itself a probability. |
| 9 | Residual-risk estimation | \(\hat r_M\), outcome-class distribution, \(\hat v_k\) (§29) | The controller's input. Quality here is what Theorem 1 bounds regret against. |
| 10 | Admission decision | Commit / review / abstain, with a recorded rationale of computable quantities | The boundary itself (Part II, §31). |
| 11 | Tamper-evident receipt | Hash-chained record of everything above (§28) | Proves the record was not altered. Says nothing about correctness. |
22.3 The ordering constraint
Two orderings in this chain are not negotiable, and both are violated routinely.
Evidence units before generation. If the model generates first and evidence is matched afterward, the match is a post-hoc rationalization and the alignment score measures the matcher rather than the extraction. Segmentation must be deterministic and prior.
Validation before risk estimation, not instead of it. A validation pass is a feature of the risk estimate, not a substitute for it. Pipelines that commit on validation pass have implicitly set \(\hat r_M = 0\) whenever checks pass, which Section 24 shows is false and Section 17 shows is unnecessary — the parameterization there expresses check outcomes as risk adjustments precisely so that this shortcut is unnecessary.
23Constrained decoding: removing syntactic error
23.1 Mechanism
Grammar-constrained or automaton-constrained decoding masks the decoder's logits at each step against the set of tokens permitted by the target schema, assigning zero probability to any continuation that would produce a structurally invalid output. Schema-invalid output becomes impossible rather than merely improbable.
This is established technique, and it is adopted here without modification. What matters for the framework is its precise effect on the decision problem.
23.2 Effect on the formalism
Let \(r_M = r_M^{\text{syn}} + r_M^{\text{sem}}\) decompose residual risk into errors of structure (wrong type, malformed value, missing required field, invalid enumeration member) and errors of content (right shape, wrong value). Constrained decoding drives \(r_M^{\text{syn}} \to 0\) and leaves \(r_M^{\text{sem}}\) untouched.
The consequence is worth stating because it is frequently mis-read in both directions. Constrained decoding is genuinely valuable: it eliminates an entire error class, removes a category of downstream parsing failure, and makes the remaining risk homogeneous enough to model. It also does nothing whatever for the errors this framework is about. A confidently produced, perfectly formatted, schema-valid wrong number is exactly what the admission controller exists to catch, and constrained decoding produces it more reliably than unconstrained decoding does.
Syntactic validity is not semantic correctness. A pipeline reporting "100% schema-valid output" has reported that it configured its decoder, not that its values are right.
23.3 A second-order effect worth watching
One non-obvious consequence deserves flagging because it interacts with Section 27. Constraining the decoder compresses output diversity, which mechanically reduces the disagreement observed under self-consistency sampling. A pipeline that adds constrained decoding and observes its sampling-divergence signal go quiet has not become more certain; it has become less able to express uncertainty through that channel.
The practical implication: if constrained decoding is introduced after a self-consistency signal has been calibrated, the signal must be re-calibrated. This is exactly the kind of silent control re-tuning that Paper 01 flags around model upgrades, arriving through a different door.
24Symbolic cross-field validation, and why SAT is not truth
24.1 Mechanism
Cross-field invariants — arithmetic reconciliation, temporal ordering, referential format, domain membership, conservation constraints — are encoded as logical constraints and discharged with a solver. This is more expressive than hand-rolled checks and, importantly, produces an unsatisfiable core on failure: a minimal subset of constraints that cannot be jointly satisfied, which localizes the problem and enables the repair of Section 25.
Typical invariants, drawn from the interchangeable domains of Section 7.5:
| Class | Form | What a pass establishes |
|---|---|---|
| Arithmetic reconciliation | Line items sum to subtotal; subtotal plus adjustments equals total | The extracted numbers are mutually consistent. Not that they match the document. |
| Temporal ordering | Effective date precedes expiry; service date within coverage period | The extracted dates are orderable as required. Not that they are the document's dates. |
| Referential format | Identifier matches a checksum or format grammar | The extracted string is well-formed. Not that it is the right identifier. |
| Domain membership | Currency in ISO 4217; code in the active code set | The value exists in a permitted set. Not that it is the value the document states. |
| Conservation | Allocations sum to one hundred percent; debits equal credits | The extracted allocation is internally coherent. Not that it reflects the document. |
Note that the third column is the same sentence eleven times. That is the point of the table.
24.2 The critical boundary
A solver returning satisfiable proves that the extracted values jointly satisfy the encoded constraints. It has no access to the document. Therefore any extraction error that preserves the constraint relation is invisible to it.
The canonical instance is worth working through because its structure generalizes. Suppose an OCR process misreads a digit 3 as an 8 in a subtotal on a degraded scan. If the same substitution occurs in the total — likely, since both figures are rendered in the same font at the same quality — then the arithmetic invariant \(\text{subtotal} + \text{adjustment} = \text{total}\) is preserved exactly. The solver returns SAT. The record is internally coherent and externally wrong, and it is now more likely to be committed than a record whose errors happened to break the relation.
The structure that makes this general: constraint checking is invariant to any transformation of the extracted values that preserves the constraints. Correlated errors — and OCR errors on a single document are strongly correlated, because they share a cause — are exactly the errors most likely to be constraint-preserving.
24.3 Consequences for the framework
Three concrete requirements follow, and they are the reason this section exists.
Validation pass is a feature, never a decision. In the formalism, a passed check enters through \(\theta_j\) in equation (17.1) as a reduction in log-odds of error. It does not set \(r_M = 0\). A pipeline that commits on validation pass has hard-coded an infinite \(\theta_j\) in the wrong direction.
Pass rate is reported as consistency yield. The metric name matters, because names propagate into dashboards and dashboards propagate into beliefs. "Validation pass rate: 97%" is read by every non-specialist as an accuracy claim. "Consistency yield: 97%" is not. Section 35 makes this a contract term.
Constraint-preserving error rate must be measured, not assumed away. The quantity of interest is \(\Pr(\text{value wrong} \mid \text{all invariants pass})\), estimable only from adjudicated data. It is UNFILLED here, and a deployment that has not measured it does not know what its validation layer is worth. This quantity is also the single most useful diagnostic for whether an invariant set is well designed: a high value means the invariants are checking relations that errors preserve.
24.4 Designing invariants that break under error
The observation above suggests a design criterion that is not usually stated. An invariant is useful in proportion to how unlikely plausible errors are to preserve it.
- Weak invariant: subtotal plus tax equals total, where all three are extracted from adjacent fields on the same degraded region. Correlated errors preserve it readily.
- Stronger invariant: extracted total matches a value computed from an independent source — a separate system, a prior document in the same series, a control total transmitted out of band. Errors in the extraction do not propagate to the comparison value, so the check has genuine discriminating power.
- Strongest: the invariant compares the extracted value against something that is not an extraction at all. This is the same principle as Paper 01's independent verifier node, which derives its value entirely from not sharing context with the thing it checks.
The general rule: an invariant's diagnostic value comes from the independence of its inputs, not from the sophistication of its logic. A solver discharging fifty constraints over one degraded scan region is less informative than a single comparison against an out-of-band control total.
25Unsat-core bounded repair
25.1 Mechanism
When the solver returns unsatisfiable, it can return a minimal unsatisfiable core: the smallest subset of constraints that conflict. Rather than regenerating the whole record, the pipeline returns only the violated constraint to the generator and requests a revision, bounded by a hard retry cap. On cap exhaustion the group escalates to review or abstention rather than looping.
25.2 Effect on the formalism
Repair is a pre-router \(r_M\)-reduction step. It shifts the distribution of \(\hat r_M\) downward before the admission decision is taken. It does not remove the decision, and post-repair residual risk enters the three-action objective unchanged.
Two properties are required of the repair loop, and both are inherited directly from Paper 01's treatment of evidence loops.
The cap is hard and small. An unbounded repair loop is the single most common source of runaway cost in pipelines of this shape, and the failure is not gradual — a small fraction of pathological documents consume unbounded resource while the median case is untouched. Cap at two or three attempts and escalate.
Escalation on exhaustion is to review or abstain, not to commit. A record that could not be repaired within the cap is a record the pipeline could not make consistent. Committing it because the loop ended is the worst available outcome, and it happens by default in implementations where the loop's exit path was not designed.
25.3 The boundary
A repair that produces a satisfiable solution has produced consistency, not correctness. The generator was told which relation was violated and asked to revise; nothing forced the revision toward the document's actual content. Repair can move a record from visibly wrong to invisibly wrong.
This suggests a measurement that is worth taking and rarely is: the post-repair error rate should be compared against the pre-repair error rate on adjudicated data, per attempt index. If repaired records are wrong more often than records that passed first time — which is expected, since they were harder — then repair success should raise \(\hat r_M\) relative to first-pass success, not lower it to the same level. The natural implementation is an attempt-index coordinate in \(S_k\), which lets the estimator learn the relationship rather than assuming it.
26Span alignment as measured anchoring
26.1 Mechanism
Each extracted value is aligned back to the source text using local sequence alignment, tolerant of the character-level noise that OCR produces — the familiar confusions between O and zero, l and one, S and five. The alignment yields both a source offset and a score.
26.2 Effect on the formalism
The score is the point. Anchoring is frequently specified as a binary requirement — every value must resolve to a source offset, and unanchorable output is rejected. That is a reasonable hard gate, and it enters the formalism through Proposition 3. But the score carries considerably more information than the gate:
| Score regime | Typical cause | Routing implication |
|---|---|---|
| High, exact | Value copied verbatim from a clean region | Lowers \(\hat r_M\) meaningfully. The strongest single-field signal available. |
| High, with substitutions | OCR noise in a legible region | Weak evidence of correctness. The substitutions themselves are the error channel — a high-scoring alignment through two digit substitutions is consistent with a wrong value. |
| Moderate | Value derived rather than copied — normalized date, computed total, resolved abbreviation | Uninformative on its own. Requires a derivation-aware check rather than an alignment threshold. |
| Low | Value not present in the source as extracted | Strong early-warning signal. Route or abstain. |
| None | Unanchorable | Hard gate. This is not an extraction; it is a generation. |
The third row is where naive implementations fail. Many legitimate extracted values are not present verbatim: dates get normalized, amounts get summed, names get canonicalized, codes get resolved. An alignment threshold applied uniformly rejects exactly the values a well-designed pipeline is supposed to produce. The correct treatment is to declare, per field, whether it is copied or derived, and to apply alignment as a gate only to the former while treating derived fields under their derivation's own check.
Alignment establishes correspondence between an extracted value and a source span. That is an L1 property. It says nothing about whether the span was the right span — a value perfectly aligned to a figure in the wrong column of the wrong table scores high and is wrong. And it says nothing at L2 or L3.
27Self-consistency as an uncertainty coordinate
27.1 Mechanism
Sample the extraction \(n\) times under stochastic decoding and measure divergence across samples: value disagreement per field, structural distance between candidate records, or the entropy of the sampled value distribution. High divergence indicates the model's output is unstable with respect to its own sampling process.
27.2 Why this is a different object from self-reported confidence
This distinction is the reason the technique appears here at all, given Section 2's rejection of confidence scores.
| Property | Self-reported confidence | Sampling divergence |
|---|---|---|
| What produces it | The model, as content. A generated assertion about itself. | The sampling process, as a measurement over multiple outputs. |
| Manipulable by prompt phrasing | Directly and substantially | Indirectly, through the output distribution |
| Reflects an actual property of the model | Not reliably | Yes — instability of the output distribution is a real property |
| Is it a probability? | No, despite looking like one | No, and it does not look like one, which is safer |
| Requires calibration before use | Yes, and often fails calibration | Yes, unconditionally |
One caution before the table is relied on. The empirical picture in [27] is that self-consistency, like the other two signals, collapses toward all-positive behaviour at practical thresholds — so the middle column's advantage over the left is a structural one about what the signal is, not a guarantee that it separates better in a given deployment. That must be established per deployment, not inherited from the table.
The critical shared row is the last. Sampling divergence is not an error probability. Low divergence means the model is stable, and a model can be stably wrong — indeed systematic errors are precisely the errors that are stable across samples. The signal must be calibrated against adjudicated outcomes before it enters \(\hat r_M\), and its relationship to error is empirical rather than definitional.
27.3 The cost, and when it is worth paying
The technique multiplies extraction cost by \(n\). Whether that is worth it follows directly from Corollary 1: sampling is worth its cost where the estimate matters, which is the boundary band. This suggests a two-stage design that follows from the theory rather than from intuition:
- Extract once. Estimate \(\hat v_k\) from cheap coordinates — check results, alignment scores, layout descriptors.
- Identify groups whose estimates fall near the current capacity boundary \(\hat v_{(B)}\).
- Re-sample only those groups, refine their estimates, and re-rank.
Cost scales with the width of the band rather than with volume, and by Corollary 1 the regret reduction is concentrated exactly where the additional information is purchased. This is the clearest instance in the paper of a theoretical result producing a non-obvious engineering decision.
28The tamper-evident decision trace
28.1 Mechanism
Each admission decision emits a receipt containing the document hash, the group identifier and graph version, the extracted values, cited evidence unit identifiers, alignment scores, check outcomes, the estimated selection value and its inputs, the policy version, the action taken, and — where review occurred — the reviewer identity, elapsed time, and the outcome. Receipts are hash-chained, and a periodic root is committed to an append-only store.
28.2 What this is for
Not correctness. The chain proves that the record of a decision has not been altered after the fact. That is a different and narrower property than most audit discussions assume, and stating the narrowness is the point.
| Question | Answered? | By what |
|---|---|---|
| Was this value admitted, and under what policy? | Yes | The receipt records the action, the policy version, and the graph version. |
| What evidence was the decision based on? | Yes | Evidence unit IDs resolve to byte ranges in the hashed source. |
| Has the record been altered since? | Yes | Hash chain. |
| Was the decision correct? | No | Nothing. Correctness is established by adjudication, not by integrity. |
| Was the source document genuine? | No | Nothing. L2 gap (§5.2). |
| Why did the model produce this value? | No, and deliberately not | See §28.3. |
28.3 What is deliberately not retained
Following Paper 01's position on decision provenance, the trace records the chain a decision rests on and not the model's intermediate reasoning. The argument transfers without modification: intermediate reasoning is unstable across runs, unstable across model versions, and not a reliable account of how an output was produced. A trace built on it is not reconstructable and not defensible.
What is reconstructable is: this document, hashed; these evidence spans; these check outcomes; this estimated risk under this estimator version; this policy; this action; this reviewer; this outcome. Every element is a computable quantity with a stable meaning, and together they answer the questions an auditor actually asks.
28.4 Retention asymmetry
One operational note. The receipt has a longer retention requirement than the working artifacts it references, because it is the audit record while they are intermediates. Evidence unit content, sampling outputs, and repair attempts can generally be discarded on a shorter clock, provided the receipt retains hashes sufficient to prove what they were. Designing this asymmetry in from the start avoids the common outcome in which storage pressure forces deletion of the audit record alongside the intermediates.
29Estimating the selection value
Theorem 1 bounds regret in terms of \(\|v - \hat v\|_\infty\) on the boundary band. This section specifies what produces \(\hat v\).
29.1 Decomposition of the estimation task
From (9.2), estimating \(v_k\) requires four components. Separating them is worthwhile because they have different data requirements and different failure modes.
| Component | Target | Data required | Status |
|---|---|---|---|
| \(\hat p^M_k(\cdot \mid S_k)\) | Outcome-class distribution under commit | Adjudicated group-level extraction outcomes (L1 annotation) | Estimable; UNFILLED |
| \(\hat p^H_k(\cdot \mid S_k)\) | Outcome-class distribution after review | Reviewer study with independent adjudication | Not obtainable from public corpora; UNFILLED |
| \(L_k(\cdot)\) | Consequence per outcome class | Policy elicitation | Policy-specified (§32.1) |
| \(c_k,\ \alpha_k\) | Review cost, abstention cost | Instrumentation; policy | Measurable / policy-specified |
Note the structural point: three of the four are tractable, and the entire difficulty concentrates in \(\hat p^H_k\). That is not an accident of this framework — it is a general property of any deferral system that declines to assume an oracle reviewer, and it is why so many systems assume one.
29.2 Coordinates of \(S_k\)
The feature vector available at decision time, assembled from the substrate:
| Coordinate | Source | Note |
|---|---|---|
| Check outcome vector \(b_1..b_J\) | §24 | Enters through (17.1). Individually weak; jointly informative. |
| Unsat-core identity | §25 | Which constraint failed is more informative than that one failed. |
| Repair attempt index | §25 | Repaired records differ systematically from first-pass records. |
| Alignment score per field | §26 | Segregate copied from derived fields (§26.2). |
| Sampling divergence | §27 | Available only for re-sampled groups under the two-stage design. |
| Layout novelty | Parse stage | Distance from seen layouts. A strong proxy in template-variant settings. |
| Source and trust class | Intake | Per Paper 01's classification. Untrusted sources warrant different priors. |
| Group structural descriptors | Schema | Group size, presence of an identity field, binding amplification \(A\). |
| Self-reported confidence | Model | Permitted as a coordinate. Never as a decision rule (§2.3). |
29.3 Estimator form
Nothing in the framework requires a particular estimator, and the choice should be driven by what makes the boundary band accurate rather than by what maximizes an aggregate score. Three workable forms, in increasing order of data appetite:
- Policy-parameterized. Set \(\hat p^H_k\) from declared parameters, estimate \(\hat p^M_k\) from adjudicated data, compute \(\hat v_k\) in closed form from Proposition 2. Requires no reviewer study. Report a sensitivity surface over the declared parameters. This is the honest minimum and the recommended starting configuration.
- Two-stage discriminative. Fit a multiclass model for \(\hat p^M_k(\cdot \mid S_k)\) over outcome classes; combine with policy-specified \(L_k\) and declared or measured \(\hat p^H_k\). Standard supervised learning against L1-adjudicated labels.
- Direct value regression. Regress realized \(v_k\) directly on \(S_k\) where adjudicated review outcomes exist. Most data-hungry, and the most directly aligned with what Theorem 1 bounds — it optimizes the quantity that appears in the bound rather than an intermediate.
29.4 Training against the right loss
A consequence of Section 13 that bears directly on estimator training: the estimator should be selected on ranking quality within the boundary band, not on aggregate probabilistic loss.
A model with better log-loss can route worse, because log-loss rewards accuracy in the bulk of the distribution where regret is zero. Concretely, the model-selection criterion should be one of:
- Realized selection regret against adjudicated values on a held-out set, at the operating capacity \(B\). Directly the quantity of interest.
- Pairwise inversion rate within the band — the fraction of pairs straddling the boundary that are ordered incorrectly. Cheaper to compute and a good proxy.
- Regret at several capacities. Because the boundary moves with \(B\), reporting at a single capacity overfits the estimator to a staffing level that will change.
30Where calibration effort should go
Corollary 1 has an unusually direct operational reading, and this section turns it into a protocol.
30.1 The band
Define the boundary band at capacity \(B\) as the set of groups whose estimated selection values lie within \(\delta\) of the \(B\)-th largest estimate:
By Corollary 1, if \(\varepsilon \le \delta/2\) then \(S^\star \triangle \hat S \subseteq \mathcal{B}_\delta\), so all regret originates inside the band. Everything outside it is decided correctly regardless of estimation error.
30.2 The protocol
- Compute the band from the current estimate distribution and current capacity. Its width in group-count terms is an empirical property of the value distribution and should be monitored — a band containing a large fraction of all groups indicates a value distribution with insufficient spread, which is itself a finding about the consequence model.
- Concentrate annotation there. Adjudication budget for estimator training should be sampled disproportionately from the band, with the sampling weights recorded so the estimator can be re-weighted for unbiased evaluation elsewhere.
- Concentrate re-sampling there. Self-consistency sampling (§27.3) is applied to band members only.
- Evaluate there. Report pairwise inversion rate within the band as a first-class metric alongside aggregate calibration.
- Recompute on capacity change. A staffing change moves \(\hat v_{(B)}\) and therefore moves the band. An estimator validated at one capacity is not thereby validated at another.
30.3 Acceptance criteria for an admission estimator
Drawing Sections 13, 29 and 30 together, an estimator should be accepted on the following, and Section 13.3's warning applies: calibration statistics alone are not among them.
| Criterion | Measured how | Why it is on the list |
|---|---|---|
| Realized selection regret at operating \(B\) | Held-out adjudicated set; compare policy value against oracle top-\(B\) | The quantity Theorem 1 bounds. Directly operational. |
| Pairwise inversion rate in \(\mathcal{B}_\delta\) | Held-out pairs straddling the boundary | Cheaper proxy for the above; localizes failures. |
| Regret stability across \(B\) | Recompute at several capacities | Guards against overfitting to a staffing level. |
| Calibration of committed residual risk | Reliability diagram on the committed region | Not a routing criterion, but committed records carry a stated risk that must mean what it says. |
| Abstention behavior on under-determined groups | Deliberately under-evidenced held-out subset | An estimator that never abstains has not been tested on cases where abstention is correct. |
| Stability under substrate change | Re-evaluate after any change to decoding, invariants, or model version | §23.3: substrate changes silently re-tune the signals the estimator consumes. |
Calibrate where decisions are close. Evaluate on ranking, not on average accuracy. Re-validate whenever capacity or substrate changes, because both move the only region where estimator quality has any effect.
The controller as a runtime component, the policy surface it reads, a fully worked numerical example under stipulated parameters, the failure modes it produces, the evaluation contract it must satisfy, and the conditions under which none of it should be built.
31The admission controller as a runtime component
31.1 Interface
The controller is a deterministic component. Given a candidate group, a policy, and a capacity ledger, it produces an action and a receipt. It contains no model and makes no inference of its own — every probabilistic quantity it consumes was produced upstream and is an input.
admission_request:
document_hash: sha256:9c1f...
graph_version: 3.2.0 # dependency graph, versioned artifact
policy_version: 2026-07-01.4
estimator_version: rme-1.9.2
group:
group_id: G-04471-A
fields: [counterparty_id, reference, effective_date,
currency, gross_amount, net_amount, rate]
has_identity_field: true
candidate:
counterparty_id: {value: "CP-88231", evidence: [EU-1194], align: 0.99}
gross_amount: {value: 412900.00, evidence: [EU-1207], align: 0.97}
# ... remaining fields
checks:
arithmetic_reconcile: pass
currency_iso4217: pass
date_within_period: fail # -> theta_j applies
identifier_checksum: pass
repair:
attempts: 1
resolved: true
signals:
alignment_min: 0.91
sampling_divergence: 0.14 # present only if group was re-sampled
layout_novelty: 0.62
self_reported_conf: 0.93 # recorded, never routed on
estimates:
outcome_class_dist:
all_correct: 0.912
payload_error_only: 0.061
binding_error: 0.027
residual_risk_committed: 0.088
capacity:
period: 2026-08-19
budget_units: 64
consumed_units: 41
unit_cost_estimate: 1.0
admission_decision:
group_id: G-04471-A
action: REVIEW
selection_value: 4.0535 # v_k, in policy loss units
baseline: 5.952 # b_k = min(C_commit, alpha)
baseline_action: COMMIT # what happens if capacity runs out
boundary_rank: 12 # position against v_(B)
in_boundary_band: false
rationale: # computable quantities only
c_commit: 5.952
c_review: 1.8985
alpha: 25.0
binding_amplification: 100.0
policy_refs: [consequence_model:txn.v4, capacity:emea_ops]
receipt: sha256:1a77...
The rationale block contains only quantities the controller computed. It contains no natural-language justification, and specifically no model-generated explanation. This follows Paper 01's rule that a gateway must authorize on state and policy rather than on agent-supplied text: a rationale string that influenced the decision would be an attack surface, and one that did not influence the decision is display content that belongs in the reviewer interface rather than in the decision record.
31.2 Ordering and the capacity ledger
Lemma 1 assumes the whole batch is visible: top-\(B\) requires knowing all \(m\) values. Real pipelines are frequently streaming, and a group arrives before its competitors for capacity are known. Three operating modes, with their trade-offs stated:
| Mode | How it works | Guarantee | When to use |
|---|---|---|---|
| Batch | Accumulate a period's groups; rank; allocate; dispatch. | Lemma 1 applies exactly. Theorem 1 bounds regret. | Periodic processing where a settlement delay of one cycle is acceptable. The default. |
| Threshold-streaming | Estimate the value distribution from history; derive a value threshold \(\tau\) expected to consume \(B\); route groups exceeding \(\tau\) on arrival. | Approximate. Degrades under distributional shift, which moves the realized consumption away from \(B\). | Low-latency requirements. Requires monitoring of realized consumption against \(B\). |
| Windowed | Batch within a short window; allocate the window's proportional share of capacity. | Lemma 1 within the window; suboptimal across windows. | The usual practical compromise. Window length trades latency against allocation quality. |
Under threshold-streaming the controller is exposed to a failure that is worth naming because it is not obvious: if the value distribution shifts upward — a batch of unusually difficult documents — a fixed threshold routes far more than \(B\) groups, and the queue absorbs the excess. The capacity ledger must therefore be enforcing rather than advisory: when consumed units reach the budget, the controller falls back to each group's baseline_action and records the fallback. A ledger that is merely observed produces a review queue that grows without bound, which is the operational form of the same failure Paper 01 describes for unbounded evidence loops.
31.3 Where the controller sits relative to the system of record
The controller does not write. It emits a decision; a separate deterministic writer acts on it, under its own service identity and with its own idempotency handling. This is the same privilege split as Paper 01's tool gateway, applied one layer earlier, and the reason is identical: the component that evaluates a probabilistic quantity should not also hold the authority that makes its evaluation binding.
32Policy surface: consequence, capacity, abstention
Three families of quantity are set by the organization rather than estimated from data. Together they are the policy surface, and their elicitation is the largest non-engineering task in a deployment.
32.1 The consequence model
\(L_k(e)\) assigns a cost to each outcome class. Practical guidance, in the order that works:
- Enumerate outcome classes per group type, coarsely. Three to five classes. Always separate binding error from payload error where an identity field exists — that separation is what makes the loss non-additive and is the whole point.
- Anchor on a reference class. Set one class to 1.0 and elicit the others as ratios. Relative consequence is far easier to elicit reliably than absolute, and every result in Part II is invariant to a common scaling.
- Elicit from the accountable owner, not from the engineering team. The question is what the organization loses, which the process owner knows and the engineer does not.
- Use realized incidents as calibration. Historical cases where an error reached production supply ratios directly, and are far more credible than survey responses.
- Report a sensitivity surface. The consequence model is an input, and results should be reported across a plausible range of it. A conclusion that holds only at one elicited setting is not a conclusion.
Asking "how much does an error in this field cost?" field by field will reliably produce a low weight for identity fields, because the honest answer for an isolated identifier is low. That is precisely the elicitation error Proposition 5 quantifies. The correct question is asked at group level: "what does it cost if this record attaches to the wrong object?" — a question with a very different answer.
32.2 Capacity
\(B\) is a staffing decision and is known exactly. Three notes.
Capacity is per period and per reviewer pool. Where pools have different capabilities, the single-scalar model breaks and the generalized-assignment treatment of Section 19.4 applies.
Capacity should be set from the value distribution, not from tradition. Once \(\hat v\) is available, the marginal value of the \(B\)-th review is \(\hat v_{(B)}\), which is directly comparable to the cost of the marginal reviewer. This turns a staffing argument into an arithmetic one, and it is one of the more immediately useful by-products of the framework — an organization that can compute \(\hat v_{(B)}\) can tell whether its review team is too small or too large.
Capacity changes relocate the boundary band. Per Corollary 1, this invalidates estimator validation performed at the previous capacity. A staffing change is therefore an estimator revalidation event, which is not intuitive and is easy to miss.
32.3 Abstention cost
\(\alpha_k\) prices holding a group unresolved. It is the most commonly omitted policy input and the one whose omission most distorts behavior, because omitting it is equivalent to setting it to infinity — declaring every field mandatory, which is almost never the actual policy.
| Regime | Character | Effect on the policy |
|---|---|---|
| \(\alpha_k = \infty\) | Mandatory field. Regulatory or contractual requirement to populate. | Recovers the two-action model; \(v_k = \Delta_k\). |
| \(\alpha_k\) large | Holding blocks a downstream process with a deadline. | Abstention available but rarely optimal. Review is usually worth its cost. |
| \(\alpha_k\) moderate | Holding triggers a manual workaround or a query to the source. | The interesting regime. Abstention competes with review, and \(v_k \ne \Delta_k\). |
| \(\alpha_k\) small | The value is discretionary enrichment; its absence costs almost nothing. | Never review. Commit if cheap, otherwise hold. Section 33's Group C. |
32.4 Per-consumer admission
Definition 6.1 makes admission a property of the read. It follows that a single group may be admissible for one consumer and not another — adequate for management reporting, inadequate for payment. Rather than forcing the strictest consumer's standard onto every use, the policy can carry per-consumer thresholds:
consumers:
- id: mgmt_reporting
max_residual_risk: 0.05
binding_error_max: 0.02
accepts_held_groups: true # renders as "unresolved"
- id: settlement
max_residual_risk: 0.002
binding_error_max: 0.0001
accepts_held_groups: false # blocks
requires_review_if: "binding_amplification > 25"
- id: regulatory_report
max_residual_risk: 0.01
binding_error_max: 0.001
accepts_held_groups: false
attestation_required: true
This is more faithful to how organizations operate than a single global gate, and it makes the trade-off explicit rather than implicit: a group committed under the reporting standard and withheld from settlement is a coherent state, and it is the state a single global gate cannot represent.
33Worked example with stipulated parameters
Every parameter below is stipulated. The loss unit is abstract and deliberately not a currency. Nothing here is measured, and none of these values should be transferred to any domain. The purpose is to show the arithmetic of the framework end to end and to demonstrate that the policies it recommends differ from the policies the alternatives recommend — which is a structural claim, provable from stipulated inputs, and not an empirical one.
33.1 The document and its groups
One document yields four dependency groups. Loss is expressed in abstract units; consequence ratios are what matter.
| Group | Content | Stipulated parameters | Review / abstention cost |
|---|---|---|---|
| A | Line-item group with identity field: counterparty ID plus payload amounts | Key-binding loss: \(L = 120\), \(\sum_{i\in P} w_i = 10\), so \(A = 12\). \(r^M_\kappa = 0.04\), \(r^H_\kappa = 0.005\), \(W^M = 1.2\), \(W^H = 0.3\). Identity field's own weight \(w_\kappa = 0.5\). | \(c_A = 1.0\), \(\alpha_A = 25\) |
| B | Header totals: gross, net, rate. No identity field; additive loss. | \(w = (6, 6, 2)\); \(r^M = (0.03, 0.03, 0.05)\); \(r^H = (0.004, 0.004, 0.01)\) | \(c_B = 0.8\), \(\alpha_B = 12\) |
| C | Effective date, handwritten and genuinely ambiguous. Two outcome classes. | \(L(\text{wrong period}) = 40\); \(p^M = 0.30\), \(p^H = 0.26\). Total-variation shift \(= 0.04\). | \(c_C = 1.5\), \(\alpha_C = 3\) |
| D | Free-text description. Singleton group. | \(w = 0.2\); \(r^M = 0.25\), \(r^H = 0.05\) | \(c_D = 0.3\), \(\alpha_D = 0.2\) |
33.2 Computing the action costs
Group A, from the key-binding model (16.1):
Group B, additive (Corollary 2):
Group C, two outcome classes (Proposition 2):
Group D:
33.3 The rankings disagree completely
| Group | Error probability | Commit cost \(C_k(\textsf{C})\) | Commit gap \(\Delta_k\) | Selection value \(v_k\) | Optimal action at \(B=1\) |
|---|---|---|---|---|---|
| A | 0.040 (4th) | 5.952 (2nd) | +4.0535 (1st) | +4.0535 (1st) | REVIEW |
| B | 0.107 (3rd) | 0.460 (3rd) | −0.408 (4th) | −0.408 (2nd) | COMMIT |
| C | 0.300 (1st) | 12.000 (1st) | +0.100 (2nd) | −8.900 (4th) | ABSTAIN |
| D | 0.250 (2nd) | 0.050 (4th) | −0.260 (3rd) | −0.260 (3rd) | COMMIT |
The first and last columns are close to reversed. The group with the lowest error probability is the only one worth reviewing; the group with the highest error probability should be neither committed nor reviewed but held.
33.4 Policy comparison
Total expected loss under four policies, at \(B = 1\) and \(B = 2\). Non-reviewed groups take their optimal non-review action except under the confidence policy, which commits them because it has no abstention concept.
| Policy | Reviews at \(B=1\) | Loss | Reviews at \(B=2\) | Loss |
|---|---|---|---|---|
| Selection value \(v_k\) (this framework) | A | 5.4085 | A only (no other \(v_k>0\)) | 5.4085 |
| Commit gap \(\Delta_k\) | A | 5.4085 | A, C | 14.3085 |
| Error probability | C | 18.3620 | C, D | 18.6220 |
| Commit everything | — | 18.4620 | — | 18.4620 |
Three readings, and each corresponds to a result in Part II.
Error-probability routing is barely better than committing everything — 18.362 against 18.462, a 0.5% improvement — because it spends its single review on Group C, where Corollary 4 caps review value at \(L^{\max}\cdot\mathrm{TV} - c = (40)(0.04) - 1.5 = 0.1\). It buys a tenth of a unit and leaves 4.05 units on the table in Group A.
The commit-gap policy is correct at \(B=1\) and wrong at \(B=2\), where it spends its second review on Group C — for which \(\Delta_C = +0.1\) looks positive — while the correct action is abstention at cost 3.000 rather than review at cost 11.900. The penalty is 8.900 units, and it arises entirely from ranking against the wrong counterfactual. This is Section 9.1's finding, instantiated.
Additional capacity is worth nothing here. Only one group has positive selection value, so \(B=2\) buys no improvement over \(B=1\). A policy that fills its capacity because it has capacity — which the other three do — spends effort to increase expected loss. Section 34 lists this among the anti-patterns.
33.5 Amplification, concretely
Group A illustrates Corollary 5 sharply. Under field-level ranking, the identity field is scored on its own weight:
Negative. A field-level policy would never review the identity field, at any capacity, because reviewing it appears to cost more than it returns. The group-level value of the same review is \(+4.0535\). The ratio of leading terms predicted by Corollary 5:
This is the elicitation trap of Section 32.1 in numbers. Asked "what does an error in the counterparty identifier cost," a respondent reasoning about the field in isolation answers with something like \(w_\kappa = 0.5\) — it is a reference string. Asked "what does it cost if this line item attaches to the wrong counterparty," the same respondent answers with something like \(L = 120\). Same field, same error, two questions, a factor of 240 between the answers. Only the second question is about the operational consequence.
33.6 Sensitivity
Because the framework's inputs are stipulated or policy-set, results must be reported across a range. Two sensitivities are worth showing for this example.
Sensitivity to \(r^H_\kappa\), which is UNFILLED. If reviewers resolve identity errors less reliably than stipulated, \(v_A\) falls:
| \(r^H_\kappa\) | 0.000 | 0.005 (stipulated) | 0.010 | 0.020 | 0.030 | 0.040 |
|---|---|---|---|---|---|---|
| \(v_A\) | 4.652 | 4.0535 | 3.455 | 2.258 | 1.061 | −0.136 |
The conclusion — review Group A — holds across the whole plausible range and reverses only when reviewers are no better than the machine at the identity field, at which point Corollary 4 is doing the work and abstention becomes correct. This is what a sensitivity analysis is for: the recommendation is robust, and the exact point at which it fails is named.
Sensitivity to \(\alpha_C\), which is policy-set. Group C's action depends entirely on the price of holding:
| \(\alpha_C\) | 1.0 | 3.0 (stipulated) | 10.0 | 11.9 | 12.0 | \(\infty\) |
|---|---|---|---|---|---|---|
| \(v_C\) | −10.90 | −8.90 | −1.90 | 0.00 | +0.10 | +0.10 |
| Optimal action | ABSTAIN | ABSTAIN | ABSTAIN | indifferent | REVIEW | REVIEW |
The switch occurs at \(\alpha_C = C_C(\textsf{REVIEW}) = 11.9\), and above \(\alpha_C = 12.0 = C_C(\textsf{COMMIT})\) the value saturates at \(\Delta_C\) because committing has become the binding baseline. The whole disagreement between the \(v_k\) and \(\Delta_k\) policies lives in the interval \(\alpha_C < 12\) — which is to say, it lives in every case where holding is cheaper than committing a probably-wrong value. That interval is not an edge case; it is the normal condition for discretionary fields.
34Failure modes and anti-patterns
The framework creates its own characteristic failures. Naming them is more useful than restating its benefits.
| Anti-pattern | Symptom | Underlying cause | Correction |
|---|---|---|---|
| Driving the hold rate to zero | Every group is committed or reviewed; abstention never fires. | \(\alpha_k\) unset, therefore effectively infinite. Every field treated as mandatory. | Elicit \(\alpha_k\) per group class. A healthy system holds a nonzero, stable fraction. |
| Filling capacity because it exists | Review queue always exactly full regardless of document difficulty. | Policy reviews top-\(B\) without checking \(v_k > 0\). | Enforce the positivity filter (9.3). Report unused capacity as a legitimate outcome. |
| Reporting consistency as accuracy | Dashboard says "97% validated." | Section 24's boundary dropped somewhere between the solver and the dashboard. | Rename the metric to consistency yield. Measure \(\Pr(\text{wrong} \mid \text{all checks pass})\) separately. |
| Assuming the reviewer is an oracle | \(r_H\) hard-coded to zero, or omitted from the model entirely. | The residual term in (8.2) dropped for tractability. | Measure \(r_H\), or declare it a policy parameter with a sensitivity surface. Never zero. |
| Field-level materiality elicitation | Identity fields carry low weights and are never routed. | The elicitation trap of Section 32.1. | Elicit at group level with the binding question. |
| Review interface with two buttons | Reviewers confirm ambiguous values because there is no way to say "the document does not settle this." | The sequential structure of Section 20 not reflected in the interface. | Third action: hold, with a reason code. This also supplies the data to estimate \(p^H\) honestly. |
| Throughput-rewarded review | Reviewer confirmation rate near 100%; review adds no measurable value. | Reviewers optimized on cases-per-hour. \(p^H \to p^M\), so by Corollary 4 review value collapses. | Measure \(\mathrm{TV}\) per reviewer and error class. A reviewer whose \(\mathrm{TV}\) is near zero is not reviewing. |
| Calibrating everywhere | Large annotation spend, no measurable regret reduction. | Corollary 1 ignored; effort spread uniformly. | Concentrate on the boundary band (§30). |
| Silent graph drift | Group definitions change with schema releases; historical decisions no longer interpretable. | Dependency graph not versioned. | Version it; record the version in every receipt (§21.4). |
| Committing on repair success | Repaired records committed at the same risk as first-pass records. | Repair treated as resolution rather than as risk reduction (§25.3). | Attempt index as an estimator coordinate. |
| Estimator selected on log-loss | Excellent calibration statistics, poor routing. | Section 13. Aggregate accuracy is not ranking quality at the boundary. | Select on realized selection regret or boundary inversion rate (§30.3). |
| Advisory capacity ledger | Review queue grows without bound under a difficult batch. | Threshold-streaming without enforcement (§31.2). | Enforce the ledger; fall back to baseline_action and record it. |
34.1 The two that are hardest to detect
Most entries above announce themselves. Two do not, and they are worth separate attention.
Throughput-rewarded review produces metrics that look excellent. Review volume is high, cases clear quickly, reviewer utilization is full, and the confirmation rate is near unity. Every operational dashboard is green. Meanwhile \(\mathrm{TV}(p^M, p^H) \approx 0\), so by Corollary 4 the entire review function is generating negative value — the organization is paying capacity to reproduce the machine's answers. The only diagnostic that detects this is independent adjudication of a review sample, which is precisely the measurement Section 35.4 requires and which throughput-oriented operations rarely fund.
Over-merged dependency groups are undetectable from outcomes, as Section 21.2 explains. Every review of an over-merged group succeeds, because it covers more fields than it needed to. The loss is entirely in the reviews that were not performed on other groups, and nothing in the system observes a counterfactual. The only defense is the elicitation discipline of Section 7.6 — bias toward omitting doubtful edges, since omission is measurable through \(\kappa\) and over-merging is not.
35The evaluation contract
The framework's empirical claims are unfilled. What can be specified now is the protocol under which they would be filled — and specifically the protocol's prohibitions, which are what distinguish an honest evaluation of a system like this from a flattering one.
35.1 Sampling unit
The unit is the document or source cluster, never the field and never the group. Fields within a document share OCR conditions, layout, source, and quality; their errors are strongly correlated. Treating them as independent samples inflates effective sample size dramatically and produces confidence intervals that are wrong by a large factor.
Consequently: paired document-level cluster bootstrap for inference; splits by source cluster so that documents from the same origin do not straddle train and test; and reporting of the intra-cluster correlation so that a reader can judge the inflation for themselves.
35.2 Primary outcomes
| Outcome | Definition | Why it is primary |
|---|---|---|
| Consequence-weighted committed loss | \(\sum_{k:\,a_k=\textsf{C}} \ell_k\) realized on adjudicated data | The quantity the framework minimizes. Everything else is diagnostic. |
| Commit risk at matched coverage | \(\Pr(\text{wrong} \mid \text{committed})\), with commit fraction held equal across arms | Comparisons at unmatched coverage are uninterpretable. |
| Risk–coverage curve and AURC | Standard, over committed groups | Comparability with the selective-prediction literature. |
| Realized selection regret | Policy value against oracle top-\(B\) on adjudicated values | Directly the quantity Theorem 1 bounds. |
| Review yield | Fraction of reviews that changed the committed state, by error class | Estimates \(\mathrm{TV}\); detects throughput-rewarded review. |
| Hold rate and hold resolution | Fraction abstained; what subsequently happened to held groups | Abstention is a first-class action and needs a first-class metric. |
| Consistency yield | Validation pass rate, named correctly | Reported, never as accuracy (§24.3). |
35.3 Fairness of comparison
Comparisons between routing policies are easy to rig accidentally. The required conditions:
- Same extractor and same model family across arms. A routing comparison confounded with an extractor change measures neither.
- Matched coverage or matched capacity. An arm that commits less will look safer.
- No threshold tuning on the final test set, and the tuning protocol reported.
- Capacity failures, timeouts, and queue overflows reported rather than excluded — an arm that silently drops hard cases is not comparable.
- Source clustering preserved in every split.
- Machine performance reported before human correction. Reporting post-review accuracy as system accuracy converts the reviewer's work into a model result, which is the most common way these systems are oversold.
35.4 Measuring the reviewer
This is the most demanding requirement in the contract and the one most often skipped.
Do not assume reviewed cases become correct. Reviewer output must be independently adjudicated, exactly as machine output is. Reviewer self-report is not measurement.
The protocol requires, per error class: residual error after review, correction rate, review time, and — where feasible — inter-reviewer agreement on a duplicated subset. From these, \(\mathrm{TV}(p^M, p^H)\) is estimable per class, which is what Corollary 4 needs and what no aggregate reviewer accuracy figure supplies.
A note on scope, following Section 5.1: the reviewer must be asked an L1 question — does the document say this? — and the adjudication must be against the same question. A reviewer implicitly asked whether the figure is true is performing an unbounded task, and the measured residual will not be the quantity the model is charged with.
35.5 The synthetic-data rule
Synthetic or templated documents may be used only to stress-test mechanics — OCR robustness, decoder validity, solver behavior, throughput. They must not ground any routing, \(r_H\), abstention, or consequence claim.
The reason is structural rather than fastidious. Synthetic documents with deterministic labels have no genuine ambiguity. Where there is no ambiguity there is no reviewer residual error, no case in which abstention is correct, and no error class on which review value varies. The very phenomena the framework exists to handle are defined out of existence by the data generation process, and a strong result on such data is evidence about the generator rather than about the framework.
35.6 What public data cannot supply
This is a finding rather than a preference, and it constrains the empirical programme permanently.
No public document-extraction corpus supplies human reviewer outputs. Every candidate — surveyed in Appendix E — provides documents and gold field values. None provides what a reviewer produced when shown an ambiguous case, nor an independent adjudication of that output. Therefore \(p^H_k\), and with it \(r_H\) and \(\mathrm{TV}\), are not measurable on public data at any budget.
| ID | Claim | Public-data testable? | Condition |
|---|---|---|---|
| H1 | Intervention-value routing reduces consequential committed loss versus confidence routing at matched capacity | Partial | Only with \(r_H\) supplied as a parameter; report as a sensitivity surface, never a point estimate |
| H2 | Dependency-aware group routing beats field-level routing where errors propagate | Yes | Requires a corpus with genuine grouping structure and a stated consequence model. Report as a function of binding amplification \(A\), per Proposition 5. |
| H3a | Review value is bounded by consequence times total-variation shift, so high error probability can carry near-zero review value | Not applicable | Discharged analytically by Corollary 4. Needs no corpus. |
| H3b | \(\mathrm{TV}\) varies materially across error classes in practice; the reviewer is not an oracle | No | Requires measured reviewer outputs. Not obtainable from any public corpus (Finding E.1). |
| H4 | Materiality-aware routing prioritizes some lower-probability cases over higher-probability low-consequence ones | Yes | Follows from the loss model with a transparently parameterized consequence function. |
| H5 | Three-action policy improves the risk–cost frontier versus forced binary | Partial | Frontier over \((\alpha, r_H)\); \(\alpha\) is policy-set, so this is a sensitivity result |
Read honestly, this table says the publishable empirical core on public data is H2 and H4. H1 and H5 become parameterized frontier results over a declared \(r_H\). H3a is already discharged and needs no data at all. H3b requires a reviewer study.
That is a coherent programme, and it is a different one from what an undifferentiated list of five hypotheses implies. An abstract should be written to it: two empirical claims, two sensitivity surfaces, one theorem, and one deferred measurement. The temptation to present H3b's mechanism and H3b's frequency as a single pending result should be resisted, because it makes a proved statement look unproved and an unmeasured one look imminent.
36Hypotheses and what would falsify them
36.1 The hypotheses
Stated formally, with all effect sizes UNFILLED.
| ID | Statement | Falsified if |
|---|---|---|
| H1 | At matched review capacity, ranking by \(\hat v_k\) yields lower consequence-weighted committed loss than ranking by self-reported confidence or by estimated error probability. | The difference is not distinguishable from zero after document-level clustering, across the plausible range of \(r_H\) and consequence parameters. |
| H2 | Group-level routing outperforms field-level routing, with the margin increasing in binding amplification \(A\). | The margin does not increase in \(A\), or is absent at high \(A\). This would indicate Proposition 5's assumptions fail empirically. |
| H3a | Review value satisfies \(\Delta_k \le L^{\max}_k\,\mathrm{TV}(p^M_k,p^H_k) - c_k\); consequently a group can carry arbitrarily high error probability and near-zero review value. Status: proved (Corollary 4). | Not falsifiable empirically. It follows from the definitions and the total-variation identity (B.3). A counterexample would indicate an error in the derivation, not in the world. |
| H3b | \(\mathrm{TV}(p^M_k, p^H_k)\) varies materially across error classes in deployment, and there exist classes where it is near zero at nonzero error probability. Status: UNFILLED. | \(\mathrm{TV}\) is approximately constant across classes — that is, the reviewer's value does not depend on the kind of error. |
| H4 | The optimal policy routes some groups with lower error probability ahead of groups with higher error probability, at nontrivial frequency. | The optimal ordering coincides with the error-probability ordering, which would mean consequence heterogeneity is operationally negligible. |
| H5 | Admitting abstention improves the risk–cost frontier relative to a forced binary policy at equal capacity. | The frontier is unchanged across the plausible range of \(\alpha\), which would mean holding is never preferable to committing or reviewing. |
36.2 What a negative result would mean
Distinguishing which negative results are fatal is part of specifying a programme, and it should be done before the data exists rather than after.
H1 negative is serious but survivable. It would mean the estimator is not good enough to realize the theoretical advantage — Theorem 1's \(\varepsilon\) is too large in practice. The theory would stand; the implementation would not be justified. The honest report is that the framework is correct and not yet operational.
H2 negative is fatal to the central claim. If group-level routing does not beat field-level routing where errors propagate, then the decision-unit commitment — the paper's primary surviving contribution after the prior-art audit — buys nothing. There is no reframing that rescues this, and it should be reported as such.
H3b negative would simplify the framework considerably. A constant \(\mathrm{TV}\) collapses Corollary 4 to a scalar and makes review value proportional to consequence times a constant. Much of Section 15 becomes unnecessary and the estimator gets simpler. This is a welcome negative result, and it should be looked for rather than avoided. Note that it would not touch H3a, which is a theorem: even under constant \(\mathrm{TV}\) the bound still holds and still separates review value from error probability. What a negative H3b removes is the operational claim that routing must be error-class aware, not the analytical claim that error probability is the wrong ranking quantity.
H4 negative means the domain does not need this. If consequence is effectively uniform, Proposition 1's degenerate corner is adequate and the framework's complexity is unjustified. This is a scoping finding rather than a refutation — the honest conclusion is that the domain was the wrong one.
H5 negative removes abstention. The framework reduces to two actions, Section 9's correction becomes moot, and \(v_k = \Delta_k\) throughout. A simplification, cleanly absorbed.
36.3 Kill criteria
Conditions under which the framework should be abandoned for a given deployment rather than tuned:
- The full system does not reduce consequential committed loss at matched coverage against a well-tuned confidence baseline.
- Benefits vanish after controlling for extractor capability and total cost.
- Low committed risk is achieved only through a hold rate high enough to make the system operationally useless.
- Review adds little conditional value in the routed region — that is, \(\mathrm{TV}\) is near zero where the policy routes, which indicts either the reviewer or the routing.
- The estimator cannot rank intervention value reliably at the operating capacity, and no amount of boundary-focused annotation improves it.
- Measured binding-error consequence does not exceed payload-error consequence, falsifying Proposition 5's binding-dominance assumption. In such a domain, field-level routing is adequate and the group formalism is overhead.
- Results fail under unseen-layout or cross-domain shift and cannot be honestly narrowed to a scope where they hold.
37When not to build this
A specification that recommends itself under all conditions is a sales document. The following conditions should lead an organization to build something simpler, and recognizing them early is worth more than any implementation guidance above.
Consequence is uniform across fields. The framework's machinery exists to exploit heterogeneity in \(\ell_k\). Where every error costs the same, materiality-aware routing degenerates to uncertainty-aware routing, and Proposition 1 is the whole theory. Use a calibrated risk score and a threshold.
There is no dependency structure. If every field is genuinely independent in consequence — no identifiers, no arithmetic relations, no binding — then groups are singletons, non-additivity never arises, and Proposition 5 has nothing to amplify. The group formalism is overhead with no return.
Review capacity is not binding. If everything worth reviewing can be reviewed, use (8.4) and stop. The allocation theory exists for the regime \(m^+ \gg B\).
Nothing can be held. Where every field is mandatory and abstention is genuinely unavailable, the three-action model reduces to two, and half the framework's distinctiveness disappears. Check this before building: many organizations believe every field is mandatory and are wrong, but some are right.
Errors are cheap relative to review. If \(c_k\) approaches or exceeds the expected consequence of error, the optimal policy is to commit everything and absorb the loss. This is a legitimate answer and is more common than practitioners expect in high-volume, low-stakes settings.
The consequence model cannot be elicited. The framework requires \(\ell_k\). If no one in the organization can state, even ordinally, what an error costs, then the model will be fabricated by the implementation team — and a fabricated consequence model produces confident, precise, wrong routing. Either fix the elicitation problem first or use the coverage-based framing of El-Yaniv and Wiener [2], which was designed for exactly this case.
There is no adjudication capability. Every estimator in Part III requires adjudicated data. An organization that cannot produce ground truth for a sample of its own documents cannot estimate \(r_M\), cannot validate the estimator, and cannot detect drift. Build the adjudication capability first; it has value independently.
Volume is too low to amortize. The framework requires a dependency graph, a consequence model, an estimator, instrumentation, and a reviewer measurement programme. Below some volume — which each organization must compute for itself from the marginal value \(\hat v_{(B)}\) — that investment cannot repay. A checklist and a competent reviewer are better.
The honest answer is a better source document. Some extraction problems exist because upstream parties send unstructured documents where a structured feed is available. Routing capacity at an ambiguity that a format change would eliminate is automation of a defect. Fix the interface; the extraction problem disappears rather than becoming manageable.
This framework earns its complexity where consequence is heterogeneous, dependency is real, and review capacity is binding. All three. A setting missing any one of them should be built differently, and the differences are not subtle — they change which of Proposition 1 through Proposition 5 is the operative result.
38Closing: authority is the scarce resource
The argument of this paper can be compressed to a single observation about what is actually scarce.
It is easy to believe that the scarce resource in a document-processing system is accuracy — that the work consists of making the extractor better, and that everything else follows. It does not follow. An extractor at ninety-four percent field accuracy and an extractor at ninety-eight percent field accuracy face the same question, which is which values may be trusted without a person looking at them, and neither answers it. Improving accuracy changes the frequency of the question, never its structure.
What is scarce is authority: the finite capacity of an organization to stand behind a value. That capacity has two forms. There is human review, which is measurable, budgeted, and small. And there is the willingness to commit a value without review, which is not usually thought of as a resource at all but behaves exactly like one — every automatic commitment spends a small amount of institutional credibility against the possibility that the value is wrong.
An admission policy is how an organization allocates that resource. Most organizations allocate it by accident, through a threshold nobody has revisited, a validation rule that measures consistency and is read as correctness, or a review queue that fills because it exists. This paper's claim is that the allocation can be specified: as a decision over dependency groups, with a loss that can represent binding errors, an action set that includes holding, and a value function that ranks by what review actually buys rather than by what the model happens to report about itself.
The specification is narrower than it first appears. Selective prediction, deferral, capacity-constrained routing, and partial abstention over structured outputs are all established, and Section 3 gives them their due. What is left is a decision unit derived from loss coupling, a loss that is non-additive because operational loss is non-additive, and the consequences that follow — of which the sharpest is that field-level materiality ranking under-prioritizes identity fields by a ratio that grows with exactly the structural property that makes a domain worth governing in the first place.
It is also unfinished, and deliberately so. Every empirical quantity is unfilled. Human residual error is not measurable on any public corpus, which makes one of six hypotheses untestable outside a reviewer study and forces \(r_H\) to be carried as a declared parameter with a sensitivity surface rather than as a measurement. The heterogeneous-cost case has no regret bound. The dependency graph is elicited and can be wrong in a way that is bounded but real. Each of these is stated rather than smoothed over, because the companion to this paper argued that borrowed numbers destroy business cases, and a sequel that quoted unmeasured figures would have retracted the argument in the act of extending it.
Paper 01 held that reasoning must not acquire execution authority. This paper holds the prior position: information must not acquire the standing of state without a decision that someone specified, can inspect, and can be held to.
The problem is not confidence estimation. It is the allocation of authority under uncertainty.
Notation, proofs restated in full, reference specifications, the evaluation protocol, the public-corpus landscape with its licensing, and a complete register of every quantity this paper leaves unfilled.
APP ANotation
| Symbol | Meaning | Where introduced / status |
|---|---|---|
| \(F, f_i\) | Schema field set; an individual field | §7.1 |
| \(\mathcal{G}, E\) | Loss-coupling graph and its edge set | Def. 7.1 — elicited, versioned |
| \(G_k, m\) | Dependency group \(k\); number of groups | §7.3 |
| \(S_k\) | Decision-time feature vector for group \(k\) | §7.3; coordinates in Table 29.2 |
| \(Y_k\) | Gold state of group \(k\). Unobserved. | §7.3 |
| \(\hat Y^M_k, \hat Y^H_k\) | Machine candidate; post-review state | §7.3 |
| \(\ell_k(\cdot,\cdot)\) | Group loss. Not assumed additive. | §7.4 — policy-specified |
| \(\mathcal{E}_k, L_k(e)\) | Outcome-class partition; loss on class \(e\) | Def. 14.1 |
| \(p^M_k(e), p^H_k(e)\) | Outcome-class probabilities under commit and under review | (14.1) — \(p^H\) is UNFILLED |
| \(a_k\) | Action on group \(k\): commit, review, or abstain | §8.1 |
| \(C_k(a)\) | Expected cost of action \(a\) on group \(k\) | (8.1)–(8.3) |
| \(c_k\) | Review cost in loss units | §8.2 — measurable |
| \(c^{\mathrm{res}}_k\) | Capacity resource consumed by reviewing group \(k\) | §8.2; \(=1\) under A3 |
| \(\alpha_k\) | Abstention cost — the price of holding the group | §8.2 — policy-specified |
| \(b_k\) | No-review baseline, \(\min\{C_k(\textsf{COMMIT}), \alpha_k\}\) | Def. 9.1 |
| \(v_k, \hat v_k\) | Selection value and its estimate | Def. 9.1 — the ranking quantity |
| \(\Delta_k\) | Commit gap, \(C_k(\textsf{COMMIT}) - C_k(\textsf{REVIEW})\) | (9.1) — special case of \(v_k\) at \(\alpha_k=\infty\) |
| \(B\) | Review capacity per period | §8.4 — policy-specified |
| \(S^\star, \hat S\) | Optimal and selected top-\(B\) review sets | §12.1 |
| \(R\) | Selection regret | (12.1) |
| \(\varepsilon_k, \varepsilon\) | Per-group and sup-norm estimation error in \(v\) | §12.1 |
| \(d\) | \(|S^\star \setminus \hat S|\) — size of the symmetric difference | Thm. 1 |
| \(\mathcal{B}_\delta\) | Boundary band at width \(\delta\) | (30.1) |
| \(r^M_i, r^H_i\) | Machine and human residual error for field \(i\) | Cor. 2 — \(r^H\) UNFILLED |
| \(w_i\) | Field materiality weight (additive case only) | A2 — recovered shadow quantity |
| \(\kappa\) (field) | Identity field within a group | §16.1 |
| \(L, W^M, W^H\) | Binding loss; expected payload loss before and after review | (16.1)–(16.2) |
| \(A\) | Binding amplification, \(L / \sum_{i\in P} w_i\) | Cor. 5 |
| \(\kappa\) (scalar) | Coupling defect — loss from omitted cross-group edges | Def. 21.1 — UNFILLED |
| \(b_j, \theta_j\) | Deterministic check outcome; its log-odds penalty | (17.1) |
| \(\mathrm{TV}(\cdot,\cdot)\) | Total variation distance | Cor. 4 |
A note on the two \(\kappa\)s. The symbol is used for the identity field in Section 16 and for the coupling defect in Section 21, following convention in both places. They never appear in the same expression. Where ambiguity is possible the identity field is written \(\kappa_f\).
APP BProofs in full
The proofs of Lemma 1, Theorem 1, Proposition 2, Corollaries 2, 4 and 5, Propositions 3, 5 and 6 appear inline in Sections 11 through 20 and are complete as given. This appendix supplies the material that would otherwise interrupt the argument: the exchange argument in explicit form, the tightness construction generalized, and two results stated without proof in the body.
B.1 The exchange argument, in full
Lemma 1 asserts that a modular objective over a uniform matroid is exactly maximized by greedy selection. Explicitly:
Let \(V(S) = \sum_{k\in S} v_k\) with \(|S| \le B\). Order the groups so that \(v_1 \ge v_2 \ge \dots \ge v_m\), and let \(T = \{1,\dots,B'\}\) where \(B' = \min\{B, |\{k : v_k > 0\}|\}\). Suppose \(S^\dagger\) is optimal and \(S^\dagger \ne T\).
Case 1: \(S^\dagger\) contains some \(k\) with \(v_k \le 0\). Removing it gives a feasible set with \(V\) weakly larger, and strictly larger if \(v_k < 0\). If \(v_k = 0\), removal leaves \(V\) unchanged and frees capacity, so we may assume without loss that \(S^\dagger \subseteq \{k : v_k > 0\}\).
Case 2: \(|S^\dagger| < B'\). Then some \(k \notin S^\dagger\) has \(v_k > 0\), and \(S^\dagger \cup \{k\}\) is feasible with strictly larger \(V\). So \(|S^\dagger| = B'\).
Case 3: \(|S^\dagger| = B'\) and \(S^\dagger \ne T\). Then there exist \(k \in T\setminus S^\dagger\) and \(j \in S^\dagger \setminus T\). By the ordering and \(k \le B' < j\), we have \(v_k \ge v_j\). The swap \(S' = (S^\dagger\setminus\{j\})\cup\{k\}\) is feasible with \(V(S') - V(S^\dagger) = v_k - v_j \ge 0\). Iterating the swap transforms \(S^\dagger\) into \(T\) without decreasing \(V\), so \(V(T) \ge V(S^\dagger)\) and \(T\) is optimal.
Note that Case 3 shows only that \(T\) is an optimum; where values tie at the boundary there may be several, all with equal objective value. This is why Theorem 1 permits arbitrary tie-breaking. □
B.2 Tightness of Theorem 1, generalized
Fix \(B \ge 1\) and \(\varepsilon > 0\). Take \(m = 2B\) groups arranged in \(B\) disjoint pairs. In pair \(t\), set
\[ v_{2t-1} = 2\varepsilon,\quad v_{2t} = 0, \qquad \hat v_{2t-1} = \varepsilon,\quad \hat v_{2t} = \varepsilon . \]Then \(\|v - \hat v\|_\infty = \varepsilon\). Every group has estimate \(\varepsilon\), so the estimated ranking is entirely tied; a tie-break that selects the even-indexed member of each pair gives \(\hat S = \{2,4,\dots,2B\}\), while \(S^\star = \{1,3,\dots,2B-1\}\). Then
\[ R = \sum_{t=1}^{B} \left(2\varepsilon - 0\right) = 2B\varepsilon , \]attaining the bound with equality. Since \(d = B\) here, the intermediate bound \(2d\varepsilon\) is attained as well. □
The construction is degenerate in its tie structure, which invites the question of whether the bound is attained under strict inequality. It is, to within any \(\eta > 0\): perturb \(\hat v_{2t}\) to \(\varepsilon + \eta\) for arbitrarily small \(\eta\), which forces the same selection without ties and changes \(\|v-\hat v\|_\infty\) to \(\varepsilon + \eta\). Taking \(\eta \to 0\) recovers the bound in the limit.
B.3 The total-variation identity
Corollary 4 uses the standard identity that for probability measures \(p, q\) on a countable set,
The second equality follows from taking \(A = \{e : p(e) > q(e)\}\) and observing that the positive and negative parts of \(p-q\) sum to the same total, since both measures have mass one. The bound in Corollary 4 uses the middle form directly.
B.4 Convexity in the Jensen step of Proposition 6
Proposition 6 applies Jensen's inequality to \(\phi(x) = \min\{x, \alpha_k\}\). This function is concave on \(\mathbb{R}\): it is the pointwise minimum of two affine functions, and the pointwise minimum of concave functions is concave. Jensen's inequality for concave \(\phi\) gives \(\mathbb{E}[\phi(X)] \le \phi(\mathbb{E}[X])\), which is the direction used. Applying it with \(X = \mathbb{E}[\ell_k \mid \mathcal{F}^H_k]\) and using the tower property \(\mathbb{E}[X] = \mathbb{E}[\ell_k]\) completes the argument.
B.5 Two results stated without proof in the body
The knapsack facts of Section 19. That (19.1) is NP-hard follows by reduction from subset-sum; that density-greedy combined with the best single item achieves a \(1/2\)-approximation, and that the LP relaxation exceeds the integer optimum by at most one item's value, are standard results in combinatorial optimization and are not reproved here. Section 19 cites them as facts rather than deriving them, and does not build any of the paper's claims on top of them.
The lower bound (15.1). Under the stated monotonicity condition, the derivation parallels Corollary 4 with the roles of the positive and negative parts exchanged, restricting attention to cells above the threshold \(\tau\) and bounding the contribution of cells below by \(\tau\). It is stated for completeness and is not used elsewhere.
APP CReference specifications
Machine-readable forms of the artifacts a deployment must maintain. These are illustrative schemas, not a standard.
dependency_graph:
version: 3.2.0
schema_ref: txn_schedule.v7
elicited_by: [process_owner, domain_sme, data_architect]
reviewed: 2026-07-14
edges:
- {from: counterparty_id, to: gross_amount, reason: binding}
- {from: counterparty_id, to: net_amount, reason: binding}
- {from: counterparty_id, to: reference, reason: binding}
- {from: effective_date, to: rate, reason: regime_selection}
- {from: currency, to: gross_amount, reason: unit_scaling}
- {from: gross_amount, to: net_amount, reason: arithmetic}
suspected_edges: # recorded, deliberately NOT applied
- {from: reference, to: effective_date, reason: "possible series ordering",
status: under_review, opened: 2026-06-02}
groups: # derived: connected components
- id: G-txn-core
fields: [counterparty_id, reference, effective_date, currency,
gross_amount, net_amount, rate]
has_identity_field: counterparty_id
binding_amplification: 12.0
- id: G-desc
fields: [description]
has_identity_field: null
coupling_defect_estimate:
kappa: UNFILLED # audit procedure: §21.3
last_audit: null
consequence_model:
version: txn.v4
units: abstract_loss_units # NOT currency; ratios are what matter
reference_class: payload_single_field # anchored at 1.0
elicited_from: process_owner
calibrated_against: [incident_2025_11_mapping, incident_2026_03_entity]
group_type: G-txn-core
outcome_classes:
- id: all_correct
loss: 0.0
- id: payload_single_field
loss: 1.0 # anchor
- id: payload_multiple_fields
loss: 2.4
- id: binding_error
loss: 120.0 # elicited with the BINDING question
elicitation_note: >
"What does it cost if this line item attaches to the wrong
counterparty?" — NOT "what does an error in this field cost?"
abstention_cost: 25.0
group_type: G-desc
outcome_classes:
- {id: all_correct, loss: 0.0}
- {id: wrong, loss: 0.2}
abstention_cost: 0.2
sensitivity_range: # results reported across this
binding_error: [60.0, 240.0]
abstention_cost: [10.0, 50.0]
capacity_ledger:
pool_id: emea_ops_tier1
period: 2026-08-19
budget_units: 64
consumed_units: 41
enforcement: hard # NOT advisory — §31.2
on_exhaustion: fallback_to_baseline_action
marginal_value: # v_(B) — informs staffing (§32.2)
current_estimate: 0.84
marginal_reviewer_cost: 0.71
implication: "marginal review value exceeds marginal cost; pool is undersized"
boundary_band: # §30.1
delta: 0.25
member_count: 19
annotation_weight: 4.0 # oversample for estimator training
receipt:
receipt_id: R-2026-08-19-004471-A
prev_receipt_hash: sha256:0b3e... # hash chain (§28)
document_hash: sha256:9c1f...
group_id: G-04471-A
versions:
dependency_graph: 3.2.0
consequence_model: txn.v4
policy: 2026-07-01.4
estimator: rme-1.9.2
extractor: ext-4.1.0
evidence:
- {field: counterparty_id, unit: EU-1194, offset: [4021,4029], align: 0.99,
trust: authoritative}
- {field: gross_amount, unit: EU-1207, offset: [5140,5150], align: 0.97,
trust: authoritative}
checks:
arithmetic_reconcile: pass
date_within_period: fail
repair: {attempts: 1, resolved: true}
estimates:
outcome_class_dist: {all_correct: 0.912, payload_error_only: 0.061,
binding_error: 0.027}
c_commit: 5.952
c_review: 1.8985
alpha: 25.0
v: 4.0535
decision:
action: REVIEW
baseline_action: COMMIT
capacity_consumed: 1.0
review:
reviewer_id: rv-2291
elapsed_seconds: 214
outcome: corrected # corrected | confirmed | held
fields_changed: [counterparty_id]
adjudication: pending # independent; §35.4
receipt_hash: sha256:1a77...
APP DEvaluation protocol
D.1 Data requirements
| Requirement | Specification | Why |
|---|---|---|
| Adjudicated extraction outcomes | Group-level L1 labels: for each group, which outcome class obtained | Estimates \(p^M_k\). Group-level rather than field-level because the loss is non-additive. |
| Reviewer outcomes | What the reviewer produced, per routed group, with time and reason codes | Estimates \(p^H_k\). Not obtainable from public corpora (§35.6). |
| Independent adjudication of reviewer outcomes | A second annotator, blind to the reviewer's answer, on a sample | Reviewer self-report is not measurement. Without this, \(\mathrm{TV}\) is unidentifiable. |
| Source cluster labels | Origin identifier per document | Clustering for inference and for splits. |
| Under-evidenced subset | Documents deliberately degraded or genuinely ambiguous | The only way to test abstention behavior. Without it, hold rate is untested. |
D.2 Reviewer study design
The minimum viable design for estimating \(p^H_k\), stated so the cost is visible before it is committed to:
- Stratify by error class, not by volume. The quantity of interest varies by class, so a proportional sample under-covers rare high-consequence classes. Oversample them and re-weight.
- Present the reviewer an L1 question. "Does the document state this value?" — not "is this value correct?" (§5.1).
- Provide the third option. Confirm, correct, or hold-as-undetermined. Without hold, ambiguous cases are forced into a confirmation and \(p^H\) is biased toward \(p^M\) (§20.3, §34).
- Adjudicate blind. A second annotator resolves the ground truth without seeing the reviewer's answer or the machine's.
- Duplicate a subset across reviewers to estimate inter-reviewer variance, which bounds how much of the measured \(\mathrm{TV}\) is reviewer-specific.
- Instrument time per group, since \(c_k\) and \(c^{\mathrm{res}}_k\) come from the same study at no extra cost.
D.3 Analysis plan
- Paired document-level cluster bootstrap for all interval estimates.
- Comparisons at matched coverage or matched capacity; unmatched comparisons reported as such.
- Holm correction across the hypothesis family.
- A pre-declared effect-size floor below which a statistically detectable difference is reported as operationally negligible.
- Sensitivity surfaces over every policy-specified quantity — \(\ell_k\), \(\alpha_k\), and any declared \(r_H\) — rather than point estimates.
- Machine performance reported before human correction, always and separately.
D.4 Reporting requirements
A report on a system built to this specification must state:
- Whether review cost was treated as unit or resource-weighted; if resource-weighted, that no regret bound is claimed (§19.2).
- Whether \(C_k(\textsf{REVIEW})\) was terminal or sequential; if terminal, that selection values are conservative lower bounds (§20.2).
- The coupling defect \(\kappa\), or an explicit statement that it was not estimated (§21.3).
- Validation pass rate labeled as consistency yield, together with \(\Pr(\text{wrong}\mid\text{all checks pass})\) (§24.3).
- The provenance of \(r_H\): measured, declared, or unfilled. Never omitted.
- The capacity at which the estimator was validated, and whether capacity has changed since (§32.2).
APP ECorpus landscape and licensing
An empirical programme for this framework needs public, redistributable, structured-extraction corpora spanning more than one domain, with genuine dependency structure. This appendix surveys the landscape and states a finding that constrains what any such programme can demonstrate.
Licence positions below are as reported in dataset papers and in third-party benchmark documentation. Every licence must be verified at source before publication relies on it. Several of these corpora are distributed under terms that are ambiguous, token-gated, or unstated, and at least one has an open and unanswered public question about its licence.
| Corpus | Domain | Scale | Dependency structure | Licence position (verify at source) |
|---|---|---|---|---|
| DocILE [19] | Business documents — invoices, orders | ~6.7k annotated real documents; 55 field classes; large synthetic and unlabeled subsets | Strong. The line-item recognition task requires grouping extracted values into tuples — an annotated dependency group in the sense of §7.3 | Reported MIT; distribution is token-gated |
| CORD [20] | Retail receipts | ~1k receipts | Moderate. Line items with subtotal, tax and total — genuine arithmetic invariants | Reported CC BY-SA 4.0 |
| SROIE [21] | Scanned receipts | 1,000 receipts, 4 fields | Weak. Four flat fields; no group structure | Reported MIT |
| Kleister-Charity [22] | Charity annual financial reports | 2,788 reports; ~61.6k pages; ~21.6k entities | Moderate. Identity plus financial fields; values often not localized in text | Unclear. A public question about the licence has been open on the repository since 2022 without resolution |
| Kleister-NDA [22] | Legal — non-disclosure agreements | 540 agreements; ~3.2k pages | Moderate. Parties, jurisdiction and term; binding structure is natural | Unclear, same repository |
| DeepForm | Political advertising disclosures | — | Moderate | Subject to a third-party terms-of-use rather than an open licence; not cleanly redistributable |
| FUNSD | Forms | — | Weak to moderate | Non-commercial academic use only |
E.1 The finding
Every corpus above provides documents and gold field values. None provides what a human reviewer produced when shown an ambiguous case, nor an independent adjudication of that output.
Consequently \(p^H_k\), \(r_H\), and \(\mathrm{TV}(p^M, p^H)\) are not measurable on public data at any budget. H3b is untestable outside a reviewer study; H1 and H5 are testable only as sensitivity surfaces over a declared \(r_H\). H3a is unaffected, being analytic.
Benchmarks exist that do supply expert decisions under capacity constraints — the learning-to-defer literature has produced at least one such resource — but they are not document-extraction tasks, and in the case known to this author the experts are synthetic. Under the synthetic-data rule of Section 35.5, synthetic experts may support mechanics and may not ground an \(r_H\) claim, so that resource does not close the gap.
E.2 Selection guidance
The corpus decision is downstream of the \(r_H\) decision, not the other way round. With \(r_H\) resolved as a declared policy parameter (§35.6), the empirical core reduces to H2 and H4 — and both require dependency structure and a consequence model, not reviewer outputs. That materially changes what a corpus must supply.
Adopted: DocILE + CORD. DocILE is the only candidate whose annotations supply dependency groups directly: its line-item recognition task requires grouping extracted values into tuples, which is an annotated dependency group in the sense of §7.3. That distinction is decisive, because a corpus without labelled grouping forces the groups to be hand-constructed — and a hand-constructed grouping tests the elicitation procedure of §7.2 rather than H2. CORD adds genuine arithmetic invariants across line items, subtotal, tax and total at low cost, which exercises the consistency-versus-correctness machinery of §24 — and numeric-field interdependence has already been quantified on that corpus independently [29], which gives H2 a measured dependency structure to test against rather than an asserted one.
Kleister-Charity is pursued in parallel, and blocks nothing. Its licence question has been open publicly since 2022. If it clears, it supplies a second domain at a different document length and the cross-domain claim is substantiated. If it does not, the evidence is scoped to business transaction documents and the scope statement narrows accordingly — see the warning below, which this paper holds itself to.
SROIE is excluded. Its licence is the most permissive of the three and that is the only argument for it. Four flat fields carry no group structure, so it cannot test H2 at all; including it would add a third business-transaction corpus that buys no domain generality while appearing to. Padding a corpus set with a benchmark that cannot address the primary hypothesis is a presentational choice, not an empirical one.
For completeness, the configurations considered:
| Configuration | Rationale | Cost |
|---|---|---|
| DocILE + CORD adopted | Only DocILE supplies annotated dependency groups, which is what H2 requires; CORD adds arithmetic invariants. Both licences reported permissive. | Both are business-transaction documents. Domain generality is not substantiated by this pair alone, and the scope statement is narrowed unless Kleister clears. |
| + Kleister-Charity pursued in parallel | Adds a second domain at a different document length and modality; entities are frequently not localized in text, which stresses the alignment treatment of §26. | Licence unresolved since 2022. Pursued without blocking; if it clears the cross-domain claim is substantiated, otherwise nothing is delayed. |
| DocILE + CORD + SROIE | Marginally lower legal risk than the adopted pair. | Rejected. Four flat fields, no group structure, cannot test H2. Adds a third business-transaction corpus while appearing to broaden coverage. |
| DocILE + Kleister-NDA | Maximum domain distance: invoices against legal contracts. | Held in reserve. 540 documents is small, and the licence uncertainty is the same as Kleister-Charity's. |
If the licence position forces the business-documents-only configuration, the domain-generality claim must be narrowed in the title, abstract and scope statement. Retaining a general claim on single-domain evidence is exactly how an otherwise honest paper acquires a dishonest sentence, and it is the specific failure this note exists to prevent.
APP FRegister of unfilled quantities
Every quantity this paper names and does not measure, with the procedure that would fill it. This register is the paper's accounting of its own incompleteness.
| Quantity | What it is | How it would be filled | Blocker |
|---|---|---|---|
| \(r_M\), \(p^M_k\) | Machine residual risk; outcome-class distribution under commit | Held-out group-level L1 adjudication | Annotation cost only. Tractable. |
| \(r_H\), \(p^H_k\) | Human residual error; outcome-class distribution after review | Reviewer study with blind independent adjudication (App. D.2) | Not obtainable from public data (Finding E.1). Requires a study or a declared parameter. |
| \(\mathrm{TV}(p^M,p^H)\) | Distributional shift induced by review, per error class | Derived from the above | Same as \(p^H\). |
| \(\ell_k\), \(L_k(e)\) | Consequence per outcome class | Policy elicitation at group level, calibrated against realized incidents (§32.1) | Organizational, not technical. Requires an accountable owner. |
| \(c_k\), \(c^{\mathrm{res}}_k\) | Review cost and capacity consumption | Instrumented review timing | None. Available from any timestamped review tool. |
| \(\alpha_k\) | Abstention cost | Policy elicitation per group class | Commonly omitted; omission silently sets it to \(\infty\). |
| \(B\) | Review capacity | Known exactly from staffing | None. |
| \(\theta_j\) | Per-check log-odds penalties | Logistic fit against adjudicated correctness, or policy-set for hard gates (§17.4) | Requires adjudicated data. |
| \(\kappa\) | Coupling defect from omitted graph edges | Audit of committed records for cross-group consequences (§21.3) | Requires incident tracing. |
| \(A\) | Binding amplification per domain | Ratio of elicited binding loss to summed payload weights | Follows from the consequence model. |
| \(\Pr(\text{wrong}\mid\text{checks pass})\) | Constraint-preserving error rate | Adjudication of records that passed all invariants (§24.3) | Rarely measured; the single most useful diagnostic of invariant design. |
| \(\varepsilon\) on \(\mathcal{B}_\delta\) | Estimator error in the boundary band | Held-out comparison against adjudicated selection values | Requires \(p^H\) to compute true \(v\). |
| H1, H2, H4, H5 effect sizes | The empirical claims | The programme in Appendix D | See Table 35.2 for what public data can and cannot reach. |
| H3b effect size | Cross-class variation in \(\mathrm{TV}\) | Reviewer study only (App. D.2) | Blocked by Finding E.1. H3a is proved and appears in no row of this register, being analytic. |
Thirteen rows. Two are unblocked and cheap, four require adjudication capability, three require policy elicitation, and one — the reviewer distribution — is blocked by a property of the public data landscape rather than by budget. That last row is why this paper is a specification and not a result.
—Selected references
Verification levels are stated for each entry, because a citation asserted at a strength it has not been checked to is a small dishonesty that compounds. M metadata verified against two or more independent sources. C claim verified — abstract or substantive passage read and the attributed claim confirmed. U unverified — encountered in reference lists only; opened before any argumentative weight is placed on it. No entry below was read end to end.
- Chow, C. K. On optimum recognition error and reject tradeoff. IEEE Transactions on Information Theory, 1970. U — universally credited as the origin of the reject option; verify directly before relying on the specific formulation.
- El-Yaniv, R. and Wiener, Y. On the foundations of noise-free selective classification. Journal of Machine Learning Research 11:1605–1641, 2010. M
- Geifman, Y. and El-Yaniv, R. Selective classification for deep neural networks. NeurIPS, 2017. arXiv:1705.08500. C
- Geifman, Y. and El-Yaniv, R. SelectiveNet: a deep neural network with an integrated reject option. ICML, PMLR 97:2151–2159, 2019. C
- Madras, D., Pitassi, T. and Zemel, R. Predict responsibly: improving fairness and accuracy by learning to defer. NeurIPS, pp. 6150–6160, 2018. arXiv:1711.06664. M
- Mozannar, H. and Sontag, D. Consistent estimators for learning to defer to an expert. ICML, PMLR 119:7076–7087, 2020. arXiv:2006.01862. C
- Verma, R. and Nalisnick, E. Calibrated learning to defer with one-vs-all classifiers. ICML, 2022. M
- Narasimhan, H., Jitkrittum, W., Menon, A. K., Rawat, A. and Kumar, S. Post-hoc estimators for learning to defer to an expert. NeurIPS 35:29292–29304, 2022. M
- Mozannar, H., Lang, H., Wei, D., Sattigeri, P., Das, S. and Sontag, D. Who should predict? Exact algorithms for learning to defer to humans. AISTATS, PMLR 206:10520–10545, 2023. M
- Mao, A., Mohri, C., Mohri, M. and Zhong, Y. Two-stage learning to defer with multiple experts. NeurIPS, 2023. M
- Alves, J. V., Leitão, D., Jesus, S., Sampaio, M. O. P., Liébana, J., Saleiro, P., Figueiredo, M. A. T. and Bizarro, P. Cost-sensitive learning to defer to multiple experts with workload constraints. Transactions on Machine Learning Research, 2024. arXiv:2403.06906. C
- De, A., Okati, N., Zarezade, A. and Gomez-Rodriguez, M. Classification under human assistance. AAAI 35:5905–5913, 2021. arXiv:2006.11845. C
- De, A., Koley, P., Ganguly, N. and Gomez-Rodriguez, M. Regression under human assistance. AAAI, pp. 2611–2620, 2020. M
- Okati, N., De, A. and Gomez-Rodriguez, M. Differentiable learning under triage. NeurIPS 34:9140–9151, 2021. M
- Reid, M., Sühr, T., Vernade, C. and Samadi, S. Online decision deferral under budget constraints. arXiv:2409.20489, 2024. C
- Raghu, M., Blumer, K., Corrado, G., Kleinberg, J., Obermeyer, Z. and Mullainathan, S. The algorithmic automation problem: prediction, triage, and human effort. arXiv:1903.12220, 2019. U
- Garcia, A., Essid, S., Clavel, C. and d'Alché-Buc, F. Structured output learning with abstention: application to accurate opinion prediction. ICML, 2018. arXiv:1803.08355. C
- Rayan, S. and Tewari, A. Learning to partially defer for sequences. arXiv:2502.01459, 2025. C
- Šimsa, Š., Šulc, M., Uřičář, M., Patel, Y., Hamdi, A., Kocián, M., Skalický, M., Matas, J., Doucet, A., Coustaty, M. and Karatzas, D. DocILE benchmark for document information localization and extraction. ICDAR, 2023. arXiv:2302.05658. C
- Park, S., Shin, S., Lee, B., Lee, J., Surh, J., Seo, M. and Lee, H. CORD: a consolidated receipt dataset for post-OCR parsing. Workshop on Document Intelligence, NeurIPS, 2019. M
- Huang, Z. et al. ICDAR 2019 competition on scanned receipt OCR and information extraction (SROIE). ICDAR, 2019. M
- Stanisławek, T. et al. Kleister: key information extraction datasets involving long documents with complex layouts. arXiv:2105.05796, 2021. C
- Hendrickx, K., Perini, L., Van der Plas, D., Meert, W. and Davis, J. Machine learning with a reject option: a survey. Machine Learning 113(5):3073–3110, 2024. U — recommended as the survey anchor for the abstention literature rather than enumerating it.
- Ramaswamy, H. G., Tewari, A. and Agarwal, S. Consistent algorithms for multiclass classification with an abstain option. Electronic Journal of Statistics 12(1):530–554, 2018. U
- Preprint. Valid per-field selective risk control for document extraction: three failure modes, a validity ladder, and when conditioning pays. arXiv:2608.14639, 2026. C — nearest published neighbour on granularity; controls marginal risk rather than allocating a budget, and its unit is the field.
- Kumar, N. Beyond logprobs: a multi-signal confidence engine for LLM-based document field extraction. Perfios Software Solutions. arXiv:2606.24420, 2026. C — reports that logprob, verbalized and self-consistency signals collapse at practical thresholds on DocILE.
- Preprint. Can you trust the confidence? ConfBench for vision-language models on document extraction. arXiv:2608.01792, 2026. C
- Preprint. Joint extraction matters: prompt-based visual question answering for multi-field document information extraction. arXiv:2503.16868, 2025. C — quantifies numeric-field interdependence on CORD; the closest independent evidence for this paper's decision-unit premise.
- Preprint. Pre-inference routing for cost-efficient document field extraction. arXiv:2608.06607, 2026. C
- Preprint. Rethinking uncertainty in segmentation: from estimation to decision. arXiv:2604.13262, 2026. C — deferral granularity at low review budgets, outside documents.
- Orangkhadivi, S. From copilots to controlled digital operations: a practical architecture for governed multi-agent orchestration and stateful workflow graphs. Independent architecture paper, August 2026.
Residual citation debt
Stated because a reference list that conceals its own gaps is worse than a short one:
- Full-text reading of [11] and [12] before any comparative claim is made about their settings. Their treatment of capacity is the closest to this framework's, and the distinction drawn in Section 11.4 rests on abstracts and secondary description rather than on complete texts.
- Direct verification of [1], which is cited from secondary sources throughout the abstention literature.
- Discharged. The targeted search for selective prediction and deferral specifically within document information extraction has been performed; results are in Section 3.6. No collision with the surviving claim, but the intersection is more crowded than the earlier reference list implied. Full-text reading of [26] and [27] is still outstanding before any comparative claim is made about their guarantees.
- A decision on whether to engage or explicitly scope out the human–AI complementarity literature. Silence there reads as unawareness rather than as scoping.
—About the author
Sammy Orangkhadivi works on AI systems for regulated operational environments. He writes independently on architecture and control for systems in which machine reasoning meets institutional accountability. This is the second of two related papers; the first, From Copilots to Controlled Digital Operations, addresses the governance of action, and this one addresses the governance of admission.
The method in both papers is applied company by company rather than sold as a product: the dependency graph, the consequence model, and the capacity constraint are different in every organization, and a framework that pretends otherwise has skipped the part that matters.
Correspondence and correction are welcome. Errors identified in this paper will be recorded and versioned rather than silently amended, on the same principle that governs the removed novelty claims in Section 3 — the record of what a framework stopped claiming is part of its evidence.