Most legal AI marketing says "accurate" and stops. I would rather show you the ledger. Since August we have run Doclayer through a set of qualification gates on public contracts, with the inputs frozen, the pass criteria written down before the run, and a person checking the output. Some gates passed. Several did not. This is what the numbers say as of 21 September 2026, taken from our internal feature ledger, which records every run whether or not it helped.
The scoreboard
| Gate | Result | What happened |
|---|---|---|
| Cited answers, ten public questions | Passed | Every answer sentence approved by a reviewer against the source. Total model cost for the run: about five cents. |
| Structured extraction, AURA public control | Passed, then failed on repeat | Nine of nine gold fields found; 19 of 24 spans copied exactly; one repeat joined a quote across 604 characters of intervening text. |
| Structured extraction, strict CUAD contracts | Failed | Four contracts passed. The fifth failed on the assignment and governing-law gold spans. Six later versions of the prompt did not get further. |
| Local open-weight model, 122B parameters | Failed | Ran the same control in 90 seconds on 85 GB of GPU memory, missed a clause, and produced shortened citations that were not in the source. |
| PageIndex retrieval | Not adopted | Both configurations failed the quality and indexing gate we set before the test. |
The gate that passed: cited answers
This is the feature you use in the product today. We froze ten questions over public documents, ran them through the production model adapter, and had an authorised reviewer read every sentence of every answer against the cited passage. All ten passed on 12 August 2026. The whole run cost about five cents in model fees, which is worth knowing when you are wondering what a Review Unit costs us.
The design that made this possible is the one I described in the earlier post: the server cuts the excerpts, the model can only select from them, and an answer that names an excerpt the server did not send is rejected. The reviewer's job was to catch the failure that design cannot prevent, a correct excerpt read wrongly. None turned up in that run. That does not mean none ever will, which is why the citation is in the product.
The gate that did not: structured extraction
Alongside the question-and-answer table we have been building a second rail: extracting a fixed set of contract fields with the exact source span for each. The bar we set is strict. For every field that has a gold answer in the public dataset, the model must return one complete span that matches it. Not a paraphrase, not most of it, the span.
On a public control contract the best prompt found all nine gold fields and copied 19 of 24 spans exactly, in 46 seconds, for about 15 cents. Then we ran it again three times without changing anything. It passed twice and failed once: on the third run it joined a warranty quote across 604 characters of intervening text. Replaying the raw model output confirmed the join was the model's, not ours. A result that passes two times in three is not a result.
Later versions of the prompt did better on the control, on an NDA, a lease and a merger agreement, and passed four contracts from the CUAD dataset in strict mode. The fifth, MJ Biotech, failed on the assignment clause and the governing-law clause: the model picked adjacent text instead of the annotated span. Versions five and six each introduced a new failure of their own, a deleted clause marker in one, a wrong section number in the other. Nothing is qualified. The estimated model spend across the batch was under five dollars, so the cost of finding this out was small; the cost of shipping it would not have been.
So structured extraction is not in the product. The review table does not depend on it. When a version passes the full run, including the repeats, this page will say so.
What a local model cost
We tried a 122-billion-parameter open-weight model on our own hardware, on the same control contract. It answered in 90 seconds, used 85 GB of GPU memory, stayed within the output schema, and still missed the audit-record clause. Worse, its citations were shortened with ellipses in places, which means they were not in the source. We did not spend the lawyer and CUAD runs on it. Smaller local models did worse. For now the answers come from a hosted model with retention disabled, and the security page says which.
Retrieval, briefly
Two retrieval experiments are worth a line each. Hybrid search, a lexical pass and a semantic pass fused by reciprocal rank rather than by adding scores, is what runs in production and is what the lawyer journeys were tested on. PageIndex, a tree-structured alternative, failed the quality and indexing gate in both configurations we tried, and we have not adopted it.
Why publish this
Because the person deciding whether to trust a cell is going to test it anyway, and I would rather they start from an accurate picture than a slogan. If you run the trial and find a citation that does not say what the cell says, I want to hear about it: info@doclayer.ai. It goes in the ledger.