Are the payment terms stated? Is there a damages cap? Batch the checklist and list what came back false.
A compliance checklist is a batch of yes/no questions. Run them in one request against the document and let the code list what is missing — then extract the fields the checklist referred to, with offsets, so the reviewer lands on the clause.
In development — the benchmark numbers on this page are measured; the Decide-1 model is in training.
Checklists are read by people
Slowly, and inconsistently between reviewers.
An LLM summarises instead of answering
You wanted twenty booleans; you got a paragraph.
Nobody can point at the clause
A yes without an offset is not auditable.
- 1
One noul per checklist item
Batched; the whole checklist is one call.
- 2
Extract in the same request
Amounts, dates, parties, clause spans — with offsets.
- 3
Flag the falses
The reviewer sees the misses and the spans, not the document.
CHECKLIST = {
"payment_terms": {"type": "noul", "instructions": "Payment terms are stated."},
"delivery_date": {"type": "noul", "instructions": "A delivery date is stated."},
"damages_cap": {"type": "noul", "instructions": "Liability is capped."},
# … up to 20
}
r = client.system_one(state=contract_text, questions=CHECKLIST)
missing = [k for k, a in r.answers.items() if a.noul < 0.5]- Noul — a calibrated yes/no probability
- Extract — spans and records with offsets
Pattern from oflight.co.jp’s 12-use-case guide. On the invoice-processing suite (100 states × 5 questions) Jev scores 0.756 zero-shot and the fine-tuned open encoder 0.734; the “duplicate invoice” noul is where zero-shot open models fail worst (0.11 vs a 0.89 baseline).
Numbers about Jev are its authors' or the cited source's; numbers from the benchmark are ours, on one harness, with the raw JSON published. Full table.
Get early access — and tell us which decision you'd hand over first
We're building Metatext Decide with the first teams that use it. Leave your email, answer as much as you like, and say yes to a short interview if you want a say in what ships.
- Early access to Decide as it opens up
- A 20-minute interview if you want one — research, not a pitch
- The benchmark harness and raw results, today
- No spam — early-access updates only
Questions
- How long a document?
- Open encoders: 512–1,024 tokens per call today; chunk by clause. Jev: 32k. We say which is which.