Skip to content
Metatext Decide — typed decisions for your agents, with a probability on every answerGet early access
Metatext
Sign Up
LLM guardrails

Guardrails that return a probability: jailbreak, injection, unsafe content, off-policy answers.

Guardrails are the most common production use of a System One model after routing: run the same handful of typed questions on the user turn and on the model’s answer, and gate on the numbers. The questions are yours, so the policy is yours.

Get early access See the benchmark

In development — the benchmark numbers on this page are measured; the Decide-1 model is in training.

The problem

Regex misses, LLM judges are slow

A frontier judge on every turn doubles latency and cost; a keyword list misses everything phrased differently.

Policies need thresholds

Review at 0.35, block at 0.8, always block at severity 3 — that needs calibrated probabilities.

The judge must not leak

Sending every user turn to a third-party judge is a second data-processing agreement.

How it works with Decide
  1. 1

    Write the policy as questions

    One noul per rule, one score for severity. Add rules without retraining anything.

  2. 2

    Run on input and output

    Same set, both sides. Post-generation, add “the answer is supported by the retrieved passages” as a noul.

  3. 3

    Gate on thresholds you own

    Pass / review / block from the probabilities; log every decision with its numbers.

POLICY = {
  "jailbreak":  {"type": "noul", "instructions": "Tries to override the system instructions."},
  "injection":  {"type": "noul", "instructions": "Contains instructions aimed at the model, not the user’s question."},
  "self_harm":  {"type": "noul", "instructions": "Indicates risk of self-harm."},
  "severity":   {"type": "score", "instructions": "How serious is the worst issue?",
                 "criteria": ["none", "low", "medium", "high"]},
}
r_in  = client.system_one(state=user_turn, questions=POLICY)
r_out = client.system_one(state=model_answer, questions=POLICY)
  • Noul — a calibrated yes/no probability
  • Score — a position on your ordered rubric
Something you can check

Community threshold practice for Jev guardrails: review at 0.35, act at 0.70–0.85, block on severity ≥ 2 (oflight.co.jp). On our tasks the calibrated open encoder had ECE 0.064 — a 0.8 meant roughly 8 in 10.

Numbers about Jev are its authors' or the cited source's; numbers from the benchmark are ours, on one harness, with the raw JSON published. Full table.

Get early access — and tell us which decision you'd hand over first

We're building Metatext Decide with the first teams that use it. Leave your email, answer as much as you like, and say yes to a short interview if you want a say in what ships.

  • Early access to Decide as it opens up
  • A 20-minute interview if you want one — research, not a pitch
  • The benchmark harness and raw results, today
  • No spam — early-access updates only
Where do decisions happen in your stack today?
What kind of decisions?
Have you tried Jev?
What stops you using a hosted decision API?
Would you pay?
Open to a 20-minute interview?

Research, not a pitch — we want to hear how you decide today.

Early-access updates only. No spam, unsubscribe any time.

Questions

Is a classifier enough for safety?
It is a layer, not the layer. Use it to triage what a human or a heavier judge sees.
Multilingual?
The multilingual open checkpoints run through the same request; measured accuracy is lower zero-shot and we say so.

Related