Best open System One model
Ranked on the public typed-decisions suite, zero-shot, with LAYA's in-domain fine-tune shown for what a fine-tune buys.
| # | Model | Decisions | Beats baseline | ECE raw → held-out fitted | p50 / case |
|---|---|---|---|---|---|
| 1 | LAYA typed-decisionsConvai Innovations fine-tuned on this suite | 0.714→ 0.762 | 18/20 | 0.161→ 0.016 | 3.5 s† |
| 2 | GLiNER2 largeFastino | 0.486 | 9/20 | 0.300→ 0.029 | 3.8 s† |
| 3 | Vonwfzyx | 0.461 | 5/20 | 0.249→ 0.027 | 597 ms |
| 4 | GLiClass large v3.0Knowledgator correct payload, no prompt | 0.446 | 7/20 | 0.315→ 0.093 | 977 ms |
| 5 | GLiNER 2.5 baseFastino | 0.398 | 3/20 | 0.415→ 0.059 | 1.3 s† |
| 6 | GLiClass multilang ultraKnowledgator correct payload, no prompt | 0.395 | 3/20 | 0.281→ 0.031 | 672 ms |
| 7 | LAYA multilingualConvai Innovations | 0.345 | 2/20 | 0.322→ 0.073 | 1.2 s† |
| 8 | LAYAConvai Innovations | 0.343 | 2/20 | 0.196→ 0.020 | 3.2 s† |
Ranked by decision accuracy. typed-decisions: LAYA's public benchmark: four workflows × 100 JSON states × five typed questions = 2,000 decisions. Synthetic; gold is a teacher distribution and the label is its argmax. Large enough to carry an ordering. Every model is zero-shot except LAYA typed-decisions, which was fine-tuned on this suite. Decisions is the task-weighted mean accuracy of the best configuration per model; ECE is the mean expected calibration error over tasks, and the arrow shows the held-out ECE after one temperature per primitive fitted on the other half. Latency is p50 per three-question customer-service case on four CPU cores (Jev: API round trip); † means taken from the benchmark run rather than a clean timing pass. Updated September 2026.
Same ranking on the other suite: Metatext decisions.
Jev-shaped models not yet run
Surveyed on Hugging Face; each takes the System One request. Their own published figures are on their model pages, attributed; none is comparable to the table above until it has a row.
- Decider 2B1.9B · Apache 2.0
A decoder-backbone System One reproduction with a proper-scoring-rule SFT stage and a calibration-aware RL stage; serves the /v1/systemone request shape.
- open-jev DeBERTa-v3-large435M · Apache 2.0
Single-pass encoder with a per-option head over state and all questions in one sequence; 512-token context with the state cut to 256 tokens.
- Gavel base150M · MIT
Von's mechanism trained deliberately for the decision task, small and multilingual, with a fitted temperature and the card's own negative results.
- Kev 0.6B0.6B · Apache 2.0
A reproduction of the architecture described in "Jev's Architecture Unmasked"; the only one that measured the real Jev on its own frozen dev sets and published a research log.
- Jev schema scorer435M · Apache 2.0
Scores the literal Jev question JSON as the second sequence, so criteria are read at inference.
- System One Qwen3.5 4B scorer4.2B · Non-commercial
Pointwise scorer trained on 46k System One questions across nine families; reports its calibration separately from its accuracy.
- Qwen-Jeff B21.7B · Apache 2.0
Pointwise scorer with a CLI and an "automate if p ≥ 0.9588" gate calibrated to at most 5% error.
- RLCD ModernBERT 151M151M · Apache 2.0
A GLiClass-mechanism model with an RLCD-style objective; small enough to run in the browser (ONNX + WebGPU demo).
Frequently asked questions
What is the best open System One model right now?
Zero-shot: GLiNER2 large. With a fine-tune on the target decisions: LAYA. Neither is a drop-in for Jev without one of those qualifiers.
Related
Every open-weights alternative to TypeSafe's Jev, ranked by measured decision accuracy on the 2,000-decision typed-decisions suite: LAYA, Von, GLiNER2, GLiClass, OpenJEV, plus the Jev-shaped checkpoints not yet run.
The benchmark for System One models: decision models that answer typed questions with calibrated probabilities. Jev and every open alternative on the same 2,000-decision suite, with accuracy, wins over the majority baseline, ECE and latency.
The open reproductions of Jev (OpenJEV, LAYA, Von, and the Kotoba, Decider, Kev, Gavel checkpoints) explained by mechanism and measured on the same suite as the real thing.