Jev alternatives
The Jev request shape is now a commodity: at least nine open reproductions appeared on Hugging Face in the ten days after launch. What is not a commodity is the accuracy. This page ranks the ones we have measured and lists the ones we have not, with their own published claims attributed.
| # | Model | Decisions | Beats baseline | ECE raw → held-out fitted | p50 / case |
|---|---|---|---|---|---|
| 1 | LAYA typed-decisionsConvai Innovations fine-tuned on this suite | 0.714→ 0.762 | 18/20 | 0.161→ 0.016 | 3.5 s† |
| 2 | GLiNER2 largeFastino | 0.486 | 9/20 | 0.300→ 0.029 | 3.8 s† |
| 3 | Vonwfzyx | 0.461 | 5/20 | 0.249→ 0.027 | 597 ms |
| 4 | GLiClass large v3.0Knowledgator correct payload, no prompt | 0.446 | 7/20 | 0.315→ 0.093 | 977 ms |
| 5 | GLiNER 2.5 baseFastino | 0.398 | 3/20 | 0.415→ 0.059 | 1.3 s† |
| 6 | GLiClass multilang ultraKnowledgator correct payload, no prompt | 0.395 | 3/20 | 0.281→ 0.031 | 672 ms |
| 7 | LAYA multilingualConvai Innovations | 0.345 | 2/20 | 0.322→ 0.073 | 1.2 s† |
| 8 | LAYAConvai Innovations | 0.343 | 2/20 | 0.196→ 0.020 | 3.2 s† |
Ranked by decision accuracy. typed-decisions: LAYA's public benchmark: four workflows × 100 JSON states × five typed questions = 2,000 decisions. Synthetic; gold is a teacher distribution and the label is its argmax. Large enough to carry an ordering. Every model is zero-shot except LAYA typed-decisions, which was fine-tuned on this suite. Decisions is the task-weighted mean accuracy of the best configuration per model; ECE is the mean expected calibration error over tasks, and the arrow shows the held-out ECE after one temperature per primitive fitted on the other half. Latency is p50 per three-question customer-service case on four CPU cores (Jev: API round trip); † means taken from the benchmark run rather than a clean timing pass. Updated September 2026.
Same ranking on the other suite: Metatext decisions.
Jev-shaped models not yet run
Surveyed on Hugging Face; each takes the System One request. Their own published figures are on their model pages, attributed; none is comparable to the table above until it has a row.
- Decider 2B1.9B · Apache 2.0
A decoder-backbone System One reproduction with a proper-scoring-rule SFT stage and a calibration-aware RL stage; serves the /v1/systemone request shape.
- open-jev DeBERTa-v3-large435M · Apache 2.0
Single-pass encoder with a per-option head over state and all questions in one sequence; 512-token context with the state cut to 256 tokens.
- Gavel base150M · MIT
Von's mechanism trained deliberately for the decision task, small and multilingual, with a fitted temperature and the card's own negative results.
- Kev 0.6B0.6B · Apache 2.0
A reproduction of the architecture described in "Jev's Architecture Unmasked"; the only one that measured the real Jev on its own frozen dev sets and published a research log.
- Jev schema scorer435M · Apache 2.0
Scores the literal Jev question JSON as the second sequence, so criteria are read at inference.
- System One Qwen3.5 4B scorer4.2B · Non-commercial
Pointwise scorer trained on 46k System One questions across nine families; reports its calibration separately from its accuracy.
- Qwen-Jeff B21.7B · Apache 2.0
Pointwise scorer with a CLI and an "automate if p ≥ 0.9588" gate calibrated to at most 5% error.
- RLCD ModernBERT 151M151M · Apache 2.0
A GLiClass-mechanism model with an RLCD-style objective; small enough to run in the browser (ONNX + WebGPU demo).
Frequently asked questions
What is the best open-source alternative to Jev?
Zero-shot, GLiNER2 large (0.486 on typed-decisions, 0.716 on the Metatext suite where it also extracts spans). If you can fine-tune on your own decisions, LAYA: its in-domain checkpoint reaches 0.714, level with Jev.
Which alternatives take the Jev request unchanged?
LAYA, Von, OpenJEV and the surveyed Decider, Kev, Gavel and open-jev checkpoints all speak the /v1/systemone shape (state + typed questions). GLiNER2 maps one to one onto its classification schema; GLiClass needs the criteria folded into each label and one hypothesis per Noul question.
Why is LAYA so low zero-shot when its card says 83.8%?
The card's 83.8% is on LAYA's own in-task families. On typed-decisions zero-shot it scores 0.343 in our harness (0.362 on its card) because it collapses onto the "do nothing" option. Its fine-tuned sibling is the one that matches Jev.
Related
An independent benchmark of TypeSafe's Jev against every open System One alternative: decision accuracy, calibration (ECE) and latency on two suites, 2,123 judgments, same harness for every row.
The open reproductions of Jev (OpenJEV, LAYA, Von, and the Kotoba, Decider, Kev, Gavel checkpoints) explained by mechanism and measured on the same suite as the real thing.
Decision models with open weights and a permissive license, ranked by measured accuracy with their latency on four CPU cores, for teams that cannot send their state to a hosted API.
Jev against LAYA on two suites: accuracy, calibration, latency, and what LAYA's in-domain fine-tune changes.