Open-source Jev
Three mechanisms claim to be an open Jev: an encoder with a decision head (LAYA), an NLI entailment head with one hypothesis per option (Von, Gavel), and log-probability scoring over a chat LLM (OpenJEV). Measured on one exam, the mechanism turns out not to be the advantage: copying it buys calibration at best, not accuracy.
| # | Model | Decisions | Beats baseline | ECE raw → held-out fitted | p50 / case |
|---|---|---|---|---|---|
| 1 | LAYA typed-decisionsConvai Innovations fine-tuned on this suite | 0.714→ 0.762 | 18/20 | 0.161→ 0.016 | 3.5 s† |
| 2 | GLiNER2 largeFastino | 0.486 | 9/20 | 0.300→ 0.029 | 3.8 s† |
| 3 | Vonwfzyx | 0.461 | 5/20 | 0.249→ 0.027 | 597 ms |
| 4 | GLiClass large v3.0Knowledgator correct payload, no prompt | 0.446 | 7/20 | 0.315→ 0.093 | 977 ms |
| 5 | GLiNER 2.5 baseFastino | 0.398 | 3/20 | 0.415→ 0.059 | 1.3 s† |
| 6 | GLiClass multilang ultraKnowledgator correct payload, no prompt | 0.395 | 3/20 | 0.281→ 0.031 | 672 ms |
| 7 | LAYA multilingualConvai Innovations | 0.345 | 2/20 | 0.322→ 0.073 | 1.2 s† |
| 8 | LAYAConvai Innovations | 0.343 | 2/20 | 0.196→ 0.020 | 3.2 s† |
Ranked by decision accuracy. typed-decisions: LAYA's public benchmark: four workflows × 100 JSON states × five typed questions = 2,000 decisions. Synthetic; gold is a teacher distribution and the label is its argmax. Large enough to carry an ordering. Every model is zero-shot except LAYA typed-decisions, which was fine-tuned on this suite. Decisions is the task-weighted mean accuracy of the best configuration per model; ECE is the mean expected calibration error over tasks, and the arrow shows the held-out ECE after one temperature per primitive fitted on the other half. Latency is p50 per three-question customer-service case on four CPU cores (Jev: API round trip); † means taken from the benchmark run rather than a clean timing pass. Updated September 2026.
Same ranking on the other suite: Metatext decisions.
Jev-shaped models not yet run
Surveyed on Hugging Face; each takes the System One request. Their own published figures are on their model pages, attributed; none is comparable to the table above until it has a row.
- Decider 2B1.9B · Apache 2.0
A decoder-backbone System One reproduction with a proper-scoring-rule SFT stage and a calibration-aware RL stage; serves the /v1/systemone request shape.
- open-jev DeBERTa-v3-large435M · Apache 2.0
Single-pass encoder with a per-option head over state and all questions in one sequence; 512-token context with the state cut to 256 tokens.
- Gavel base150M · MIT
Von's mechanism trained deliberately for the decision task, small and multilingual, with a fitted temperature and the card's own negative results.
- Kev 0.6B0.6B · Apache 2.0
A reproduction of the architecture described in "Jev's Architecture Unmasked"; the only one that measured the real Jev on its own frozen dev sets and published a research log.
- Jev schema scorer435M · Apache 2.0
Scores the literal Jev question JSON as the second sequence, so criteria are read at inference.
- System One Qwen3.5 4B scorer4.2B · Non-commercial
Pointwise scorer trained on 46k System One questions across nine families; reports its calibration separately from its accuracy.
- Qwen-Jeff B21.7B · Apache 2.0
Pointwise scorer with a CLI and an "automate if p ≥ 0.9588" gate calibrated to at most 5% error.
- RLCD ModernBERT 151M151M · Apache 2.0
A GLiClass-mechanism model with an RLCD-style objective; small enough to run in the browser (ONNX + WebGPU demo).
Frequently asked questions
Is there an open-source Jev?
Several projects reproduce the request shape and the API. None reproduces the zero-shot accuracy: the closest open model on typed-decisions is at 0.486 against Jev's 0.726. LAYA fine-tuned on the suite reaches 0.714, which is the honest answer: open Jev exists after a fine-tune on your traffic.
What is OpenJEV?
The open-source implementation of Jev's mechanism: score each allowed answer by its log-probability under a chat LLM. On a 0.5B backbone it scores 0.427 on the Metatext suite and takes about 50 seconds per case on CPU.
Related
Every open-weights alternative to TypeSafe's Jev, ranked by measured decision accuracy on the 2,000-decision typed-decisions suite: LAYA, Von, GLiNER2, GLiClass, OpenJEV, plus the Jev-shaped checkpoints not yet run.
Jev against OpenJEV, the open implementation of its log-probability scoring mechanism on a 0.5B LLM.
Jev against LAYA on two suites: accuracy, calibration, latency, and what LAYA's in-domain fine-tune changes.
Jev against Von, the System One API over a ModernBERT-large NLI head.