LAYA vs Von
Both speak the /v1/systemone shape and both are ModernBERT-large underneath. LAYA adds a decision head trained with RLCD; Von borrows an NLI head. Same backbone, different head, measurable difference.
ModernBERT-large + decision head (RLCD) · 421M · Apache 2.0
ModernBERT-large NLI head · 395M · Apache 2.0
Metatext decisions
Three hand-written workflows (support tickets, security incidents, agent traces), seven questions, about 123 judgments per model. Small, so treat a ±0.1 difference as a tie; every question has a majority baseline the model has to beat.
| Metric | LAYA | Von |
|---|---|---|
| Decision accuracy | 0.520 | 0.509 |
| Questions over baseline | 3/7 | 4/7 |
| ECE raw → held-out fitted | 0.064 | 0.234 → 0.122 |
| p50 per case | 378 ms | 597 ms |
| Extraction F1 | n/a | n/a |
Per-question accuracy
| Task | Question | Type | Baseline | LAYA | Von |
|---|---|---|---|---|---|
| agent_traces | outcome | choice | 0.357 | 0.357 | 0.500 |
| agent_traces | needs_human | noul | 0.714 | 0.429 | 0.286 |
| customer_service | department | choice | 0.400 | 0.750 | 0.650 |
| customer_service | refund_requested | noul | 0.700 | 0.850 | 0.950 |
| customer_service | frustration | score | 0.500 | 0.400 | 0.300 |
| security_incidents | severity | choice | 0.312 | 0.562 | 0.562 |
| security_incidents | actively_exploited | noul | 0.688 | 0.438 | 0.438 |
typed-decisions
LAYA's public benchmark: four workflows × 100 JSON states × five typed questions = 2,000 decisions. Synthetic; gold is a teacher distribution and the label is its argmax. Large enough to carry an ordering.
| Metric | LAYA | Von |
|---|---|---|
| Decision accuracy | 0.343 | 0.461 |
| Questions over baseline | 2/20 | 5/20 |
| ECE raw → held-out fitted | 0.196 → 0.020 | 0.249 → 0.027 |
| p50 per case | 3.2 s | 597 ms |
Per-question accuracy
| Task | Question | Type | Baseline | LAYA | Von |
|---|---|---|---|---|---|
| td_agent_trace_observability | action | choice | 0.360 | 0.310 | 0.320 |
| td_agent_trace_observability | needs_review | noul | 0.550 | 0.580 | 0.450 |
| td_agent_trace_observability | outcome | choice | 0.500 | 0.400 | 0.440 |
| td_agent_trace_observability | risk | score | 0.340 | 0.340 | 0.370 |
| td_agent_trace_observability | urgency | score | 0.370 | 0.370 | 0.330 |
| td_customer_service | action | choice | 0.520 | 0.050 | 0.260 |
| td_customer_service | category | choice | 0.380 | 0.480 | 0.720 |
| td_customer_service | churn_risk | score | 0.470 | 0.380 | 0.680 |
| td_customer_service | needs_human | noul | 0.670 | 0.670 | 0.670 |
| td_customer_service | urgency | score | 0.420 | 0.300 | 0.610 |
| td_invoice_processing | discrepancy_severity | score | 0.470 | 0.010 | 0.450 |
| td_invoice_processing | disposition | choice | 0.450 | 0.440 | 0.470 |
| td_invoice_processing | duplicate | noul | 0.890 | 0.110 | 0.110 |
| td_invoice_processing | matches_order | noul | 0.540 | 0.490 | 0.540 |
| td_invoice_processing | urgency | score | 0.510 | 0.320 | 0.490 |
| td_security_incidents | credential_compromise | noul | 0.550 | 0.550 | 0.550 |
| td_security_incidents | disposition | choice | 0.740 | 0.050 | 0.280 |
| td_security_incidents | severity | score | 0.490 | 0.310 | 0.420 |
| td_security_incidents | true_positive | noul | 0.730 | 0.520 | 0.730 |
| td_security_incidents | urgency | score | 0.500 | 0.170 | 0.340 |
Verdict
On accuracy they trade: LAYA 0.520 vs Von 0.509 on the Metatext suite, Von 0.461 vs LAYA 0.343 on typed-decisions (where LAYA collapses onto "do nothing"). On calibration LAYA wins clearly (ECE 0.064 vs 0.234) because it ships a fitted temperature and Von runs at T = 1.0 from the Hub. LAYA is also the only one with a fine-tuned sibling that reaches Jev.
Related
Jev against LAYA on two suites: accuracy, calibration, latency, and what LAYA's in-domain fine-tune changes.
Jev against Von, the System One API over a ModernBERT-large NLI head.
The open reproductions of Jev (OpenJEV, LAYA, Von, and the Kotoba, Decider, Kev, Gavel checkpoints) explained by mechanism and measured on the same suite as the real thing.