LAYA typed-decisions

Name: LAYA typed-decisions
Author: Convai Innovations

Convai InnovationsOpen weights

The English LAYA checkpoint fine-tuned on the typed-decisions suite. In-domain on that suite by construction; the number to read is how far a fine-tune closes the gap to Jev.

Not a zero-shot row on typed-decisions: the workflow names match its training data.

Hugging Face

typed-decisions: 0.714
Metatext suite: 0.475
ECE untuned: 0.161
p50 per case: 375 ms

Specifications

Checkpoint: convaiinnovations/laya-typed-decisions
Publisher: Convai Innovations
Mechanism: Encoder with a decision head (System One)
Backbone: ModernBERT-large + decision head, fine-tuned on typed-decisions
Parameters: 421M
License: Apache 2.0
Runs as: open weights, self-hosted
Library: laya
Primitives: Choice, Score, Noul
Pricing: none — your compute

Metatext decisionsrank 16 of 21

Full leaderboard →

Per-question accuracy — default payload, systemone mode, 120 judgments

Task	Question	Type	Accuracy	Baseline	Beats?
agent_traces	outcome	choice	0.214	0.357	❌
agent_traces	needs_human	noul	0.357	0.714	❌
customer_service	department	choice	0.850	0.400	✅
customer_service	refund_requested	noul	0.850	0.700	✅
customer_service	frustration	score	0.400	0.500	❌
security_incidents	severity	choice	0.438	0.312	✅
security_incidents	actively_exploited	noul	0.438	0.688	❌

typed-decisionsrank 2 of 9

Full leaderboard →

Per-question accuracy — default payload, systemone mode, 2,000 judgments

Task	Question	Type	Accuracy	Baseline	Beats?
td_agent_trace_observability	action	choice	0.700	0.360	✅
td_agent_trace_observability	needs_review	noul	0.840	0.550	✅
td_agent_trace_observability	outcome	choice	0.650	0.500	✅
td_agent_trace_observability	risk	score	0.600	0.340	✅
td_agent_trace_observability	urgency	score	0.450	0.370	✅
td_customer_service	action	choice	0.580	0.520	✅
td_customer_service	category	choice	0.940	0.380	✅
td_customer_service	churn_risk	score	0.780	0.470	✅
td_customer_service	needs_human	noul	0.760	0.670	✅
td_customer_service	urgency	score	0.670	0.420	✅
td_invoice_processing	discrepancy_severity	score	0.400	0.470	❌
td_invoice_processing	disposition	choice	0.800	0.450	✅
td_invoice_processing	duplicate	noul	0.920	0.890	✅
td_invoice_processing	matches_order	noul	0.940	0.540	✅
td_invoice_processing	urgency	score	0.610	0.510	✅
td_security_incidents	credential_compromise	noul	0.810	0.550	✅
td_security_incidents	disposition	choice	0.730	0.740	❌
td_security_incidents	severity	score	0.670	0.490	✅
td_security_incidents	true_positive	noul	0.870	0.730	✅
td_security_incidents	urgency	score	0.560	0.500	✅

Fitted temperature per primitive (fold 0, judged on fold 1): choice 0.3, score 0.4, noul 0.3 — ECE 0.155 → 0.016, accuracy → 0.762.

Published numbers

Reported by the authors or by third parties, on their own suites. Not comparable to the leaderboard above; attributed so you can check them.

Third partyLuni

laya-jev-benchmark

Measured on: PhishNChips (2,000 emails) and typed-decisions

typed-decisions: 0.767 (ECE 0.212) vs Jev 0.727 (ECE 0.144)

Third-party scripts; the Jev rows are quoted from elsewhere, not measured here.

Model cardConvai Innovations

LAYA typed-decisions model card

Measured on: typed-decisions test split (400 cases, 2,000 decisions)

Accuracy / ECE / score MAE: 0.766 / 0.213 / 0.242

Author-reported; the Jev row is quoted as "published", not measured by them. Trained on the suite's own train split.

Frequently asked questions

How does LAYA typed-decisions compare to Jev?

On typed-decisions LAYA typed-decisions scores 0.714 against Jev's 0.726, beating the majority baseline on 18 of 20 questions (Jev: 18/20).

Is LAYA typed-decisions open source?

Yes. convaiinnovations/laya-typed-decisions is released under Apache 2.0 and runs on your own hardware with the laya library.

Can LAYA typed-decisions extract entities?

No. LAYA typed-decisions answers questions about text and never returns spans; use a GLiNER-family model for extraction.