LAYA multilingual

Name: LAYA multilingual
Author: Convai Innovations

Convai InnovationsOpen weights

The 100+ language LAYA checkpoint on an mmBERT-base backbone. Ships with temperature [1, 1, 1], so it runs uncalibrated out of the box.

Specifications

Per-question accuracy — default payload, systemone mode, 120 judgments

Task	Question	Type	Accuracy	Baseline	Beats?
agent_traces	outcome	choice	0.143	0.357	❌
agent_traces	needs_human	noul	0.500	0.714	❌
customer_service	department	choice	0.650	0.400	✅
customer_service	refund_requested	noul	0.650	0.700	❌
customer_service	frustration	score	0.300	0.500	❌
security_incidents	severity	choice	0.375	0.312	✅
security_incidents	actively_exploited	noul	0.500	0.688	❌

Per-question accuracy — default payload, systemone mode, 2,000 judgments

Task	Question	Type	Accuracy	Baseline	Beats?
td_agent_trace_observability	action	choice	0.060	0.360	❌
td_agent_trace_observability	needs_review	noul	0.510	0.550	❌
td_agent_trace_observability	outcome	choice	0.330	0.500	❌
td_agent_trace_observability	risk	score	0.210	0.340	❌
td_agent_trace_observability	urgency	score	0.130	0.370	❌
td_customer_service	action	choice	0.100	0.520	❌
td_customer_service	category	choice	0.730	0.380	✅
td_customer_service	churn_risk	score	0.340	0.470	❌
td_customer_service	needs_human	noul	0.530	0.670	❌
td_customer_service	urgency	score	0.380	0.420	❌
td_invoice_processing	discrepancy_severity	score	0.160	0.470	❌
td_invoice_processing	disposition	choice	0.410	0.450	❌
td_invoice_processing	duplicate	noul	0.110	0.890	❌
td_invoice_processing	matches_order	noul	0.540	0.540	❌
td_invoice_processing	urgency	score	0.330	0.510	❌
td_security_incidents	credential_compromise	noul	0.580	0.550	✅
td_security_incidents	disposition	choice	0.140	0.740	❌
td_security_incidents	severity	score	0.270	0.490	❌
td_security_incidents	true_positive	noul	0.710	0.730	❌
td_security_incidents	urgency	score	0.320	0.500	❌

Fitted temperature per primitive (fold 0, judged on fold 1): choice 7, score 4, noul 7 — ECE 0.347 → 0.073, accuracy → 0.329.

Reported by the authors or by third parties, on their own suites. Not comparable to the leaderboard above; attributed so you can check them.

Third partyLargitData · 2026-09-20

Measured on: 100 multi-turn RAG-routing decisions (20 conversations × 5), multilingual

Third-party, per-item logs on GitHub.

Model cardConvai Innovations

Measured on: typed-decisions test split (400 cases, 2,000 decisions)

Author-reported; the Jev row is quoted as "published", not measured by them. Trained on the suite's own train split.

Model cardConvai Innovations

Measured on: MASSIVE (51 languages, 20-way), XNLI

MASSIVE macro accuracy / ECE: 0.366 / 0.387 (ships uncalibrated; 0.106 after refit)
XNLI: en 0.843, 14 other languages 0.731

Author-reported.

How does LAYA multilingual compare to Jev?

On typed-decisions LAYA multilingual scores 0.345 against Jev's 0.726, beating the majority baseline on 2 of 20 questions (Jev: 18/20).

Is LAYA multilingual open source?

Yes. convaiinnovations/laya-multilingual is released under Apache 2.0 and runs on your own hardware with the laya library.

Can LAYA multilingual extract entities?

No. LAYA multilingual answers questions about text and never returns spans; use a GLiNER-family model for extraction.