The best GLiClass checkpoint on Knowledgator's own zero-shot benchmark. Labels are prepended to the text and scored by relevance, so it needs the Jev criteria folded into each label and one hypothesis per Noul question to work.
Score is the one primitive GLiClass cannot answer: its output is relevance, not an ordered level.
| Configuration | Mode | Decisions | Beats baseline | ECE |
|---|---|---|---|---|
| correct payload, no promptheadline | serial | 0.642 | 5/7 | 0.250 |
| criteria + single-hypothesis Noul (correct payload) | serial | 0.621 | 5/7 | 0.241 |
| default payload | serial | 0.424 | 3/7 | 0.510 |
| criteria folded into labels | serial | 0.404 | 3/7 | 0.494 |
| Task | Question | Type | Accuracy | Baseline | Beats? |
|---|---|---|---|---|---|
| agent_traces | outcome | choice | 0.429 | 0.357 | ✅ |
| agent_traces | needs_human | noul | 0.714 | 0.714 | ❌ |
| customer_service | department | choice | 0.850 | 0.400 | ✅ |
| customer_service | refund_requested | noul | 0.950 | 0.700 | ✅ |
| customer_service | frustration | score | 0.200 | 0.500 | ❌ |
| security_incidents | severity | choice | 0.625 | 0.312 | ✅ |
| security_incidents | actively_exploited | noul | 0.750 | 0.688 | ✅ |
| Task | Question | Type | Accuracy | Baseline | Beats? |
|---|---|---|---|---|---|
| td_agent_trace_observability | action | choice | 0.310 | 0.360 | ❌ |
| td_agent_trace_observability | needs_review | noul | 0.560 | 0.550 | ✅ |
| td_agent_trace_observability | outcome | choice | 0.510 | 0.500 | ✅ |
| td_agent_trace_observability | risk | score | 0.340 | 0.340 | ❌ |
| td_agent_trace_observability | urgency | score | 0.350 | 0.370 | ❌ |
| td_customer_service | action | choice | 0.240 | 0.520 | ❌ |
| td_customer_service | category | choice | 0.730 | 0.380 | ✅ |
| td_customer_service | churn_risk | score | 0.560 | 0.470 | ✅ |
| td_customer_service | needs_human | noul | 0.330 | 0.670 | ❌ |
| td_customer_service | urgency | score | 0.740 | 0.420 | ✅ |
| td_invoice_processing | discrepancy_severity | score | 0.150 | 0.470 | ❌ |
| td_invoice_processing | disposition | choice | 0.450 | 0.450 | ❌ |
| td_invoice_processing | duplicate | noul | 0.110 | 0.890 | ❌ |
| td_invoice_processing | matches_order | noul | 0.540 | 0.540 | ❌ |
| td_invoice_processing | urgency | score | 0.690 | 0.510 | ✅ |
| td_security_incidents | credential_compromise | noul | 0.400 | 0.550 | ❌ |
| td_security_incidents | disposition | choice | 0.330 | 0.740 | ❌ |
| td_security_incidents | severity | score | 0.510 | 0.490 | ✅ |
| td_security_incidents | true_positive | noul | 0.680 | 0.730 | ❌ |
| td_security_incidents | urgency | score | 0.390 | 0.500 | ❌ |
Fitted temperature per primitive (fold 0, judged on fold 1): choice 7, score 1, noul 7 — ECE 0.293 → 0.093, accuracy → 0.456.
Reported by the authors or by third parties, on their own suites. Not comparable to the leaderboard above; attributed so you can check them.
Measured on: 14-dataset zero-shot classification average (F1)
Author-reported; standard text-classification sets, not decisions.
On typed-decisions GLiClass large v3.0 scores 0.446 against Jev's 0.726, beating the majority baseline on 7 of 20 questions (Jev: 18/20).
Yes. knowledgator/gliclass-large-v3.0 is released under Apache 2.0 and runs on your own hardware with the gliclass library.
No. GLiClass large v3.0 answers questions about text and never returns spans; use a GLiNER-family model for extraction.