GLiNER2 vs GLiClass
Not a bake-off between two classifiers: GLiNER2 is a schema-conditioned encoder with a native label-description slot and a span head; GLiClass is a label-relevance classifier with neither. They see the question differently and need different payloads.
DeBERTa-v3 (span grid) · 340M · Apache 2.0
DeBERTa-v3-large · 439M · Apache 2.0
Metatext decisions
Three hand-written workflows (support tickets, security incidents, agent traces), seven questions, about 123 judgments per model. Small, so treat a ±0.1 difference as a tie; every question has a majority baseline the model has to beat.
| Metric | GLiNER2 large | GLiClass large v3.0 |
|---|---|---|
| Decision accuracy | 0.716 | 0.642 |
| Questions over baseline | 6/7 | 5/7 |
| ECE raw → held-out fitted | 0.261 | 0.250 |
| p50 per case | 627 ms | 977 ms |
| Extraction F1 | 0.968 | n/a |
Per-question accuracy
| Task | Question | Type | Baseline | GLiNER2 large | GLiClass large v3.0 |
|---|---|---|---|---|---|
| agent_traces | outcome | choice | 0.357 | 0.571 | 0.429 |
| agent_traces | needs_human | noul | 0.714 | 0.643 | 0.714 |
| customer_service | department | choice | 0.400 | 0.850 | 0.850 |
| customer_service | refund_requested | noul | 0.700 | 0.950 | 0.950 |
| customer_service | frustration | score | 0.500 | 0.950 | 0.200 |
| security_incidents | severity | choice | 0.312 | 0.438 | 0.625 |
| security_incidents | actively_exploited | noul | 0.688 | 0.812 | 0.750 |
typed-decisions
LAYA's public benchmark: four workflows × 100 JSON states × five typed questions = 2,000 decisions. Synthetic; gold is a teacher distribution and the label is its argmax. Large enough to carry an ordering.
| Metric | GLiNER2 large | GLiClass large v3.0 |
|---|---|---|
| Decision accuracy | 0.486 | 0.446 |
| Questions over baseline | 9/20 | 7/20 |
| ECE raw → held-out fitted | 0.300 → 0.029 | 0.315 → 0.093 |
| p50 per case | 3.8 s | 977 ms |
Per-question accuracy
| Task | Question | Type | Baseline | GLiNER2 large | GLiClass large v3.0 |
|---|---|---|---|---|---|
| td_agent_trace_observability | action | choice | 0.360 | 0.310 | 0.310 |
| td_agent_trace_observability | needs_review | noul | 0.550 | 0.450 | 0.560 |
| td_agent_trace_observability | outcome | choice | 0.500 | 0.250 | 0.510 |
| td_agent_trace_observability | risk | score | 0.340 | 0.460 | 0.340 |
| td_agent_trace_observability | urgency | score | 0.370 | 0.300 | 0.350 |
| td_customer_service | action | choice | 0.520 | 0.470 | 0.240 |
| td_customer_service | category | choice | 0.380 | 0.740 | 0.730 |
| td_customer_service | churn_risk | score | 0.470 | 0.660 | 0.560 |
| td_customer_service | needs_human | noul | 0.670 | 0.700 | 0.330 |
| td_customer_service | urgency | score | 0.420 | 0.610 | 0.740 |
| td_invoice_processing | discrepancy_severity | score | 0.470 | 0.160 | 0.150 |
| td_invoice_processing | disposition | choice | 0.450 | 0.450 | 0.450 |
| td_invoice_processing | duplicate | noul | 0.890 | 0.110 | 0.110 |
| td_invoice_processing | matches_order | noul | 0.540 | 0.620 | 0.540 |
| td_invoice_processing | urgency | score | 0.510 | 0.320 | 0.690 |
| td_security_incidents | credential_compromise | noul | 0.550 | 0.580 | 0.400 |
| td_security_incidents | disposition | choice | 0.740 | 0.740 | 0.330 |
| td_security_incidents | severity | score | 0.490 | 0.510 | 0.510 |
| td_security_incidents | true_positive | noul | 0.730 | 0.730 | 0.680 |
| td_security_incidents | urgency | score | 0.500 | 0.540 | 0.390 |
Verdict
GLiNER2 on decisions (0.716 vs 0.642), on Score (which GLiClass cannot answer), on extraction (0.968 F1 vs none). GLiClass base is the better-calibrated and faster option at the base size; GLiClass edge is the latency floor of either family.
Related
Jev against GLiNER2 large: the best zero-shot open model on decisions, and the only one that also extracts spans.
Jev against GLiClass large v3.0 with the correct payload.
Micro-F1 on an eight-case entity extraction task for the models that have a span head. Jev, LAYA, Von and GLiClass never return spans; only the GLiNER family does.