Skip to content
Metatext Decide — typed decisions for your agents, with a probability on every answerGet early access
Metatext
Sign Up
workflow

Customer service triage

Support tickets: which team, is a refund requested, how frustrated, churn risk, urgency, does a human need to step in. The domain TypeSafe uses in its own docs.

On Metatext decisions, Jev leads with 0.933 mean accuracy against a 0.533 majority baseline.

Metatext decisions

Every model is zero-shot on this suite.

#ModelMean accuracyBaselineBeats baseline
1JevTypeSafe0.9330.5333/3
2GLiNER2 largeFastino0.9170.5333/3
3GLiNER2 baseFastino0.7500.5332/3
4GLiNER 2.5 baseFastino0.7330.5332/3
5GLiClass modern large v3.0Knowledgator0.7170.5332/3
6LAYA typed-decisionsConvai Innovations0.7000.5332/3
7GLiClass multilang ultraKnowledgator0.6830.5332/3
8GLiNER 2.5 smallFastino0.6830.5332/3
9GLiClass large v3.0Knowledgator0.6670.5332/3
10LAYAConvai Innovations0.6670.5332/3
11Vonwfzyx0.6330.5332/3
12GLiClass base v3.0Knowledgator0.6170.5332/3
13GLiClass instruct base v1.0Knowledgator0.6000.5331/3
14GLiNER 2.5 multiFastino0.6000.5332/3
15GLiClass instruct large v1.0Knowledgator0.5670.5332/3
16GLiNER multitask v1.0Knowledgator0.5500.5331/3
17LAYA multilingualConvai Innovations0.5330.5331/3
18GLiGuard 300MFastino0.5170.5331/3
19GLiNER2 multiFastino0.5170.5331/3
20GLiClass edge v3.0Knowledgator0.5170.5332/3
21OpenJEVopen-jev0.4830.5331/3

typed-decisions

Every model is zero-shot except LAYA typed-decisions, which was fine-tuned on this suite.

#ModelMean accuracyBaselineBeats baseline
1JevTypeSafe0.7900.4924/5
2LAYA typed-decisionsConvai Innovations0.7460.4925/5
3GLiNER2 largeFastino0.6360.4924/5
4Vonwfzyx0.5880.4923/5
5GLiClass multilang ultraKnowledgator0.5520.4923/5
6GLiNER 2.5 baseFastino0.5500.4923/5
7GLiClass large v3.0Knowledgator0.5200.4923/5
8LAYA multilingualConvai Innovations0.4160.4921/5
9LAYAConvai Innovations0.3760.4921/5

The questions

  • customer_service.department · choice · our tasks
    Which team should handle this request?
    billing and payments · technical support · sales enquiry
  • customer_service.refund_requested · noul · our tasks
    The customer explicitly requests a refund or money back
  • customer_service.frustration · score · our tasks
    How frustrated does the customer appear?
    calm and neutral · frustrated but civil · very angry
  • td_customer_service.action · choice · LAYA's public suite
    What should the assistant do next with this conversation?
    answer_directly · close_no_action · escalate_to_human · execute_refund · request_information
  • td_customer_service.category · choice · LAYA's public suite
    What is this customer conversation primarily about?
    account · billing · delivery · refund · technical
  • td_customer_service.churn_risk · score · LAYA's public suite
    How likely is this customer to stop doing business with us because of this interaction?
    No sign of dissatisfaction. · Mild frustration, but the relationship is intact. · Clearly unhappy; repeat problems or explicit complaints. · Imminent: threatening to cancel, dispute or leave.
  • td_customer_service.needs_human · noul · LAYA's public suite
    This conversation requires a human agent rather than automated handling.
    false · true
  • td_customer_service.urgency · score · LAYA's public suite
    How time-sensitive is this conversation?
    No time pressure; can wait indefinitely. · Routine; handle within the normal queue. · Elevated; should be handled within the same week. · Critical; requires action within the same day.

Frequently asked questions

What does customer service triage involve?

Support tickets: which team, is a refund requested, how frustrated, churn risk, urgency, does a human need to step in. The domain TypeSafe uses in its own docs.

Which model is best at customer service triage?

Jev leads on Metatext decisions at 0.933 mean accuracy over 3 questions.

Related

Security incident triage

Alerts and incident write-ups: severity, active exploitation, credential compromise, true positive, disposition, urgency.

Agent trace observability

Traces of an autonomous agent run: outcome, risk, urgency, what the observability system should do, whether a human should review it.

Invoice processing

Invoices against purchase orders: does it match, is it a duplicate, discrepancy severity, disposition, urgency.