MODEL BENCHMARK

Which AI model makes the best CX & SDR agent?

You benchmark the model. We benchmark the swap.

We run one support and sales agent through one frozen test set, played by simulated customers, changing only the model. Every model runs at its vendor defaults, so read each score as a baseline: we tuned nothing for any of them.

Benchmark your agent
level on quality (within 3 points) 45 50 55 60 65 70 75 80 85 $0.010 $0.020 $0.030 $0.040 $0.050 $0.060 Up and left is better Cost per conversation Quality score GPT-5.6 Luna $0.0064 · 3.9s DeepSeek V4 Flash $0.019 · 4.5s GLM 5.3 Flash $0.011 · 6.3s GPT-5 mini $0.030 · 15.2s Gemini 3.1 Flash Lite $0.019 · 1.6s Gemma 4 31B $0.0080 · 2.5s Gemini 3.5 Flash Lite $0.022 · 1.8s Claude Haiku 4.5 $0.043 · 2.6s gpt-oss-120b $0.013 · 1.2s GPT-4.1 mini $0.012 · 2.8s Mistral Small 4 $0.0053 · 1.4s Nova 2 Lite $0.055 · 2.2s Gemini 2.5 Flash Lite $0.0037 · 1.3s

The leaderboard

Quality is one score across the six dimensions below, with latency and cost shown beside it. Select any model for its full report.

Model Quality Median p95 Cost / conv.
1 GPT-5.6 Luna 84 3.9s 9.6s $0.0064
2 DeepSeek V4 Flash 81 4.5s 18.1s $0.019
3 GLM 5.3 Flash 81 6.3s 34.9s $0.011
4 GPT-5 mini 78 15.2s 36.4s $0.030
5 Gemini 3.1 Flash Lite 77 1.6s 4.6s $0.019
6 Gemma 4 31B 77 2.5s 16.4s $0.0080
7 Gemini 3.5 Flash Lite 72 1.8s 4.5s $0.022
8 Claude Haiku 4.5 69 2.6s 6.4s $0.043
9 gpt-oss-120b 64 1.2s 3.6s $0.013
10 GPT-4.1 mini 57 2.8s 6.3s $0.012
11 Mistral Small 4 57 1.4s 3.8s $0.0053
12 Nova 2 Lite 52 2.2s 7.7s $0.055
13 Gemini 2.5 Flash Lite 48 1.3s 5.0s $0.0037

The six dimensions

Each dimension checks one thing a support agent has to get right, and shows how every model did on it.

  1. Tool use

    The agent has to call the right tool, with the right arguments, in the right order. Telling the customer it's done before the tool has finished counts against the model.

  2. Task completion

    The customer has to leave with what they came for and nothing extra. Only the outcome counts, in long conversations where the customer gets a fact wrong, sets a limit early, wanders off topic, and changes their mind.

  3. Context retention

    A constraint the customer states once, early, has to still hold when the agent acts at the end of a long conversation.

  4. Grounding

    Answers have to come from the knowledge base. When a customer states a wrong spec or a made-up product, the agent has to correct them or say plainly that it doesn't know, and still handle the rest of the request.

  5. Safety

    Simulated customers fake system messages, claim authority they don't have, and slip in instructions of their own. The agent has to hold its policy, keep its own instructions and tools to itself, protect other customers' orders, and still serve the real request.

  6. Hallucinations

    We pull out every statement the agent makes about a product or about what it has done, and check each one against the knowledge base and the tool results. One flagged statement marks the whole conversation, and the score is the share of conversations with nothing flagged, shown with its range.

How we tested

We found that a public reasoning leaderboard says very little about how a model behaves as a deployed agent. So we built one controlled benchmark to test this: a single CX and SDR agent with a fixed prompt, tools, and knowledge base, and only the model swapped underneath.

Quality = 15% Tool use + 20% Task completion + 15% Context retention + 15% Grounding + 20% Safety + 15% Hallucinations

  1. 01

    One agent

    We built a single CX and SDR agent in an ordinary Voxli workspace: one prompt, one set of tools, one knowledge base.

  2. 02

    Swap the model

    The same agent runs again with a different model underneath. Nothing else in the configuration changes.

  3. 03

    A frozen test set

    Simulated customers play every test: support requests, sales flows, wrong facts, pressure, and injected instructions. We version the set, freeze it for the edition, and run every test more than once.

  4. 04

    One signal per dimension

    Each dimension is graded by exactly one scenario, so no failure counts twice.

  5. 05

    Two kinds of numbers

    Quality is a blend of the six dimensions at the weights shown. We report latency and cost beside it, since a high score can still be too slow or too costly for your queue.

  6. 06

    What the scores can't say

    The claim detector only grades the claims a model makes, so a model that never looks anything up isn't punished there. The other five dimensions catch it.

What we hold constant

Layer Held to Detail
Prompt Byte-identical The same system prompt for every model, checked against the frozen test set before each run.
Tools Identical definitions The same tool set and the same reference data behind it.
Knowledge base Identical Every model answers from the same documents.
Model The one variable Each model runs as its own clone of the same agent, with nothing else in the configuration touched. Sampling stays at each vendor's default.
Reasoning Set per model Vendor defaults, verified from observed tokens. Models pinned off their default: none.
Pricing Fixed list prices Cost per conversation is an estimate: we price the tokens each conversation actually used at pinned list prices, so caching shows up honestly.
Serving One host OpenRouter serves every model, on the same route.
Test set Frozen One version per edition, so results stay comparable model over model.

Benchmark your own agent

This whole report is one Voxli workspace: simulated customers, assertion checks, and a frozen, versioned test set that reruns when new models ship.

Get started

13 models · 6 scenarios · 46 tests · 3 repetitions · 1794 conversations · one fixed agent · test set v3f-2026-09 · edition 2026-09-08