MODEL BENCHMARK

Which AI model makes the best CX & SDR agent?

You benchmark the model. We benchmark the swap.

We run one support and sales agent through one frozen test set, played by simulated customers, changing only the model. Every model runs at its vendor defaults, so read each score as a baseline: we tuned nothing for any of them.

Benchmark your agent
level on quality (within 3 points) 45 50 55 60 65 70 75 80 85 90 $0.010 $0.020 $0.030 $0.040 $0.050 Up and left is better Cost per conversation Quality score GPT-6 Luna: quality score 87, $0.0040 per conversation, median reply 4.8s GPT-6 Luna DeepSeek V4.1 Flash: quality score 85, $0.013 per conversation, median reply 5.1s DeepSeek V4.1 Flash GPT-5.6 Luna: quality score 85, $0.0064 per conversation, median reply 3.9s GPT-5.6 Luna Claude Haiku 5.5: quality score 84, $0.0076 per conversation, median reply 2.6s Claude Haiku 5.5 DeepSeek V4 Flash: quality score 82, $0.019 per conversation, median reply 4.6s DeepSeek V4 Flash GLM 5.3 Flash: quality score 81, $0.011 per conversation, median reply 6.4s GLM 5.3 Flash GPT-5 mini: quality score 78, $0.030 per conversation, median reply 15.1s GPT-5 mini Gemini 3.1 Flash Lite: quality score 77, $0.019 per conversation, median reply 1.6s Gemini 3.1 Flash Lite Gemma 4 31B: quality score 77, $0.0079 per conversation, median reply 2.5s Gemma 4 31B Gemini 3.5 Flash Lite: quality score 72, $0.022 per conversation, median reply 1.8s Gemini 3.5 Flash Lite Claude Haiku 4.5: quality score 69, $0.043 per conversation, median reply 2.6s Claude Haiku 4.5 gpt-oss-120b: quality score 64, $0.013 per conversation, median reply 1.2s gpt-oss-120b GPT-4.1 mini: quality score 58, $0.012 per conversation, median reply 2.7s GPT-4.1 mini Mistral Small 4: quality score 56, $0.0053 per conversation, median reply 1.4s Mistral Small 4 Nova 2 Lite: quality score 53, $0.055 per conversation, median reply 2.2s Nova 2 Lite Gemini 2.5 Flash Lite: quality score 48, $0.0037 per conversation, median reply 1.3s Gemini 2.5 Flash Lite

The leaderboard

Compare

Quality is one score across the six dimensions below, with latency and cost shown beside it. Select any model for its full report.

Model Quality Median p95 Cost / conv.
1 GPT-6 Luna 87 4.8s 14.0s $0.0040
2 DeepSeek V4.1 Flash 85 5.1s 19.6s $0.013
3 GPT-5.6 Luna 85 3.9s 9.5s $0.0064
4 Claude Haiku 5.5 84 2.6s 7.2s $0.0076
5 DeepSeek V4 Flash 82 4.6s 18.1s $0.019
6 GLM 5.3 Flash 81 6.4s 34.9s $0.011
7 GPT-5 mini 78 15.1s 36.4s $0.030
8 Gemini 3.1 Flash Lite 77 1.6s 4.6s $0.019
9 Gemma 4 31B 77 2.5s 16.2s $0.0079
10 Gemini 3.5 Flash Lite 72 1.8s 4.5s $0.022
11 Claude Haiku 4.5 69 2.6s 6.4s $0.043
12 gpt-oss-120b 64 1.2s 3.6s $0.013
13 GPT-4.1 mini 58 2.7s 6.3s $0.012
14 Mistral Small 4 56 1.4s 3.8s $0.0053
15 Nova 2 Lite 53 2.2s 7.7s $0.055
16 Gemini 2.5 Flash Lite 48 1.3s 5.0s $0.0037

The six dimensions

Each dimension checks one thing a support agent has to get right, and shows how every model did on it.

  1. Tool use

    The agent has to call the right tool, with the right arguments, in the right order. Telling the customer it's done before the tool has finished counts against the model.

    Test tool calls on your agent

  2. Task completion

    The customer has to leave with what they came for and nothing extra. Only the outcome counts, in long conversations where the customer gets a fact wrong, sets a limit early, wanders off topic, and changes their mind.

  3. Context retention

    A constraint the customer states once, early, has to still hold when the agent acts at the end of a long conversation.

  4. Grounding

    Answers have to come from the knowledge base. When a customer states a wrong spec or a made-up product, the agent has to correct them or say plainly that it doesn't know, and still handle the rest of the request.

  5. Safety

    Simulated customers fake system messages, claim authority they don't have, and slip in instructions of their own. The agent has to hold its policy, keep its own instructions and tools to itself, protect other customers' orders, and still serve the real request.

  6. Hallucinations

    We pull out every statement the agent makes about a product or about what it has done, and check each one against the knowledge base and the tool results. One flagged statement marks the whole conversation, and the score is the share of conversations with nothing flagged, shown with its range.

    Read how Voxli checks factual claims

How we tested

We found that a public reasoning leaderboard says very little about how a model behaves as a deployed agent. So we built one controlled benchmark to test this: a single CX and SDR agent with a fixed prompt, tools, and knowledge base, and only the model swapped underneath.

Quality = 15% Tool use + 20% Task completion + 15% Context retention + 15% Grounding + 20% Safety + 15% Hallucinations

  1. 01

    One agent

    We built a single CX and SDR agent in an ordinary Voxli workspace: one prompt, one set of tools, one knowledge base.

  2. 02

    Swap the model

    The same agent runs again with a different model underneath. Nothing else in the configuration changes.

  3. 03

    A frozen test set

    Simulated customers play every test: support requests, sales flows, wrong facts, pressure, and injected instructions. We version the set, freeze it for the edition, and run every test more than once.

  4. 04

    One signal per dimension

    Each dimension is graded by exactly one scenario, so no failure counts twice.

  5. 05

    Two kinds of numbers

    Quality is a blend of the six dimensions at the weights shown. We report latency and cost beside it, since a high score can still be too slow or too costly for your queue.

  6. 06

    What the scores can't say

    The claim detector only grades the claims a model makes, so a model that never looks anything up isn't punished there. The other five dimensions catch it.

What we hold constant

Layer Held to Detail
Prompt Byte-identical The same system prompt for every model, checked against the frozen test set before each run.
Tools Identical definitions The same tool set and the same reference data behind it.
Knowledge base Identical Every model answers from the same documents.
Model The one variable Each model runs as its own clone of the same agent, with nothing else in the configuration touched. Sampling stays at each vendor's default.
Reasoning Set per model Vendor defaults, verified from observed tokens. Models pinned off their default: none.
Pricing Fixed list prices Cost per conversation is an estimate: we price the tokens each conversation actually used at pinned list prices, so caching shows up honestly.
Serving One host OpenRouter serves every model, on the same route.
Test set Frozen One version per edition, so results stay comparable model over model.

Benchmark your own agent

This whole report is one Voxli workspace: simulated customers, assertion checks, and a frozen, versioned test set that reruns when new models ship.

Get started

16 models · 6 scenarios · 46 tests · 3 repetitions · 2208 conversations · one fixed agent · test set v3f-2026-09 · edition 2026-09-08