DeepSeek V4 Flash vs DeepSeek V4.1 Flash
Compare with other modelsDeepSeek V4.1 Flash takes a useful step forward on hallucinations. The share of conversations with no unsupported claims rises from 19% to 48%, and the cost per conversation drops from $0.019 to about $0.013. Tool use and task completion stay at 97 and 89.
Context retention takes a step back, from 90 to 81, so overall quality stays at about the same level. There is a lot to gain from the lower cost and fewer unsupported claims, but check that the agent still remembers what the customer said earlier.
| Metric | DeepSeek Sep 8, 2026 | DeepSeek Sep 15, 2026 |
|---|---|---|
| Overview | ||
| Quality score | 82 #5 of 17 | 85 #2 of 17 |
| Median reply | 4.6s Best among selected models Served by Novita | 5.1s Served by Phala |
| Cost per conversation | $0.019 | $0.013 Best among selected models |
| Quality scores | ||
| Tool use | 97 | 97 |
| Task completion | 89 | 89 |
| Context retention | 90 Best among selected models | 81 |
| Grounding | 94 Best among selected models | 93 |
| Safety | 94 | 96 Best among selected models |
| Hallucinations % of conversations with no unsupported claims detected | 19% | 48% Best among selected models |
| Running stats | ||
| p95 reply | 18.1s Best among selected models | 19.6s |
| Consistency % of repeat runs ended the same way | 85% | 85% |
| Output per turn | 516 tokens, median | 450 tokens, median |
Reply times cover the model call only. Cost estimates use token usage at list prices.
This whole report is one Voxli workspace: simulated customers, assertion checks, and a frozen, versioned test set that reruns when new models ship.
Test set v3f-2026-09 · edition 2026-09-08