Qwen3.8 27B vs DeepSeek V4.1 Flash
Compare with other modelsQwen3.8 27B is strong on task completion, with a score of 94 compared with 89 for DeepSeek V4.1 Flash. Context retention is better too, at 85 against 81, while their overall quality scores are similar.
DeepSeek has fewer conversations with unsupported claims and costs about half as much: $0.013 against $0.024 per conversation. Both have a median reply time of 5.1 seconds. Start with DeepSeek, but Qwen is worth trying if task completion is where your agent needs to improve.
| Metric | Alibaba Oct 22, 2026 | DeepSeek Sep 15, 2026 |
|---|---|---|
| Overview | ||
| Quality score | 82 #6 of 17 | 85 #2 of 17 |
| Median reply | 5.1s Served by Wafer | 5.1s Served by Phala |
| Cost per conversation | $0.024 | $0.013 Best among selected models |
| Quality scores | ||
| Tool use | 89 | 97 Best among selected models |
| Task completion | 94 Best among selected models | 89 |
| Context retention | 85 Best among selected models | 81 |
| Grounding | 92 | 93 Best among selected models |
| Safety | 90 | 96 Best among selected models |
| Hallucinations % of conversations with no unsupported claims detected | 33% | 48% Best among selected models |
| Running stats | ||
| p95 reply | 22.1s | 19.6s Best among selected models |
| Consistency % of repeat runs ended the same way | 79% | 85% Best among selected models |
| Output per turn | 693 tokens, median | 450 tokens, median |
Reply times cover the model call only. Cost estimates use token usage at list prices.
This whole report is one Voxli workspace: simulated customers, assertion checks, and a frozen, versioned test set that reruns when new models ship.
Test set v3f-2026-09 · edition 2026-09-08