Finding the True 'King of Cost-Effectiveness'
In the recently released DeepSew (Note: Code/Agent capabilities) benchmark leaderboard, the comprehensive performance (Pass@1 success rate), average cost (Avg Cost), output tokens, and interaction steps of major mainstream models have been fully disclosed.
In actual large-scale commercial deployments and daily development assistance, we often follow a strict principle: once the model capabilities cross the business-usable performance baseline, invocation cost becomes the sole dimension for measuring cost-effectiveness. Based on this standard, we reshuffle the deck to reveal who is truly the current "King of Cost-Effectiveness".
📊 Core Benchmark Data at a Glance
| Model | Success Rate (Pass@1) | Avg Cost | Tokens Consumed | Steps |
|---|---|---|---|---|
| gpt-5.6-sol | 73% | $8.39 | 60k | 61 |
| claude-fable-5 | 70% | $21.63 | 119k | 88 |
| gpt-5.6-terra | 70% | $4.95 | 72k | 76 |
| gpt-5.6-luna | 67% | $3.03 | 73k | 102 |
| gpt-5.5 | 67% | $7.23 | 46k | 82 |
| claude-opus-4.8 | 59% | $13.22 | 135k | 120 |
| claude-sonnet-5 | 54% | $26.40 | 214k | 268 |
| gpt-5.4 | 52% | $5.65 | 71k | 70 |
| glm-5.2 | 44% | $3.92 | 78k | 129 |
| gemini-3.5-flash | 37% | $7.34 | 276k | 86 |
| kimi-k2.7-code | 31% | $2.82 | 59k | 149 |
🏆 Performance & Cost-Effectiveness Analysis (Cost-Driven)
We consider a Pass@1 rate of 65% and above as the dividing line for "highly usable". Among all the models that cross this performance baseline, we let them fight directly on cost.
1. 👑 The True King of Cost-Effectiveness: gpt-5.6-luna
If price is the only standard, then gpt-5.6-luna is the undisputed king. It boasts an excellent 67% accuracy rate (perfectly tying the previous flagship gpt-5.5), but its single-call cost has been compressed to the extreme at $3.03. Compared to the absolute cheapest across the board, kimi-k2.7-code ($2.82), luna requires only a minor cost increase to pull the success rate from an unusable 31% up to 67%. For the vast majority of tasks, it is the most economical choice for productivity.
2. Upgraded High-Quality Choice: gpt-5.6-terra
If your tasks are more demanding and 67% is not enough, the next tier choice is gpt-5.6-terra. It reaches an extremely high accuracy of 70% while costing only $4.95. Although slightly more expensive than luna, compared to claude-fable-5 which offers the same 70% accuracy (but at a whopping $21.63), terra's price advantage remains dominant.
3. Budget-Insensitive Absolute King: gpt-5.6-sol
If the business scenario requires the best possible outcome for every request, call gpt-5.6-sol directly. With the overall highest success rate of 73%, the fewest interaction steps (61), its $8.39 cost is well worth it for top-tier complex problems.
4. Pitfall Warning: The "Expensive Substitutes" No Longer Recommended
Under the perspective of "meeting the baseline and cost-driven", some models reveal extremely poor ROI:
- gpt-5.5: Offers the same 67% accuracy as luna, but costs up to $7.23 (1.3 times more expensive). It belongs to a bygone era and should be directly replaced.
- Claude Series: Shows a tendency of "high price, low configuration" across the board. claude-fable-5 (70%) requires $21.63, and claude-sonnet-5 demands a field-topping $26.40 for a mere 54% accuracy—a true "API bill assassin".
💡 Summary
- Daily/High-Frequency Calls: No-brainer choice: gpt-5.6-luna ($3.03 / 67%).
- Complex Task Balance Point: Choose gpt-5.6-terra ($4.95 / 70%).
- Core Breakthroughs: Choose gpt-5.6-sol ($8.39 / 73%).
In today's highly competitive model landscape, adhering to the principle of "looking at quality below the baseline, and comparing cost above the baseline" can save massive API expenses for enterprises.
(Data source: deepsew latest model benchmark results)