In brief
- Measure total task cost, including retries and review.
- Keep quality thresholds explicit.
- Compare Pareto-efficient choices by workload.
Request cost is not task cost
A low-priced request that fails twice or requires extensive review may cost more than a stronger first pass. Track tokens, attempts, tool calls, wall time, and human intervention through the accepted outcome.
Set the quality floor first
Remove candidates that do not meet the task's correctness or safety bar. Cost optimization below the quality floor is false economy.
Do not average a severe failure into an attractive price-performance score.
Find the decision frontier
A model is dominated when another is at least as good on quality, latency, and cost for the workload. Focus decision-making on the remaining candidates and their operational tradeoffs.
| Workload | Primary constraint | Secondary constraint |
|---|---|---|
| Interactive chat | First-token latency | Quality |
| Code change | Accepted-task quality | Total task cost |
| Batch extraction | Schema success rate | Throughput cost |
| High-risk review | Recall and evidence | Latency |
Refresh when the environment changes
Pricing, model versions, prompts, caching, tools, and traffic mix change. Store the evaluation inputs and rerun the frontier instead of treating one chart as permanent truth.
Frequently asked
Questions, answered plainly.
What is the cheapest AI model?+
Prices vary and the cheapest request may not produce the lowest task cost. Compare live pricing alongside success rate, retries, and review effort.
How should latency be measured?+
Track queue time, time to first token, and total completion separately, then choose the measure that matches the user experience.
Can quality and cost be one score?+
You can create a utility score after setting explicit thresholds, but keep the underlying dimensions visible so tradeoffs are not hidden.
Sources and next paths