Model Tiering
Also known as: tiered model selection, model size routing, LLM tier strategy
- Model Tiering
- Model tiering organizes AI language models into capability levels — small, medium, and large — then routes each task to the least powerful model that handles it adequately, reducing API costs by matching computational resources to actual task requirements.
Model tiering assigns AI tasks to different model sizes based on complexity — using cheap, fast models for simple requests and powerful models only when the task demands it.
What It Is
API costs for large language models are not flat. A small, fast model might cost a fraction of a cent per request; a large, high-capability model can cost far more for the same token count. If every request in your application goes to the most powerful model available, you pay that premium price even for tasks a simpler model handles just as well. Model tiering addresses this directly.
Think of it like staffing: you do not pay a senior consultant’s rate to sort mail. You define roles matched to the complexity of the work, assign the simplest tasks to the most cost-efficient option, and save the expensive resources for problems that actually need them. Model tiering applies the same logic to AI.
In practice, tiering means organizing your available models into two to four levels — often described as small (fast, cheap, limited reasoning), medium (balanced cost and capability), and large (powerful, expensive, capable of complex reasoning and code generation). You then define which task types belong to each tier. Text classification, short-form summarization, and FAQ retrieval typically belong in the small tier. Multi-step analysis, nuanced decision support, and code review belong in the large tier. Tier assignment follows the output quality requirements of each task type, not arbitrary assumptions about model prestige.
The result is an AI application whose computational spend reflects the actual distribution of task complexity. A customer support system might handle most interactions through a small model and reserve the large model for escalated, complex cases — shifting cost from a single flat premium rate to a weighted average that tracks real workload.
In the context of cutting LLM API costs with tools like LiteLLM, model tiering is the foundational strategy. It defines what gets routed where — model routing is the mechanism that executes the tiering decisions you make.
How It’s Used in Practice
The most common setup is a routing proxy that sits between your application and your model providers. You configure the proxy with tier definitions: small model for low-complexity tasks, large model for high-complexity ones. A classifier — a simple rule based on prompt length or task type tag, a lightweight scoring model, or a dedicated prompt-analysis function — decides which tier each request belongs to before it reaches the API.
In an application using LiteLLM, you might configure a router where requests tagged as “classification” or “short-summary” go to a smaller, cheaper model, while requests tagged as “code-generation” or “multi-step-reasoning” go to the large tier. The routing logic lives in configuration, not scattered across application code.
Pro Tip: Start by auditing your existing API calls. Classify a sample of recent requests by the complexity of the output they produced, then test those same requests against a smaller model. Most teams find that a large share of their requests are simpler than the model they are currently using for them — and that the smaller model handles those cases at the same quality bar.
When to Use / When Not
| Scenario | Use | Avoid |
|---|---|---|
| Mixed-complexity workload — simple FAQs alongside complex analysis in the same system | ✅ | |
| API cost is material to unit economics or product margins | ✅ | |
| Tasks are uniformly complex and all require the same model capability | ❌ | |
| Task complexity can be classified reliably before the request is sent | ✅ | |
| Task classification is unreliable or edge cases are frequent and hard to catch | ❌ | |
| Quality degradation at high volume is a real risk and you lack monitoring across all tiers | ❌ |
Common Misconception
Myth: Model tiering means accepting lower quality to save money.
Reality: Quality only drops if tasks are misrouted. For the large share of requests that are genuinely simple — short-form classification, FAQ answers, template-based generation — smaller models perform as well as large ones on those specific tasks. Tiering does not lower quality; it stops you from paying for capability you are not using. The quality risk sits in the classification layer, not in the tiers themselves.
One Sentence to Remember
Route simple work to cheap models and reserve your powerful models for tasks that actually need them — model tiering makes LLM API costs proportional to the complexity of the work being done, not to the most demanding task in your system.
FAQ
Q: What is the difference between model tiering and model routing? A: Model tiering defines the structure — which models form which tiers and what task types belong to each. Model routing is the mechanism that executes those decisions. Tiering is the strategy; routing is the implementation that puts it in motion.
Q: How do I decide which tasks belong to which tier? A: Run a sample of your actual requests through both a small and a large model, then compare outputs against your specific quality bar. Where the cheaper model meets your standard, assign that tier. Start with your highest-volume, lowest-complexity task type — that is where the largest cost reduction typically lives.
Q: Does model tiering require switching AI providers? A: No. You can tier within a single provider’s model family, or mix providers using a routing proxy like LiteLLM. The provider question is separate from the tiering decision — define your tier structure first, then map tiers to the available models that fit each level.
Expert Takes
From a systems standpoint, model tiering treats token generation as a resource allocation problem. Different tasks have different information-processing requirements. A classification task draws on a narrow slice of the model’s learned representations; a multi-step reasoning task activates broader, more costly computational paths. Sizing the model to the task is not a cost workaround — it is matching the computational substrate to the actual complexity of the output being requested.
In a well-designed LLM integration, model tiering is a configuration decision, not an architecture decision. Tier thresholds live in your routing config, not scattered across application logic. The critical spec is task classification: inputs must be categorized reliably before they reach the router. If your classifier has edge cases, those edge cases become quality bugs at scale. Get the classification layer right first. Once that is solid, the routing itself is straightforward to implement and easy to adjust.
The teams spending the most on LLM APIs are running everything through their largest model because that was the path of least resistance in the prototype phase. Model tiering is the operational shift that separates prototype economics from production economics. When API costs start appearing on your P&L, tiering is not optional — it is the first lever that delivers meaningful reduction without touching your product’s output quality.
Model tiering optimizes for cost, but the tiers themselves embed assumptions about what counts as simple work. A request routed to a smaller model because it looks like classification might actually require nuanced judgment that the smaller model lacks. The savings are real. The risk is that degraded outputs at high volume can affect decisions that touch real people — and that degradation is invisible without deliberate quality monitoring across all tiers, not just the expensive one.