Skip to content

Which AI Model for Which Job

The decision

Your team does the same kinds of AI work again and again: drafting replies, summarizing calls, pulling details out of documents, and writing code. You have to decide what that work runs on.

The short version: start with one default model. Move any high-volume, simple job to a cheaper model once it passes your own test. Keep a tested fallback. Software that picks a model for every request, and running models on your own servers, pay off only in narrow conditions.

This guide is for the team lead or manager who owns that call, and for the AI assistant helping them decide.

Does this apply to you?

Use it when a piece of AI work recurs, matters, and could plausibly run on more than one model or tool. If your organization approves only one tool, that part is decided for you. The test in "How to decide" still tells you whether that tool is doing the job.

Deciding for yourself rather than a team? The same test answers the personal version: whether a thinking mode is worth it, or a higher plan. A thinking mode makes the model work through a problem before it answers, which costs more.

This guide ranks no models and compares no prices, because both change within months. Depending on the task, the price of a given level of capability has fallen 9 to 900 times a year (Epoch AI, 2025). For the engineering side of model selection, see the Technology Architecture Framework and Designing an AI Platform Function.

The options

  • One model for everything. Pick a single model or tool and use it for every job. It is the simplest to support, to train people on, and to govern.
  • A model per type of work, chosen by a person. For example, a fast, inexpensive model for sorting incoming requests and a stronger one for drafting contracts. Someone decides the assignment once, tests it, and writes it down.
  • Software that picks a model for each request. A router looks at each request as it arrives. It sends easy ones to a cheaper model and hard ones to a stronger one.
  • An open-weight model. A model its maker publishes for anyone to download and run. A cloud provider can run it for you, or you can run it on your own servers.

These combine. You might use one of the first two options day to day, with a second model tested and ready in case the first becomes unavailable.

What actually matters

  • Quality on your own work. How often the output is good enough to use, judged against a test you wrote before trying anything. Public leaderboards measure someone else's tasks.
  • Cost per accepted result. Everything spent to get output you could use, divided by the number of results that passed your test. That includes failed attempts, reruns, and the time people spend checking and fixing.
  • Being able to switch. Models get retired, changed, and occasionally withdrawn. What matters is how much work it takes to move.
  • Constraints you cannot trade away. Your data rules, the model's license, and where the model comes from. These rule options out before cost or quality come into it.

How the options compare

The price list does not tell you the cost. One study compared eight reasoning models on 12 benchmark tasks, pair by pair. In 32% of those comparisons, the model with the lower listed price cost more in total (Chen et al., 2026). On single-question tasks, more than 95% of the extra cost came from thinking tokens, the working a model does before it answers (Chen et al., 2026). The study measured cost alone, so the comparison that matters is cost per accepted result.

More reasoning is a setting to test, not a default. The Holistic Agent Leaderboard, built by Princeton researchers, tests AI agents: systems that take several steps in a row, such as searching and then drafting. It also varied how much reasoning the models did. In 21 of 36 combinations, more reasoning did not improve accuracy (Kapoor et al., 2025). Its authors argue for judging agents on accuracy and cost together (Kapoor et al., 2025).

Staying on one model within a conversation keeps a caching discount. Anthropic and OpenAI both discount text a model has already read in a session, which they call caching. Anthropic bills that text at a tenth of the usual input price or less (Anthropic, 2026). OpenAI says caching can cut input costs by up to 90% (OpenAI, 2026). A post sponsored by a router seller notes the catch: switch models mid-conversation, and the new model rereads everything at full price (Chawla, 2026).

Using several models is increasingly common. The investment firm a16z, which backs AI companies, surveyed 100 chief information officers across 15 industries in 2025 (a16z, 2025). Of those, 37% used five or more models, up from 29% a year earlier (a16z, 2025). For simpler or internal tasks, "model choice often comes down purely to cost" (a16z, 2025). A person choosing once per type of work needs one test per type, not a system deciding every request.

A router pays off only in narrow conditions. Academic tests found large savings on single-question benchmarks, where each request stands alone. One cascade tries a cheap model first and passes weak answers up to a stronger one. It cut cost by 50% to 98%, depending on the dataset (Chen, Zaharia and Zou, 2023). A Berkeley-led research group, LMSYS, more than halved cost on one benchmark while keeping 95% of GPT-4's quality (LMSYS, 2024).

Multi-step work is different. Manifest, which sells software for managing AI model traffic, retired its own router after four months and 7,000 users. It concluded that "for most use cases, sticking to a single battle-tested model is the best thing you can do" (Manifest, 2026).

One reason: a request's real difficulty often appears only after the model starts using tools such as search (Manifest, 2026). The same sponsored post argues that switching models inside a multi-step task can cost more than staying on one (Chawla, 2026). We found no independent audit of what routers save in practice.

Open-weight models come close on quality, and cost less through a host. The OECD, an intergovernmental economic body, wrote a paper for the G7. It found open-weight models reach about 90% of closed models' benchmark performance at launch (OECD, 2026). Cloud providers that host them charge about a fifth of the price of closed models for the same task (OECD, 2026). The paper makes the case for openness and, by its own account, leaves risks out of scope (OECD, 2026).

Running one yourself is a volume decision. A token is a fragment of a word; models read and write in tokens. The same paper found no saving from self-hosting below 100 million tokens a month (OECD, 2026). At medium volume, self-hosting took about 30 months to break even (OECD, 2026). The paper's text puts medium at 1 billion tokens a month, and its table at 500 million (OECD, 2026). Large open models also need serious hardware. Its maker's demo of DeepSeek-V3, an open-weight model released in December 2024, uses two machines of eight graphics processors (GPUs) each (DeepSeek-AI, 2024). Those break-even figures assume buying the hardware. The paper says renting GPUs can pay off at more modest volumes, under some assumptions (OECD, 2026).

License and origin need a review before either path. "Open weight" is not open source. Its licenses may restrict commercial use, volume, or training a competing model (Cooley, 2026). Self-hosted deployments may also lack indemnity, the provider's promise to cover you if the output causes a legal claim (Cooley, 2026).

Cooley, a law firm that advises on this, notes that a substantial share of top open-weight models come from outside the US (Cooley, 2026). Export controls are government limits on who may use a technology. Cooley treats checking jurisdiction and export controls as standard diligence, not a bar to adoption (Cooley, 2026). Where your data goes also depends on how you reach a model: the maker's app, a cloud host, or your own servers. Check each one's terms.

Plan for your model disappearing. OpenAI promises at least six months' notice before retiring a finished model, and much shorter notice, such as two weeks, for preview releases. Safety or compliance concerns can shorten either (OpenAI, 2026). Anthropic promises at least 60 days (Anthropic, 2026).

In June 2026, neither notice period applied. A US export-control order led Anthropic to disable two of its newest models for all customers, the day the order arrived (Anthropic, 2026). One returned on July 1 on Anthropic's own apps and API worldwide, with the cloud platforms to follow (Anthropic, 2026). The other had only gone to a small partner group, and came back for a set of US organizations (Anthropic, 2026). The Cloud Security Alliance now recommends at least two tested substitute models for each critical AI workflow (Cloud Security Alliance, 2026).

When each one wins

One model for everything wins when:

  • The work is varied, the stakes are modest, and nobody has time to test several models.
  • Your volume is small enough that a cheaper model would save little.
  • You want the least to support, train, and govern.

A model per type of work wins when:

  • A high-volume job is simple and repetitive, such as sorting, tagging, or pulling fields out of forms.
  • One job needs something your default model does badly, such as long documents or code, or has a data rule it cannot meet.
  • You can afford one test per type of work and a written record of what was chosen and why.

Software that picks per request wins when:

  • Requests are single questions, not conversations or multi-step tasks, and they arrive in large volume.
  • Telling an easy request from a hard one is cheap and reliable before the model runs.
  • You have engineers to run the router and a test set to keep checking it.

It loses on long conversations and multi-step tasks, where every switch gives up the caching discount.

An open-weight model wins when:

  • A high-volume job passes your test on a hosted open-weight model, at a fraction of the price.
  • Data must stay inside your own systems, and your volume justifies running the model yourself.

Either way, clear the license and origin review first.

How to decide

Answer these in order, for one recurring piece of work at a time:

  1. Write the acceptance test first. Before you try any model, write down what a usable result looks like for this job. Make it specific enough that two people would agree on a pass.
  2. Check the constraints. Confirm your data rules, the license, and where the model comes from. Drop any option they rule out.
  3. Pull 20 real tasks. Use recent examples of the actual work, including the awkward ones. Twenty is enough to spot a clear difference, not to prove a small one.
  4. Run your current choice and one alternative. Record every attempt, the time spent checking and fixing, and whether each result passed.
  5. Work out cost per accepted result for each. Add up everything spent, including people's time, and divide by the number of results that passed.
  6. Choose, write it down, and keep a fallback tested. The runner-up is your fallback if your first choice is retired or withdrawn. For critical work, keep at least two, as the Cloud Security Alliance recommends.

This test compares the first two options directly. If a router or self-hosting looks promising, the same numbers tell you whether the savings justify bringing in engineers.

One person deciding whether a thinking mode or a higher plan is worth it can run the same steps.

Watch out for

  • Choosing by leaderboard or list price. Leaderboards test someone else's work, and list prices leave out thinking and rework.
  • Leaving out people's time. Review and rework never appear on a model invoice, so they are easy to forget. They belong in the comparison.
  • Adding a router before you need one. A person assigning models to types of work is simpler to run and to check.
  • Treating "open" as "free." Hosting, staff, and license terms all cost something.
  • Having no fallback. If your only model disappears, so does the work that depends on it.
  • Locking in by accident. One model, or a model per type of work, is cheap to change if your prompts and tests do not depend on one vendor. Running a model on your own servers is the hardest choice to unwind.

Sources

  • Epoch AI — LLM inference prices have fallen rapidly but unequally across tasks, 2025. Across all of these benchmarks and performance thresholds, we found prices declining between 9x per year and 900x per year, with a median of 50x per year. View source · verified 2026-07-01 · primary
  • Chen et al. — The Price Reversal Phenomenon: When Cheaper Reasoning Models Cost More, 2026. in 32% of model-pair comparisons, the model with a lower listed price actually incurs a higher total cost ... we examine all (8 choose 2) = 28 model pairs across 12 tasks, yielding 336 pairwise cost comparisons. Of these, 106 comparisons (32%) exhibit pricing reversal. View source · verified 2026-09-13 · primary
  • Chen et al. — The Price Reversal Phenomenon: When Cheaper Reasoning Models Cost More, 2026. Overall, the number of thinking tokens is the dominating factor for single-turn tasks. More than 95% of the cost difference among reversed model pairs is attributed to thinking tokens. For multi-turn tasks, on the other hand, both the number of turns and the cached input tokens contribute substantially. View source · verified 2026-09-13 · primary
  • Kapoor et al. (Princeton) — Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation, 2025. we observe that in 21 of 36 model-agent-benchmark combinations, increased reasoning effort produces equal or lower accuracy. More reasoning does not always mean better results. View source · verified 2026-09-13 · primary
  • Kapoor et al. (Princeton) — Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation, 2025. we present Pareto frontiers of accuracy versus cost (both dollar cost and token cost), enabling practitioners to select agents based on real-world constraints ... A computationally expensive model that achieves marginally better accuracy may be less suitable for deployment than a cheaper model with slightly lower performance. View source · verified 2026-09-13 · primary
  • Anthropic — Prompt caching (Claude Platform Docs), 2026. Cache read tokens are 0.1 times the base input tokens price (see the table footnote for per-model exceptions) ... By default, the cache has a 5-minute lifetime. View source · verified 2026-09-13 · primary
  • OpenAI — Prompt caching (OpenAI API guide), 2026. Pay the model's reduced cached-input rate for reused tokens, discounted up to 90%. ... The minimum cacheable prompt length is 1,024 tokens for GPT-5.6 and later and varies by request settings for earlier models. View source · verified 2026-09-13 · primary
  • Daily Dose of Data Science (Avi Chawla) — LLM Routing Can Cost More Than Not Routing, 2026. That works for isolated requests. Inside an agent loop, though, a naive router can cost more than using one model throughout the session. ... But if you switch models mid-session, the new model must process the conversation again at full price. View source · verified 2026-09-13 · primary
  • Andreessen Horowitz (a16z) — How 100 Enterprise CIOs Are Building and Buying Gen AI in 2025, 2025. ...surveyed 100 CIOs across 15 industries ... In this year's survey, 37% of respondents are now using 5 or more models as opposed to 29% last year. View source · verified 2026-09-13 · primary
  • Andreessen Horowitz (a16z) — How 100 Enterprise CIOs Are Building and Buying Gen AI in 2025, 2025. For simpler or internal tasks, model choice often comes down purely to cost. View source · verified 2026-09-13 · primary
  • Chen, Zaharia, Zou (Stanford University) — FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance, 2023. Table 3 displays the overall cost savings of FrugalGPT, which range from 50% to 98%. This is feasible because FrugalGPT identifies the queries that can be accurately answered by smaller LLMs and, as a result, only invokes those cost-effective LLMs. View source · verified 2026-09-13 · primary
  • LMSYS (UC Berkeley, Anyscale, Canva) — RouteLLM: Learning to Route LLMs with Preference Data, 2024. We demonstrate that our router models significantly reduce costs-by over 2x-without substantially compromising quality. ... MT Bench 3.66 (95% GPT-4 quality) ... MMLU 1.41 (92% GPT-4 quality) ... GSM8K 1.49 (87% GPT-4 quality) ... Table 6: Cost saving ratio of our best performing routers over GPT-4. View source · verified 2026-09-13 · primary
  • Manifest — Everyone is building LLM routers, we deprecated ours, 2026. We don't believe in model routing anymore. For most use cases, sticking to a single battle-tested model is the best thing you can do. ... After four months of usage across 7000 cloud users, we saw mixed results and a lot of GitHub issues and discussions about it. View source · verified 2026-09-13 · primary
  • Manifest — Everyone is building LLM routers, we deprecated ours, 2026. The prompt alone does not contain the whole task; it is just the trigger. A lot of the context that determines complexity is only discovered later through tool calls, web searches, and so on. View source · verified 2026-09-13 · primary
  • OECD — Benefits of AI openness: OECD discussion paper for the G7, 2026. While open models achieve approximately 90% of the performance of closed models at launch, they are often available at significantly lower cost, resulting in a higher quality-to-price ratio. View source · verified 2026-09-13 · primary
  • OECD — Benefits of AI openness: OECD discussion paper for the G7, 2026. In practical terms, a price ratio of 0.2 means that cloud-available open-weight models cost about 20% of the price of closed-weight models for the same task, offering substantially better value for money. View source · verified 2026-09-13 · primary
  • OECD — Benefits of AI openness: OECD discussion paper for the G7, 2026. The analysis focuses on the benefits associated with AI openness – potential risks or downsides fall outside the scope of this paper and are left for future research. View source · verified 2026-09-13 · primary
  • OECD — Benefits of AI openness: OECD discussion paper for the G7, 2026. Self-hosting of open-weight models becomes cost-effective only at scale. ... The economic benefits of self-hosting are not evident for small workloads (less than 100 million tokens per month). Under medium workloads (1 billion tokens per month), private hosting becomes cheaper than pay-as-you-go cloud services only after 2.5 years (around 30 months). ... Table 3.3. Breakeven months for cloud API vs private GPU installation scenarios ... Medium (500 million tokens per month) 30.4 ... Under certain assumptions, GPU rental can therefore be cost-effective even at relatively modest usage levels. View source · verified 2026-09-13 · primary
  • DeepSeek-AI — DeepSeek-V3 (model card, README), 2024. We present DeepSeek-V3, a strong Mixture-of-Experts (MoE) language model with 671B total parameters with 37B activated for each token. ... torchrun --nnodes 2 --nproc-per-node 8 generate.py ... --config configs/config_671B.json. View source · verified 2026-09-13 · primary
  • Cooley LLP — Unlocking the Weights: What Enterprises Should Know Before Deploying Open-Weight AI Models, 2026. "Open weight" is not "open source." ... Depending on the model, organizations may encounter provisions addressing commercial use limitations, acceptable use restrictions, volume restrictions, and attribution or branding requirements. Some licenses also restrict using the model, its outputs or derivative works to train, develop or improve a competing AI model. View source · verified 2026-09-13 · primary
  • Cooley LLP — Unlocking the Weights: What Enterprises Should Know Before Deploying Open-Weight AI Models, 2026. Unlike many hosted AI services, self-hosted open-weight deployments may lack provider indemnification or related contractual and technical protections – not only for IP-infringing outputs, but also more broadly for harmful, inaccurate or discriminatory outputs. View source · verified 2026-09-13 · primary
  • Cooley LLP — Unlocking the Weights: What Enterprises Should Know Before Deploying Open-Weight AI Models, 2026. a substantial share of today's top open-weight models are developed outside the United States, including by developers based in China. ... Organizations evaluating an open-weight model of foreign origin should treat these considerations as part of standard diligence – alongside licensing and IP review – rather than as a bar to adoption. View source · verified 2026-09-13 · primary
  • OpenAI — Deprecations, 2026. Unless safety or compliance concerns require a faster timeline, we provide the following minimum notice periods before model retirement: Generally available models: At least 6 months... Preview models... may be retired with much shorter notice, such as 2 weeks. View source · verified 2026-09-13 · primary
  • Anthropic — Model deprecations, 2026. Anthropic notifies customers with active deployments for models with upcoming retirements, providing at least 60 days' notice before model retirement for publicly released models. View source · verified 2026-09-13 · primary
  • Anthropic — Statement on the US government directive to suspend access to Fable 5 and Mythos 5, 2026. The US government, citing national security authorities, has issued an export control directive to suspend all access to Fable 5 and Mythos 5 by any foreign national... The net effect of this order is that we must abruptly disable Fable 5 and Mythos 5 for all our customers to ensure compliance. We received the directive from the government today at 5:21pm (ET). View source · verified 2026-09-13 · primary
  • Anthropic — Redeploying Fable 5, 2026. Because the order took effect immediately and we had no reliable way to verify nationality in real-time, we suspended access to both models for all users. As of today, June 30, the export controls on Fable 5 and Mythos 5 have been lifted. Fable 5 will be available starting tomorrow, Wednesday, July 1, to users globally on the Claude Platform, Claude.ai, Claude Code, and Claude Cowork. ... We will re-enable access on AWS, Google Cloud, and Microsoft Foundry as quickly as possible. ... Mythos 5 ... was only released to a small number of trusted Project Glasswing partners for use in defensive cybersecurity. ... We have also restored access to Mythos 5 for a set of US organizations, following the US government's approval on June 26. View source · verified 2026-09-13 · primary
  • Cloud Security Alliance (AI Safety Initiative) — Fable 5 Suspension: Enterprise AI Under Export Controls, 2026. Organizations should maintain at least two tested substitute models for each critical AI-enabled workflow, with validated performance benchmarks and integration documentation stored in offline-accessible form. View source · verified 2026-09-13 · primary