“What’s the best AI model” has no answer on its own, and chasing one costs founders money in both directions. Some pay for frontier intelligence on work that never needed it. Others put a cheap model on customer-facing output to save a few dollars and ship a worse product. The model that’s right in one seat is wrong in the next, and what decides it is who runs the model and who pays its bill. The benchmark barely comes into it.
Here’s the framework I use, built from running it across my own products and client work.
Profile One: You, the Operator
If you’re a founder using AI for your own work, buy the best and stop tinkering with it.
I spend around $400 a month across Claude, Codex and Grok. That covers everything I personally do, and the idea of swapping in a cheaper model to shave that number is a false economy. The model is one of the cheapest inputs in your business. Your time is the expensive one. A frontier model that’s right the first time beats a cheaper one you have to correct twice, and the correction happens in your hours.
The open-weight models are genuinely good now, and for your own daily driving that still doesn’t matter. GLM 5.2 costs roughly a sixth of a frontier model and runs many tasks well. If you already have frontier access, routing your own work to it to save $18 a month is chasing the wrong line on the P&L. Spend the attention on the seats where model cost actually scales, which are the next three profiles.
Profile Two: Agents You Run for Clients
When you build an agent a client runs, the model stops being a subscription and becomes your cost of goods.
Your own $400 covers you and transfers to nobody. A client’s agent needs its own model access, and its bill is either on their frontier account or on something you provision. Frontier pricing on a client’s high-volume, unattended workflow either eats your margin or inflates their quote. At a sixth of the price, an open model is either margin you keep or a lower number you can win the deal with.
Most client agent work suits it. CRM operations, data migrations, helpdesk triage, routine backend calls: this isn’t frontier-hard reasoning, it’s competent execution at volume, and that’s exactly where open models are good enough. These agents also run unattended and often, which is where capped frontier subscriptions break and per-token pricing on a cheap model pulls ahead.
Two arguments apply here that never apply to your own laptop. A client on a regulated or privacy-sensitive footing may not be allowed to send data to a US API at all, and an open-weight model you can host in their own or EU infrastructure is sometimes the only acceptable option. And there’s provider risk: an agent built on one frontier vendor is one pricing change or one access restriction away from a problem, while a portable open model is insurance against both.
The caveat is real. Validate the specific task before you switch, keep the genuinely hard steps on a frontier model, and weigh reliability if you’re carrying an SLA, because a cheap gateway is not a frontier provider’s uptime.
Profile Three: A Productized AI Feature at Volume
This is where the cost question gets sharpest, and where the answer is usually routing rather than a single model.
Say your product has an AI feature customers use constantly: summarizing content, extracting fields, classifying what comes in. Every use is a model call, and every call is a cost against whatever that customer paid you. Summarizing an article isn’t frontier-hard work. Paying frontier prices to do it at scale leaves margin on the table on every single call, multiplied across every customer you have.
Swapping the whole feature onto the cheapest model and hoping nobody notices is the wrong version of this. The right version is routing: a cheap open model as the default for the bulk, a frontier model held back for the cases that need it or for a premium tier. The economics of a productized feature reward this more than any other profile, because the model cost is a per-unit COGS that compounds with every customer you add.
The discipline that makes it safe is benchmarking on your real inputs before you change anything. That means your actual customer data, including the non-English cases, since multilingual quality is where cheaper models most often fall short. A visible quality drop on a feature people pay for costs you more in churn and complaints than the model ever saves. Test first, keep a fallback to frontier on error, then switch whatever holds up.
Profile Four: The Access-Risk Hedge
This one isn’t about today’s bill. If you’re a non-US company building on US frontier models, you’re exposed to a decision made in another country: a pricing change, an export restriction, an access directive. That risk is small and it’s not zero, and it’s been rising.
An open-weight, self-hostable model is the hedge: a proven fallback you keep tested and ready, so you could move a critical workflow onto it if frontier access got restricted or repriced. For most founders this is a reason to keep an open model tested and ready in the toolbox, not a reason to run on it now. The value is that you’ve de-risked a dependency you don’t control.
How to Actually Decide
Three questions settle almost every case.
Who pays the model’s bill? If it’s your own subscription for your own work, buy frontier and move on. If it’s a per-call cost against revenue or margin, cheap models start to matter.
Is the task frontier-hard? Complex reasoning, hard coding and nuanced judgment still need the frontier. Summarizing, extracting, classifying, routine tool-calling: a good open model handles these, and paying frontier rates for them is waste.
Is it one call or ten thousand? A model choice you make once, for yourself, isn’t worth much thought. A choice that repeats on every customer action is a unit-economics decision, and the cheap model can be the difference between a feature that makes money and one that doesn’t.
Notice that none of these is “which model scored highest.” The leaderboard tells you what a model can do. It can’t tell you whether you should be paying for that capability in a given seat, and that’s the only question that moves your numbers.
One thing worth separating out: choosing a model and constraining what an agent is allowed to do are different problems. The model behind an agent has no bearing on whether it can merge your code or email your customers, and treating a cheaper model as a safety concern confuses the two. That’s a credential question, and it deserves solving on its own terms.
Related
Leave a Reply