Teams building agents hear the same objection from whoever tried the premium option first. We put our math on it in writing, eval gaps included.
Every few weeks someone tells us the same thing. They opened Claude Code, handed it a messy, long-running task, went to get a coffee and came back to working code. Then they tried the same job on a cheaper model and spent the afternoon fixing small mistakes. If you have lived this, the conclusion feels obvious: premium models win, and building agents on anything else is saving pennies to lose hours.
We won't argue with the experience; it matches ours. But the conclusion doesn't follow, because the mental math most people run skips two things: what "better" costs, and how much of the gap is the model at all.
Where the expensive models genuinely win
Let's concede the whole list first. On complex, long-running tasks, a frontier model like Opus 5.5 makes fewer wrong turns when your instructions are ambiguous. It self-corrects better over a long horizon. It needs less hand-holding: fewer retries and dead ends, fewer of those moments where the agent confidently walks into a wall. If the task is genuinely hard, the difference is real and you can feel it.
A comparison that hid this gap would be marketing, so we show ours with the gap visible.
Our evals, gaps included
We ran 15 tasks taken from real work on our platform (we build Vybe, so weigh our numbers accordingly), across sales, marketing and recruiting, from easy to hard. Each task has a checklist: the facts a good answer must include, the instructions it must follow, the mistakes it must avoid, like inventing a detail or editing a file it was only asked to read. Deterministic checks and a separate AI judge review every run from a fresh agent, and cost is not part of the score. All of it is public on our model showcase, scores averaged over five runs per task.
The scores: GLM 5.3 Flash averages 97.1%, GPT-6 Astra 98.4% and Opus 5.5 99.0%. Recruiting is where the gap widens: 95.0% for GLM against 99.7% for Opus and a full 100% for GPT-6 Astra. Those are real gaps. A 95 is not a 99 when a missed check means a wrong answer reaches a candidate.
Now the cost side of the same table. At public provider rates, Opus 5.5 costs $254 per 1,000 tasks. GLM 5.3 Flash costs $18 for the same 1,000 tasks, 14× cheaper. GPT-6 Astra sits at $662, 2.6× the Opus price.
One concrete example from the showcase: the three models got a marketing task, analyze a 90-day Search Console export and propose fixes. GLM 5.3 Flash finished at $0.012 per run with a 98.2% score. Opus 5.5 scored 100% for $0.23, 19 times the price, and GPT-6 Astra also scored 100% for $0.50. So the cheap model missed fewer than two checks in a hundred. The premium models paid 19× to 42× more to close that last stretch. Whether that stretch matters depends on the task.
What the gap costs at scale
Those per-task prices look tiny. At 10,000 tasks a month, the all-Opus bill is $2,540 and the GLM bill is $180. The difference is $2,360 every month, about $28,000 a year. We wrote before about how we cut our AI agent bill. This is the same logic applied to model choice.
"Better" is only a useful word once it has a price. An extra 1.9 points of checklist completion on routine operational work is rarely worth $28,000 a year. If your workload is twenty genuinely hard tasks a month and ten thousand easy ones, run the twenty on the premium model. Your premium bill for the month comes to about five dollars.
Model choice is set per task
That trade only exists because of granularity. Model choice is set per task: per agent, per scheduled task, per skill and even per step inside an agent. The same agent can draft customer replies on a frontier model while a scheduled task tags support tickets on the cheap one. If you are evaluating agent platforms, ask where the model choice lives. When one model has to do everything, you end up with the all-premium bill or the all-cheap quality loss. A mixed setup avoids both.
One of our engineers, Yann, describes the goal like this: "our job is to get Vybe as close as possible to that experience through various AI engineering efforts". Routing, retry logic, prompts, guardrails: a good part of the "Claude Code feels better" experience is engineering around the model. Engineering iterates faster than weights do, so we work on closing the gap instead of paying for it on every task.
The onboarding question we have not answered
Yann has also been asking something we don't have a clean answer to. In his words: "I wonder how different the user experience would be if we are using more premium models for the onboarding?"
His point deserves better than a shrug. First impressions form in the first session, and onboarding is short, ambiguous and guidance-heavy, the kind of interaction where premium models are at their best. The cost math allows a hybrid policy: route the first session through a premium model, then hand the steady-state work to GLM 5.3 Flash. At the showcase's $0.25 per Opus task, a new user's first ten tasks cost about $2.50 even at premium rates, because onboarding is a small share of the calls a user will ever make.
The risks are real. Two models behave differently, and a user who gets a brilliant first session and a slightly sloppier second one may read that as the product degrading. Mixing models also means two sets of failure modes to debug. We have not shipped any of this. Read the whole section as a working hypothesis.
What would settle it
Everything above is argument and arithmetic. Proof would look like this: task completion rate and retries per task, onboarding drop-off, time to first successful outcome and cost per completed task, each measured for a premium-onboarding group against a cheap-everywhere group. We have not run that experiment. When we do, we will publish it either way.
In the end, "better" only means something once you attach a workload: a task, a price and a failure mode you can live with. We run a model 14× cheaper than Opus and we publish the evals showing where it loses. The premium experience is a fine product too. The interesting question, the one Yann keeps asking, is how much of the distance between the two turns out to be engineering.
