Not Everything Should Cost a Token: The Case for Deterministic AI
A team I talked to recently wired up an agent to do something simple: pull a metrics API every morning, reshape the JSON, and drop the result into a table. Clean idea. It worked on day one.
Then the invoice came in. Every morning, that "simple" job was loading a few thousand tokens of raw JSON into a context window, asking a language model to reformat data it did not need to reason about, and paying for the privilege. The reformatting was occasionally wrong, too, because a probabilistic model was doing a job that a five-line script does the same way every time. The context window was so stuffed with stale data that the actual reasoning the agent was supposed to do got worse.
That trap catches almost everyone right now. Teams treat the language model as a universal runtime. If you can describe a task in a sentence, you prompt it. But a model is not a cron job and it is not a database. It is a reasoning engine you are renting by the token, and the fastest way to blow up your bill and degrade your output is to make it do work that never needed a brain in the first place.
The skill worth learning is knowing what not to tokenize.
Why "everything is a prompt" feels right and costs so much
The appeal is obvious. Prompting is fast. You skip the schema design, the endpoint, the deploy. You just ask. For a one-off, that is genuinely the right call.
The problem shows up when the one-off becomes a recurring job. Every deterministic task you route through a model inherits three properties it should never have had: it becomes non-deterministic, it becomes slow, and it becomes metered. You now pay, per run, for work that a scheduled function would do for free and never get wrong.
Two symptoms follow, and they compound each other.
The first is cost that scales with the wrong thing. Your token spend should track the value of the judgment you are asking for. Instead it tracks the volume of mechanical work you have shoved through the model. Pull ten APIs a day instead of one and your bill grows tenfold, even though none of that pulling required intelligence.
The second is context bloat. To "process" data, teams read it into the context window. But the context window is finite and it is where reasoning happens. Fill it with raw records and you crowd out the thing the model is actually good at. Quality drops, and because most providers meter input tokens, the cost of a long context climbs at the same time. You pay more to get worse answers. That is the part that stings.
The distinction that fixes it: probabilistic vs. deterministic work
Here is the mental model I keep coming back to. Sort every task into one of two buckets before you decide where it runs.
| Belongs in the Agent (probabilistic) | Belongs in the App (deterministic) |
|---|---|
| Judgment and interpretation | Scheduled and recurring tasks |
| Drafting, summarizing, classifying | API calls and data transforms |
| Deciding what to do and whether something matters | Storing, retrieving, and querying data |
| Handling ambiguity and edge cases | Anything that must be exact and repeatable every single run |
The line is simple. If a task needs judgment, it belongs with the agent. If it needs to be exact and repeatable, it belongs in an app. Determinism is a feature, not a limitation. A scheduled job that runs identically every morning is something you want to be boring and predictable.
And here is the anchor line for the whole idea: you build the app with tokens once, then you stop paying tokens every time it runs. The intelligence goes into creating the machine. The machine itself runs for free.
"Learn what not to tokenize" is a discipline you practice on every task, not a setting you toggle once.
The one everyone gets wrong: memory notes as a database
There is a specific version of this mistake that is worth calling out on its own, because I see smart people do it constantly.
People use their agent's memory or notes as a database.
They start dropping structured data into notes: records, metrics, pipeline stages, customer lists, inventory counts. It feels natural because the note is right there and the agent already reads it. But notes are the wrong tool for structured data, and the reasons pile up fast. There is no schema, so nothing enforces consistency. There is no query, so the agent reads the entire note into context every time it needs one field, which is both slow and metered. And notes degrade as they grow, because a wall of semi-structured text is exactly what a real table exists to replace.
Now here is the part that gets lost in the "just use a database" advice, and it matters: memory notes are not the problem. Using them for the wrong job is.
Notes are genuinely the right tool for a whole category of context that has no clean schema and never will. Preferences. Tone and voice. Patterns the agent has learned. Decisions and the reasoning behind them. The "here is how we do things" knowledge that shapes how an agent thinks rather than something you would ever run a SELECT against. That context is unstructured on purpose, it exists to inform judgment, and forcing it into a rigid table would strip out the nuance that made it useful.
So the rule cuts both ways:
- Structured, queryable, high-volume data goes in a real database.
- Unstructured, interpretive, reasoning-shaping context stays in notes.
Put a customer list in your notes and you will feel the pain. Put your brand voice guide in a database and you have over-engineered something that was fine as prose. Match the tool to the shape of the data and both problems disappear.
The Vybe model: AI and apps, each in its own lane
This is where the architecture stops being a philosophy and starts being a product decision.
On Vybe, agents do not just chat. They build and operate real applications. That single fact is what makes "learn what not to tokenize" actually achievable instead of just good advice, because the agent has somewhere to put the deterministic work.
The pattern looks like this. Recurring work gets scheduled inside the app, not held together by asking the model to remember to run it. Data pulls and transforms live behind an app endpoint that the agent calls when it needs them, so the mechanical steps run as code instead of as prompts. State lives in a real database that the agent queries deterministically. The agent orchestrates and reasons. The app executes and remembers. Each layer does the job it was actually built for.
Two examples make it concrete, and one of them is our own.
Competitor Radar. We built a competitive-tracking agent internally, and it would have been easy to let it hoard competitor data in its context or its notes. It does not. It built a live database app with scheduled refreshes. The scheduling, the pulling, and the storing all happen at the app layer as deterministic work. The agent only spends tokens on the part that needs a brain: deciding what is actually worth flagging. That system is shipped and running, and its token cost is a fraction of what it would be if the agent were doing the plumbing itself.
Falcon. In our gallery, Falcon is a Competitive Intelligence Analyst that monitors competitor changelogs, blogs, and social activity every day and delivers scheduled intelligence briefs by email. Same architecture, customer-facing this time. The monitoring cadence and the email delivery are deterministic app-level plumbing. They run on a schedule, identically, every day. The only thing Falcon spends tokens on is the judgment call: is this change actually a signal, or is it noise? The architecture doing exactly what it is supposed to.
Notice what both have in common. The expensive, intelligent resource is pointed at the one part of the job that needs intelligence. Everything else runs as cheap, reliable, deterministic code.
Why this is hard to copy
Most tools on the market cannot do this, and the reason is structural rather than a matter of polish.
Pure chat-agent assistants, the single-model copilots and Slack bots, have no app layer to offload to. There is nowhere to push the deterministic work, so it all stays inside the model. Every scheduled task, every data pull, every lookup runs as tokens because tokens are the only thing those tools have. Their architecture forces the exact cost problem we started with.
Pure no-code and app builders have the opposite gap. They can run deterministic apps all day, but there is no reasoning layer sitting on top deciding what matters.
Vybe is the platform where the reasoning agent and the deterministic app are both first-class and built together, in the same place, by the same system. And because Vybe is model-agnostic, you are not locked into one vendor's token rates for whatever mechanical work does still run through a model. You get the best of both worlds instead of paying a premium to force one world to imitate the other.
A checklist for what not to tokenize
Before you prompt, run the task through these questions.
- Does this need judgment, or is it just mechanical? Judgment goes to the agent. Mechanical goes to an app.
- Does it repeat on a schedule? Move it into an app-level scheduled task and stop paying per run.
- Is the agent making an API call directly? Replace it with an app endpoint that makes the call, and have the agent invoke the endpoint.
- Are you storing structured data in notes? Move it to a database and query it when you need it.
- Is the data in your context there to be reasoned over, or just to be stored? If it is storage, get it out of the window and reserve that space for actual thinking.
Run through that list a few times and it becomes instinct. You start feeling which tasks deserve a token and which ones are quietly robbing you.
The point was never "more AI"
The goal of a good agent system is not to route everything through a model. It is to apply intelligence exactly where it earns its cost, and to let deterministic code handle everything else quietly and cheaply.
That is the whole game. Work out what actually deserves a token, build the machine once, and let it run on its own. Your model should be spending its time on judgment, not plumbing.
Vybe is built for precisely this: agents that reason and apps that execute, together, so you stop paying a premium to make one do the other's job. See what the AI plus apps model looks like in practice, or browse the agent gallery to see the pattern already running.
