Insights · Field Notes 04
Gateway or Not?
In Token cost: $/1M tokens? No!, we explain why AI cost should be measured in dollars per outcome, not just dollars per million tokens. Here we go deeper into the technical problem and what we believe is the right solution.
Today, putting a gateway in front of OpenAI / Claude / Gemini is popular. Teams tag each call by team, user, app, or workflow and track tokens + cost.
This is a suboptimal approach for long-horizon agents. One agent job can create dozens of model calls, tool calls, retries, branches, subagents, evals, human approvals, and even resume the next day. The gateway sees many calls, and engineers try to connect everything back together using tags, traces, and workflow IDs.
We believe the root cause is treating agents as ephemeral objects.
We take a different approach. We treat agents as first-class, durable, persistent objects from the start, with their own state, lineage, budget, and history. Much like an employee.
Everything the agent does (model calls, retries, branches, subagents, waits, resumes) stays attached to the same execution.
This gives you two things:
- While the agent is running: control how much that job is allowed to spend.
- After it finishes: connect the full cost of the execution to the outcome.
Cost control is just one piece of the puzzle. Our approach gives you full control over agents. More to come.
