An API endpoint has a price, and the price is not the number on the pricing page. For a human developer, a call costs some milliseconds and a fraction of a quota. For an agent, a call costs three things at once: wall clock time the user is waiting through, context tokens that could have been spent reasoning, and money that somebody is paying per token.
The third one is the change. A model that calls an endpoint pays for the response by the token, in the same budget it uses to think. A verbose response is not merely inefficient. It consumes the capacity the model needed to do something useful with the data it just received.
Designing an API for agents means designing against a budget the caller cannot expand. Endpoints that are cheap in the traditional sense, fast and free, can still be expensive enough that a model stops using them.
Three currencies, spent simultaneously
Every tool call draws on three separate accounts, and an endpoint can be cheap in one and ruinous in another.
Latency is the most visible. A user waiting on an agent experiences the sum of every call in the chain, plus the model's thinking time between them. Agents make many calls, and the calls are frequently sequential because each one informs the next. Ten calls at two seconds each is twenty seconds of dead time, which is enough for a person to decide the system is broken.
Tokens are the least visible and usually the largest. Every byte of a response enters the context window. It stays there for the rest of the conversation, occupying space, competing with the reasoning the model needs to do. A response that is technically small by network standards can be a meaningful fraction of a working context.
Money follows tokens, but not linearly. Context is reprocessed on subsequent turns, so a large response is paid for repeatedly rather than once. An oversized payload early in a long session keeps costing for the rest of it.
The consequence is that the traditional optimisation targets are wrong for this caller. Compression does not help, because the model sees the decompressed text. Field count matters more than byte count, because structure costs tokens. And a fast endpoint that returns a wall of data is worse, from the agent's point of view, than a slower one that returns a summary.
A fixed reasoning budget changes which tools get used
Agents operate under limits: a maximum number of steps, a context window, a time or cost ceiling on the task. Within those limits, the model is implicitly triaging. Every call it makes is a call it does not make elsewhere.
This produces selection effects that API designers do not usually think about. An endpoint that reliably answers in one call gets used. An endpoint that requires three calls to produce anything useful, a discovery call, then a lookup, then a fetch, competes badly against a single call that does the whole thing, even if the multi-step version returns better data. An endpoint that returns a large payload gets used once and then avoided, because the model has learned within the session that it is costly.
There is also a first impression effect. Early failures or slow responses shape tool selection for the rest of a session. A tool that times out on its first call frequently gets dropped entirely, and the model completes the task using worse sources or its own priors.
The design implication is that the unit of value is the completed question, not the endpoint. If answering a common question requires four of your endpoints, that is a four call question, and it should probably be one endpoint. Composite endpoints that would be considered poor REST design are often exactly right for agent traffic, because they collapse a sequence of dependent calls into a single exchange.
Response size is the hidden cost
This is the failure that most often makes an otherwise good API unusable by agents, and it is entirely a design choice.
A spatial query can return an enormous amount of data legitimately. Every building in a district, with full geometry, comes to a large payload of coordinate pairs. To a browser rendering a map, that is normal. To a model, it is a flood of numbers that conveys almost nothing per token and displaces the reasoning it was supposed to enable.
Several specific patterns cost far more than they return:
- Full precision geometry when the question was about counts or presence. Coordinate arrays are the single largest token sink in geospatial responses.
- Every attribute a record happens to have, including internal identifiers, source keys, and fields that exist only for backwards compatibility.
- Deeply nested envelopes, where each value is wrapped in an object with metadata. Structure is not free, and every bracket and key name is tokens.
- Verbose null representation, where absent fields are included explicitly across thousands of records.
- Repeated boilerplate per record: the same licence string, the same source name, the same coordinate system declaration, once per feature instead of once per response.
The remedies are unglamorous and effective. Default to the summary and let the caller ask for detail. Simplify geometry unless full precision was requested, or return a representative point and an area rather than a polygon. Omit null fields. Hoist anything constant across the response into a header object. And give the caller a field selection parameter so a model that needs three attributes is not paying for thirty.
A reasonable test: take a typical response, and ask what fraction of its tokens could influence an answer. If most of them are coordinates the model will never inspect individually, the default shape is wrong.
Pagination and sampling defaults
Defaults are the actual interface, because the first call a model makes uses them. A model that has to discover a limit parameter through failure has already paid for the discovery.
The default page size should be small enough that a first call cannot blow a context budget. What that number is depends on the record size, which is the point: the limit that matters is tokens, not records. A hundred records with two fields each and a hundred records with full polygon geometry are wildly different costs under the same limit. Enforcing a byte or token ceiling alongside the record count is more honest than a record count alone.
Pagination itself has to be cheap to use. Cursor based paging is better than offset paging here, because the cursor is a single opaque value the model can carry forward without arithmetic, and because it stays correct while the underlying data changes. Whatever the mechanism, the response must state whether more results exist, and ideally how many, so the model can decide whether to keep going rather than paging blindly until an empty result arrives.
Sampling deserves more use than it gets. For many analytical questions, a representative subset plus a total count answers the question better than the full set. An agent asked what kind of businesses are in a district needs the distribution and a few examples, not every record. Offering a sampled mode, clearly labelled as sampled, with the total reported honestly, is often the single largest cost reduction available.
Two guard rails prevent this from becoming dishonest. The response must always distinguish truncation from completeness, because a model that reads a truncated list as the full list will produce a wrong answer with full confidence. And a sample must never be presented in a way that allows it to be summed or averaged as though it were the population.
Cold starts, and declaring latency before it is discovered
Agent runtimes apply a default timeout to tool calls, chosen without knowledge of any particular tool. When a call exceeds it, the runtime abandons the call and reports failure. The model then usually stops using the tool for the rest of the session.
That behaviour is catastrophic for endpoints whose first call is slow and whose subsequent calls are fast. Cold containers, index loading, cache warming, first fetch from a slow upstream: all of them produce exactly this shape, and all of them get abandoned before the fast path is ever observed.
The remedy is declaration. A tool definition that states typical latency and worst case latency allows a runtime to configure a timeout that fits. Cold start should be called out separately, because it is the case a caller is most likely to encounter first and least likely to expect. There is no cost to saying that the first call may take considerably longer than the rest, and considerable benefit: a caller that has been warned waits.
For work that genuinely takes minutes, the synchronous shape is the wrong one no matter how the timeout is configured. Accept the request, return a handle immediately, and tell the caller how long to wait before polling. That converts an unpredictable blocking call into two predictable short ones and removes the abandonment risk entirely.
Where a slow path exists alongside a fast one, splitting them into separate endpoints is worth doing. A caller can then choose the cheap approximate answer or the expensive exact one, rather than discovering the difference through a timeout.
Expensive endpoints need a declaration and a tighter limit
Some endpoints are not merely slower. They trigger inference, queue background work, or fetch from a metered upstream. Their marginal cost is real and it is not the caller's to absorb silently.
Two things are required, and one alone is insufficient.
The declaration comes first. The tool description should say that this endpoint is expensive, in what way, and what a caller should do instead when the cheap answer will do. Models respond to this. A description that says a vision endpoint runs inference on imagery and should be called only when a question cannot be answered from vector data measurably changes which tool gets selected. Cost signals in a description function as guidance, not just as documentation.
The limit comes second, because declaration is advisory and an agent in a loop does not read advisories twice. Expensive endpoints need their own rate limits, tighter than the general one, enforced independently. A shared limit across cheap and expensive endpoints means a caller can exhaust the expensive capacity while nominally staying within their allowance.
The limit should also be legible. A rejection that says the caller has exceeded the limit for this specific class of endpoint, that the general limit is unaffected, and when the expensive one resets, lets the agent continue its task using the cheap endpoints. A rejection that says only "rate limited" causes the agent to back off from everything.
Where cost per call varies by request rather than by endpoint, quoting is better than guessing. Let the caller ask what an operation would cost before committing to it. That is a small amount of work and it makes budgeting possible instead of speculative.
Designing for the agent that calls in a loop
At some point, something will call your endpoint in a tight loop. Not maliciously. A model will write an iteration over a list, or a retry policy will fire without backoff, or a task will be structured so that a call happens once per candidate in a set of thousands.
Building for this is not the same as defending against abuse. Abuse defence is about blocking. This is about making the loop cheap for everyone, including the caller who did not realise what they had written.
A few properties help disproportionately. Idempotent reads with strong caching mean the second thousand calls cost almost nothing. Batch endpoints let a loop over a thousand items become ten calls instead of a thousand, and they should be advertised prominently in the tool description, because a model that knows a batch endpoint exists will usually use it. Cheap rejection matters: a rate limited response should be the cheapest thing your service can produce, so that being hammered does not consume the resources needed to serve everyone else. And backpressure should be explicit, with a delay in the response, because without one a runtime retries immediately and turns a limit into an outage.
The complementary practice is to look at your own traffic for loops that should not exist. Repeated identical calls within seconds usually mean a retry policy without backoff or a model that did not realise it already had the answer. Sequences that always occur together are a missing composite endpoint. Callers paging through everything are a missing aggregate. Each of those patterns is a design gap on your side, visible in the logs, and fixing it reduces cost for both parties.
Measuring what you actually charge
Most API teams measure latency and request volume. Neither captures the cost imposed on an agent caller.
The number worth tracking is the token cost of a typical response, per endpoint. It can be approximated well enough by size, and it should be watched over time, because responses grow. A field added for one customer stays in every response forever, and the cumulative effect is an endpoint that has quietly doubled in cost with no single change responsible.
Two secondary measures earn their keep. Calls per completed task, which reveals whether a common question requires an unreasonable number of round trips. And the abandonment rate, meaning calls that were opened and never read to completion, which is the observable signature of timeouts being hit.
Treating response size as a budget with a stated ceiling, defended in review the way latency budgets are defended, is the practice that prevents slow inflation. Adding a field should require justifying its cost to every caller who did not ask for it.
The honest limit
None of this makes an endpoint free. Some questions require large answers, some computations take real time, and pretending otherwise by truncating without saying so trades a cost problem for a correctness problem, which is a worse trade.
The goal is not minimal responses. It is honest ones, with the cost visible in advance and controllable by the caller. An endpoint that says what it will cost, defaults to the cheap answer, offers the expensive one on request, and states plainly when it has returned less than everything, can be budgeted against. An endpoint that is merely small cannot be trusted, and an endpoint that is merely complete cannot be afforded.