Designing Error Responses an Agent Can Recover From

Designing Error Responses an Agent Can Recover From

E
By Etzal Earth
13 min read

Error handling was designed for two audiences, and neither of them is a language model. Status codes were written for a client library, which needs to know whether to raise. Error messages were written for a developer reading a log at midnight, who can open the documentation, look at the request, and work out what went wrong.

A model has neither capability. It cannot open the documentation. It has whatever was in the tool description, whatever it inferred from previous calls in this session, and the response body in front of it. From that alone it has to decide, in one step, whether to retry the same call, change the call, try a different tool, or tell the user the task cannot be completed.

Error responses that were adequate for both traditional audiences turn out to produce specific, expensive, repeatable failures with this one. Getting them right is not a courtesy. It is the difference between an agent that recovers and an agent that burns a reasoning budget hammering an endpoint that was never going to answer.

What a model actually does when a call fails

Watching agent traces against a real API, failure behaviour falls into a small number of patterns, and they are not the patterns a human exhibits.

Immediate retry is the most common. The model sees a failure, has no information suggesting the failure is permanent, and calls again with identical arguments. A human would pause and think. A runtime with a retry policy does it automatically, sometimes several times, sometimes without any backoff at all. If the error was permanent, every one of those calls is waste. If the error was a rate limit, the retries make it worse.

Reformulation is the second. The model assumes the request was malformed and changes something, often the wrong thing. Asked for a bounding box that was rejected, it will try a smaller box, then a point, then a different endpoint, wandering through the parameter space in search of something that returns a two hundred. This can burn a large share of a task budget and frequently ends with a call that succeeds but answers a different question than the one the user asked.

Abandonment is the third, and it is the quiet one. The model concludes the tool is unavailable and either says so or, worse, proceeds without it and produces an answer from its own priors. A single unexplained timeout can remove a data source from the rest of a conversation.

The fourth is the one worth designing for: the model reads the error, understands what happened, and issues a corrected call. This happens reliably when the error says what to do, and almost never when it does not.

Why a bare status code teaches nothing

A four hundred with an empty body tells a model that something was wrong with its request. It does not say which parameter, in what way, or what would be acceptable. The model's only available strategy is guessing, and guessing over an unknown parameter space is expensive.

The failure is worse for spatial APIs than for most, because spatial requests have many interacting constraints. A bounding box can be rejected because the coordinates were transposed, because the box crosses the antimeridian, because it exceeds an area limit, because the coordinate reference system was not the one assumed, because minimum and maximum are the wrong way round, or because the area is outside the coverage of the dataset. Every one of those is a four hundred. Every one has a different fix. A response that does not distinguish them forces the model to try the fixes in whatever order its priors suggest.

The same applies to a bare five hundred. It does not say whether the problem is transient, whether a retry is sensible, or whether one upstream is down while everything else works. A model treats an unexplained five hundred as a coin flip and usually retries, because retrying is what its training suggests.

Status codes are still worth getting right, because tooling depends on them. But the code is the envelope. The recovery information has to be in the body.

The four things a recoverable error has to say

An error a model can act on answers four questions. Anything less and the model is inferring.

What happened, in specific terms. Not "invalid request" but which field was invalid and what was wrong with it. If a value was out of range, give the range. If a value was unrecognised, say what kind of thing was expected. Name the field using the same identifier the request used, so the mapping is mechanical rather than interpretive.

Whether retrying helps. This is a single boolean and it is the highest value field in the whole response. If the call cannot succeed without a change, say so explicitly, and the retry loop stops. If it can succeed unchanged, say that too, because it converts a guess into a plan.

When to retry, if retrying helps. A delay in seconds. Without it, a model either retries immediately, which is usually harmful, or picks an arbitrary wait, which is usually wrong in one direction or the other. Rate limit responses should always carry this. So should responses from a service that is warming up.

What a valid request would look like. This is the field most often omitted and the one that changes behaviour most. A rejected bounding box that comes back with an example of an acceptable one, or a corrected version of the submitted box, converts a search problem into a copy. An unrecognised layer name that comes back with the list of valid layer names ends the reformulation loop in one step.

The last of these has a cost worth naming: it makes error bodies bigger, and bigger bodies consume context. The resolution is to keep the guidance short and specific rather than dumping a schema. A list of valid values is useful when the list is short. When it is long, a pointer to the discovery endpoint that returns it is better than truncating.

Whose fault it was, and why the model needs to know

The client error against server error distinction exists in HTTP for a reason, and it maps directly onto the only decision the model has to make.

If the fault is the caller's, retrying identically is guaranteed to fail, and the correct behaviour is to change the request. If the fault is the server's, changing the request is usually pointless and may produce a worse question, and the correct behaviour is to wait and retry the same call.

Getting this wrong in either direction produces a characteristic pathology. A validation failure returned as a five hundred causes a model to retry a request that can never succeed. A transient upstream failure returned as a four hundred causes a model to mutate a request that was perfectly good, and it will keep mutating until something returns a two hundred, at which point it will confidently answer a different question.

Three cases deserve their own treatment rather than being forced into one bucket. A rate limit is the server declining, but it is the caller's behaviour that caused it, and the only correct response is to wait, so it needs both an unambiguous code and a delay. An authorisation failure is the caller's fault but is not fixable by reformulating the request at all, and should say so plainly, because a model that does not understand this will try every endpoint in the manifest looking for one that works. And an upstream dependency failure is the server's fault from the caller's point of view even though the server itself is healthy, and it should be reported as such, with the name of the affected data source, because that tells the model which parts of its plan are still viable.

Partial success is the normal case, not the exception

Any API that assembles an answer from several sources will regularly have some of them fail. The binary model of success and failure has no place to put this, so it gets forced into one of two lies: return two hundred and silently omit the missing part, or return an error and discard the parts that worked.

Both are bad, and the first is worse. Silent omission is indistinguishable, to the caller, from the missing thing not existing. An agent asked whether a site is exposed to flooding, receiving a response with no flood layer because the hazard feed timed out, will report no flood exposure. That is not a degraded answer. It is a wrong one.

The workable pattern is to make completeness explicit in the response. Return the parts that succeeded, and alongside them a record of what was requested, what was delivered, and what failed with a reason for each failure. The status code should still be a success, because the response contains usable data, but the body should make the gap impossible to miss.

This changes what a well built agent does. Instead of answering from partial data as though it were complete, it can decide: proceed with what it has and caveat the answer, retry only the failed component rather than the whole call, or tell the user which specific part of the question cannot be answered right now. All three are better than a confident answer built on a silent hole.

The corollary is an operational rule: a partial response must never be cached at the lifetime of a complete one. A degraded answer stored for hours and served to every subsequent caller turns a brief upstream outage into a long data quality incident, and none of the affected callers has any way to know.

Empty result against no coverage

This is an error design problem even though it does not look like one, and it is the most consequential distinction in a geospatial API.

An empty array can mean two entirely different things. It can mean the query ran correctly against complete data and there is genuinely nothing there: no hospitals within two kilometres, no recorded flood events in this district. Or it can mean the query ran correctly against data that does not cover this area at all: no hospitals returned because the dataset has never been extended to this country.

The first is a finding. The second is a gap. They are opposite in meaning and identical in JSON, and the pattern of where coverage is thin is not random. Coverage tends to be worst in exactly the places where an answer would matter most, so treating absence as evidence produces a bias that runs in a consistent direction.

The response has to separate them, and it has to do so in a field a model will actually read rather than in prose in the documentation. Practically:

  • State coverage explicitly for the requested area: whether the dataset covers it, and if known, how recently it was surveyed there.
  • When coverage is absent, say so as the primary content of the response rather than returning an empty list with a note attached.
  • When coverage is partial, give the extent that was actually searched, so the caller knows the answer applies to a smaller area than requested.
  • Name the dataset, because the caller may know of another source for that area even when this service does not have one.

Timeouts, and the abandonment that follows them

Timeouts deserve separate treatment because their failure mode is the most damaging and the least visible.

An agent runtime typically has a default tool timeout, chosen without reference to any particular tool. If a call has not returned by then, the runtime abandons it. The model is told the tool failed, and from that point it usually stops using the tool. It does not know whether the call was two seconds from completing.

Several perfectly ordinary things exceed a default timeout. A cold container that has to load an index. A query over a large area. An upstream fetch that is itself slow. A computed layer that has not been generated for this region yet. In each case the call would have succeeded given a few more seconds, and the runtime gave up first.

The fix is to declare expected latency rather than leaving it to be discovered. A tool definition that states typical and worst case latency lets a runtime set a timeout that fits the tool instead of the default. An endpoint that knows it will be slow can say so before doing the work, either by returning a header with an expected duration or by accepting the request and returning a handle to poll. For anything that can genuinely take minutes, the asynchronous shape is the only honest option: acknowledge, return a job identifier, tell the caller how long to wait before checking.

Cold start deserves an explicit mention in the declaration, because it is the case where the first call is dramatically slower than every subsequent one. A caller that has been told the first call may take significantly longer will wait. A caller that has not will abandon on the first call and never see the fast ones.

Writing for a reader who is a model

Error text is prose that a model will read, and the conventions that make it useful are not quite the conventions of good developer messages.

Say the specific thing first. Models attend strongly to the beginning of a field. A message that opens with a service name and a correlation identifier and gets to the actual problem in the third clause loses some of the signal.

Use the caller's vocabulary. If the request field is named bbox, the error says bbox, not "bounding box parameter" or the internal name of whatever validated it. The model has to map the message onto something it can change, and an exact string match makes that mapping trivial.

Do not blend guidance with apology. "We are sorry, an unexpected error occurred, please try again later or contact support" contains one usable bit and a great deal of text that consumes context without informing a decision. Support contact details help a human and do nothing for a model. If both audiences need serving, put the human oriented material in a separate field.

Keep the shape identical across every error the service can produce. A model that has seen the error schema once, in a tool description or in an earlier failure, can parse every subsequent error without re-learning. Endpoints that each invent their own error shape force the model to interpret prose every time.

The failure that actually ships

The error design that causes the most damage is not the ugly one. It is the polite two hundred that carries an incomplete answer without saying so.

Ugly errors get fixed, because somebody sees them. A response that returns eighty percent of the requested layers with a cheerful success status produces no alert, no log line anyone reads, and no complaint, until an analysis built on it turns out to have been wrong for a month. The agent that consumed it had no way to know. It was told everything was fine.

The rule that follows is uncomfortable but simple: an API should be more willing to fail loudly than to succeed quietly with less than it promised. A caller that gets a clear, specific, actionable failure can recover. A caller that gets a confident partial answer cannot, because it does not know there is anything to recover from.