GPT-5.5 and the Long-Horizon Test

By Daniel Ensminger

Week of published 4 min read

GPT-5.5 made long-running agent work the headline. The harder question is how to evaluate whether an agent can finish responsibly.

OpenAI announced GPT-5.5 on April 23, 2026, then made GPT-5.5 and GPT-5.5 Pro available through the API on April 24. This retrospective is dated for the week of that release; it was published later as part of a researched archive.

The factual part of the announcement is straightforward. In its launch post, OpenAI described GPT-5.5 as a model for agentic coding, computer use, knowledge work, scientific research, and other tasks that require moving between tools. The company also published results for Terminal-Bench 2.0, SWE-Bench Pro, and its internal Expert-SWE evaluation.

Those numbers are OpenAI's measurements, not independent proof of how the model will perform in every codebase. The more durable part of the release is the question it puts in front of anyone adopting agents: what should count as a successful long-horizon result?

A longer run is not automatically a better run

It is easy to measure whether an agent stayed busy. It is harder to determine whether the activity converged on the right outcome.

A useful long-horizon task usually contains ambiguity, several tools, and more than one kind of evidence. Updating a product may require reading its existing contracts, changing code, running checks, inspecting the real interface, and confirming that the deployment serves the intended result. An agent can spend an hour on that sequence and still fail because it optimized the wrong layer.

That makes duration a poor proxy for autonomy. The real test is whether the system preserves intent as the task expands. It should notice when a local fix conflicts with an established product rule, distinguish a build from a release, and report a missing credential or external approval as a remaining gate rather than quietly redefining “done.”

Four things worth evaluating

The first is continuity. Does the agent retain the important constraints after many tool calls, branches, and intermediate failures? A model that writes excellent code but forgets the requested compatibility boundary halfway through the task is not reliable enough for unsupervised work.

The second is recovery. Real repositories contain dirty worktrees, flaky services, undocumented assumptions, and partial migrations. A capable agent should diagnose a failed path, preserve unrelated work, and choose a narrower alternative when appropriate. Repeating the same command with more confidence is not recovery.

The third is verification. Completion should be tied to evidence that matches the claim. Type checking can support a code claim. A browser run can support an interaction claim. Neither alone proves that an account setting, production environment, or physical device works. Long-horizon agents need to keep these evidence classes separate.

The fourth is stopping judgment. Some of the most important agent behavior is knowing when not to act. A request can reach a point where continuing would require a product decision, a consequential external write, or authority the agent does not have. A good run makes that boundary legible instead of stepping over it for the sake of a clean completion message.

Benchmarks are a starting point

Coding benchmarks are useful because they create repeatable tasks and make model changes easier to compare. They cannot fully represent the surrounding work of maintaining a product.

A repository task often includes local conventions, visual acceptance, deployment state, and facts that live outside the code. It may also require the agent to challenge an incorrect premise. These are system properties produced by the model, its tools, its instructions, and the review process together.

That suggests a practical evaluation set for a small product team: use a group of real completed tasks, remove the final changes, and ask the agent to recreate the result. Score not just the patch, but also preservation of unrelated work, tests chosen, visual inspection, source attribution, and accuracy of the final status report. Record the human interventions required along the way.

The point is not to manufacture one overall score. It is to expose where trust breaks.

The release shifted the useful question

OpenAI's announcement presented GPT-5.5 as a stronger model for work that spans context and tools. Whether it earns that description in a particular workflow has to be tested there.

The useful question is no longer only, “Can the model produce the change?” It is also, “Can the surrounding agent preserve the contract, gather the right evidence, and stop with an honest account of what remains?”

That is the long-horizon test. More tokens and more time make it possible to attempt. They do not make it pass automatically.

👍
❤️
🔥
👏
🤯

Get infrequent updates on new projects.

By subscribing you consent to email updates. You can unsubscribe anytime. See our Privacy Policy.