ARC Prize published two figures for GPT-6 Astra on ARC-AGI-3: 62.7% at $26K under the Standard harness, 99.9% at $19K under a vendor-supplied Provider Adapter that preserves reasoning state between turns. The cheaper, higher score rules out extra compute as the explanation — the Provider Adapter runs used 49% fewer tokens. The gap measures how much of a hard interactive task is rediscovery, not capability.
On 3 September, OpenAI released GPT-6 Astra, and within hours a single number was circulating: 99.9% on ARC-AGI-3, the hardest of the ARC Prize benchmarks. The number is real. So is the other one ARC Prize published the same day, from the same model, on the same benchmark: 62.7%.
The 37-point gap is not a discrepancy, a correction, or a retraction. Both figures come from ARC Prize's own results, disclosed voluntarily and side by side. They differ because they were produced under two different test harnesses -- the software layer that sits between the benchmark and the model and decides what the model is allowed to remember between turns. That layer used to be invisible. It is now the most important variable in the number.
ARC Prize published both results on the ARC-AGI-3 Semi-Private evaluation set. Under the Standard harness, GPT-6 Astra scored 62.7% at a compute cost of roughly $26,000. Under a configuration ARC Prize calls the Provider Adapter, it scored 99.9% at roughly $19,000. Note the ordering of those costs; it matters later.
The Standard harness is the configuration every model is measured under, which is what makes cross-model comparison possible at all. Under it, 62.71% is itself a record. Claude Opus 5 scored 30.2% on the same benchmark. GPT-5.6 Sol, OpenAI's own prior frontier model, scored 7.8%. Astra roughly doubled the best prior score under shared conditions. That is a substantial result, and it is not the one being shared.
ARC-AGI-3 is an interactive benchmark rather than a static one. Instead of presenting fixed puzzles to be answered in one shot, it drops the model into novel game environments it has never encountered and requires it to work out the rules by acting and observing consequences. The design is deliberate: it tests whether a system can generalise into a genuinely unfamiliar environment rather than pattern-match against something in its training data.
That design makes the benchmark unusually sensitive to memory. Under the Standard harness, Astra could carry visible notes between requests -- text it had written to itself and re-read at the start of the next turn. Under the Provider Adapter harness, ARC Prize preserved the model's opaque reasoning state between requests, with compaction for longer conversations. The model was not reloading a summary of what it had worked out. It was resuming from where it left off.
The distinction matters more here than it would on a static test. When the difficulty of a task is figuring out unfamiliar rules by experiment, the expensive phase is the first one -- forming hypotheses, testing them, discarding most of them. A harness that preserves working state carries that expensive work forward intact. A harness that forces the model to serialise its findings into notes and reload them loses whatever did not survive the round trip.
A benchmark score is a measurement of a model plus the scaffolding it was measured through. Once that scaffolding becomes a variable the vendor can supply, the score is partly a claim about the scaffolding.
The higher score cost less. Nineteen thousand dollars against twenty-six thousand -- roughly 27% cheaper for 37 points more. That ordering is worth sitting with, because it rules out the simplest available explanation.
If the Provider Adapter were simply buying more compute -- more attempts, longer chains of reasoning, more sampling -- it would cost more, not less. ARC Prize's own measurements point firmly the other way. Across the 167 game-reasoning pairs that both harnesses solved, the Provider Adapter runs consumed 49% fewer tokens and completed roughly 3.66 times faster.
So the gain is not additional thinking. It is the elimination of repeated thinking. Under the Standard harness, a meaningful share of the compute was being spent re-deriving conclusions the model had already reached in an earlier turn and then lost at the boundary between requests. Read that way, the 37-point gap is a measurement of how much of a hard interactive task consists of rediscovery.
That is a more interesting finding than the headline, and a more useful one for anyone building agentic systems: it puts a number on what state persistence is worth on tasks of this shape. It is also not what '99.9% on ARC-AGI-3' communicates to someone looking at a screenshot.
Several limits apply, and none of them is small.
One point should be stated plainly, because it tends to get lost in the retelling. The harness split was published by ARC Prize itself, in its own write-up, with both scores and both cost figures presented together. Nobody caught anybody out. The organisation running the benchmark voluntarily disclosed a methodological variable that made its own headline number look softer.
That is the right behaviour, and it is the only reason this piece can be written. The failure is downstream. Coverage and social sharing picked up the larger number and dropped the condition attached to it. OpenAI's framing did not help: the company presented Astra in AGI terms and described it as the most precise symbolic model of novel environments recorded to date. Those are the vendor's characterisations of its own product, and characterisations travel further than footnotes about which harness produced which score.
For anyone selecting models on published evaluations, the practical change is that a score without a harness specification is no longer a complete claim. Two questions now belong next to every headline figure: which harness produced it, and was that the same harness the comparison models ran under.
The Standard-harness number is the one that supports comparison, because it is the shared condition. 62.7% is the record, and it is a large one. The Provider Adapter number answers a different question that is also worth asking -- what the model can do when its reasoning state is preserved, which is considerably closer to how an agent actually runs in production than a fresh-context evaluation is.
Both numbers are true. They answer different questions. The trouble with 'GPT-6 hits 99.9%' is not that it is false. It is that it silently picks one of the two questions and never says which.
ARC-AGI-3 is an interactive benchmark from ARC Prize. Rather than posing fixed questions, it places a model in novel game environments it has never seen and requires it to infer the rules by acting and observing outcomes. The design targets generalisation into unfamiliar situations rather than recall of training data, which is why it is unusually sensitive to whether the model can retain state between turns.
DrafterDaily covers model evaluation, pricing and deployment mechanics -- the parts that determine what a capability claim is worth.
Read more AI coverageClaude Fable 5.1 costs exactly what Fable 5 cost per token. The 25-to-45% saving Anthropic advertises comes from one repriced line item — cached input, now billed at 2.5% of list instead of 10%. That is a discount you only collect if you keep the agent running.
Anthropic was not breached. The malware was already on the customer's machine, and it took a session cookie rather than a password — which is why two-factor authentication did nothing and why server-side revocation is the only lever the vendor has.
OpenAI's incident report on the Hugging Face breach leads on a security failure. The remarkable part is a verbatim chain-of-thought in which a model identifies its action as unauthorised and proceeds anyway — and an escape route that was a package manager, not a superintelligence.