GPT-6 Astra’s 99.9% ARC-AGI-3 Score Needs a Harness Label: 62.7% Standard, $19K Adapter Run

GPT-6 Astra has produced one of the most striking benchmark results of the September 2026 model cycle, but the headline number is easy to misread without the harness label. ARC Prize reports a best observed 99.9% score on ARC-AGI-3 Semi-Private for GPT-6 Astra at high reasoning when it is run through a Provider Adapter harness. Under ARC Prize’s provider-neutral Standard harness, the best observed Astra result is 62.7% at max reasoning.

Both results are real ARC Prize measurements. They answer different questions.

The Standard harness asks how a model performs through a common provider-neutral interface. It uses a visible text history and lets the model choose what discoveries to preserve in notes between environment actions. The Provider Adapter harness instead preserves provider-native conversation state, including opaque reasoning state, and uses native compaction to manage long interactions. ARC Prize’s open benchmarking repository says the two harnesses use the same games, actions, limits and scoring, but explicitly instructs researchers to report their results separately.

That distinction matters because ARC-AGI-3 is not a one-shot puzzle benchmark. It is an interactive test in which an agent explores unfamiliar abstract environments, infers rules and goals, builds a working model, then plans and executes actions. Context management is therefore part of the system behavior being measured.

The verified numbers: 62.7% Standard versus 99.9% Provider Adapter

ARC Prize’s verified Astra results page lists 12 harness configurations across reasoning settings. On ARC-AGI-3 Semi-Private:

• Standard harness, max reasoning: 62.71%, with reported evaluation cost of $26,098.
• Provider Adapter, high reasoning: 99.95%, with reported evaluation cost of $18,817.
• Provider Adapter, max reasoning: 98.55%, with reported evaluation cost of $17,332.
• Standard harness, high reasoning: 54.82%, with reported evaluation cost of $40,705.

This produces two practical lessons. First, the highest score is not the same configuration as the lowest-cost strong run. High reasoning produced the 99.9% Provider Adapter result, while max reasoning produced 98.6% for roughly $1,485 less in ARC Prize’s reported run. Second, more reasoning effort did not simply mean a higher bill. ARC Prize says higher reasoning settings often solved games in fewer environment actions, reducing the number of model calls and tokens enough to offset the heavier reasoning setting.

The 99.9% figure therefore should not be described as “Astra scores 99.9% on ARC-AGI-3” without stating the Provider Adapter condition. The 62.7% Standard result is equally important because it is the provider-neutral number intended for controlled cross-provider comparison.

Why the harness changes the result so dramatically

ARC-AGI-3 games can span many model calls. In the Standard harness, useful discoveries survive only if the model deliberately carries them forward in visible notes. In the Provider Adapter harness, the provider’s native system can preserve opaque reasoning state across requests and compact older context.

ARC Prize reports that, across the 167 game-and-reasoning-setting pairs solved by both harnesses in its matched analysis, Provider Adapter runs were about 3.66 times faster by aggregate recorded elapsed time and used 49% fewer total tokens.

This is not merely a latency optimization. Better continuity can change the agent’s ability to retain a discovered rule, avoid re-exploring a solved mechanic, and execute a multi-step plan. ARC Prize’s replay analysis says Astra often compressed environment mechanics into concise symbolic notes and, in a separate tool-enabled PRO-LONG setup, built game-specific parsers, state models, search procedures and planners.

The right interpretation is therefore “model plus harness,” not “model score independent of harness.” A production agent also includes context management, memory, tool routing and compaction, so Provider Adapter results can be useful for practical system evaluation. But a cross-model leaderboard should not compare a provider-native adapter result against another model measured only through the Standard harness and call that an apples-to-apples model ranking.

Human parity here means action efficiency on a bounded benchmark

ARC Prize reports that Astra at max reasoning with the Provider Adapter used fewer environment actions than the median tested human on 96.0% of completed levels and used 51.7% fewer actions per level on average.

ARC-AGI-3 scores systems using Relative Human Action Efficiency. The benchmark evaluates both completion and the number of environment-changing actions required relative to a human baseline. Internal reasoning steps and tool calls that do not alter the environment are not counted as environment actions.

ARC Prize’s human-testing work is deliberately first-exposure oriented: people are tested on unfamiliar environments, and the scoring baseline uses a median human action count at the level level rather than an unusually lucky best run. Its public-demo dataset contains 342 human step-by-step replays across 25 public environments, while the broader Astra analysis says approximately 500 members of the general public were used to establish the action-efficiency baseline used in the benchmark program.

So “human parity” in ARC Prize’s Astra write-up is specific. It means Astra matched or exceeded the benchmark’s human action-efficiency reference on most levels it completed under the stated harness. It does not mean the model matches humans across open-ended work, social judgment, robotics, long-term learning, or every form of intelligence.

ARC Prize itself says saturating ARC-AGI-3 is not evidence that a system is AGI. The environments are deterministic, closed-ended and intentionally abstract. That makes the benchmark useful for measuring exploration, model-building, goal discovery and planning while still leaving a large gap between benchmark competence and real-world generality.

Do not compare 99.9% directly with older 7.8% or 30.2% rows

This is the biggest reporting trap around Astra.

The older GPT-5.6 Sol ARC-AGI-3 result page reports 7.78% at max reasoning, and Claude Opus 5 was previously reported at 30.16% at high reasoning. Those headline rows were produced under the earlier provider-neutral evaluation conditions. Astra’s 99.9% figure comes from the Provider Adapter harness.

Astra’s Standard-harness result, 62.7%, is still a very large jump over those older Standard-style results, but the magnitude is much smaller than a direct 99.9-versus-7.8 comparison suggests. ARC Prize has now formalized the distinction by publishing Standard and Provider Adapter results as separate columns and explaining that they answer different questions.

Public discussion reflects this disagreement. A September 3 r/singularity thread argued that screenshots of the 99.9% number could mislead readers when the harness differences are omitted, and several September 4–5 Reddit discussions similarly focused on whether provider-native memory and compaction should count as part of the evaluated system. Other commenters argued the opposite: persistent state and compaction are normal features of real agents and therefore should be included when judging practical capability.

Neither side is a measured population survey. These are self-selected community reactions. The stronger conclusion comes from ARC Prize’s own methodology: report both harnesses, keep the labels visible, and avoid ranking unlike configurations as if they were identical experiments.

No sufficiently attributable, technically detailed X post about this exact Astra harness comparison was reliably retrievable during this verification pass, so this article does not claim an X consensus.

Cost is substantial, and the API price alone does not predict benchmark cost

OpenAI’s current API documentation lists GPT-6 Astra at $10 per million uncached input tokens and $50 per million output tokens for standard short-context pricing, with $1 per million cached-input tokens and $12.50 per million cache-write tokens. Requests above 272K input tokens are priced at 2x input/cache rates and 1.5x output for the full request. Batch and Flex are listed at half Standard rates, while Fast mode is listed at 2x the applicable rate.

The model page lists a 1,050,000-token context window and 128,000 maximum output tokens. OpenAI supports low, medium, high, xhigh and max reasoning for Astra.

ARC Prize’s reported $17K–$50K totals are evaluation-run costs across the benchmark, not per-user task prices. They depend on how many environment actions the agent takes, how much context is retained or compacted, reasoning effort, token volume and the harness. That is why the Provider Adapter can be both more capable and cheaper in the aggregate run: a better state-management strategy can reduce repeated exploration and token use.

For application teams, the more relevant metric is cost per successful task under a fixed workload. A high per-token price can still be economical if the model finishes a task with fewer retries, fewer tool calls and fewer repeated context tokens. Conversely, a spectacular benchmark score can be commercially unattractive if the workflow requires thousands of dollars of inference for a small number of successful tasks.

SWE-bench Verified and SWE-bench Pro remain separate—and neither should be inferred from ARC-AGI-3

ARC-AGI-3 is an interactive abstract-reasoning benchmark, not a software-repository patching benchmark. It should not be used to infer performance on SWE-bench Verified or SWE-bench Pro.

No exact GPT-6 Astra SWE-bench Verified result was found in the primary OpenAI launch material checked for this article. That omission is consistent with OpenAI’s February 2026 position that SWE-bench Verified has become increasingly contaminated for frontier-model comparison; OpenAI now recommends newer uncontaminated evaluations and specifically points developers toward SWE-bench Pro.

No exact primary-source GPT-6 Astra SWE-bench Pro score was located in the checked Astra launch and model documentation either. OpenAI instead reports other software-engineering and terminal evaluations for Astra, including Terminal-Bench 4.0, but those are different benchmarks with different task sets and harnesses. They should not be silently substituted for SWE-bench Pro.

That separation is important for any model comparison: “no verified score found” is better than importing a number from a different benchmark, model revision, agent scaffold or third-party leaderboard without a reproducible run manifest.

What the result does establish

Even after the harness correction, Astra’s ARC-AGI-3 result is important.

The provider-neutral 62.7% result represents a large advance on the same class of interactive environment where earlier frontier systems struggled. The Provider Adapter result then shows how much additional capability can emerge when a strong model is paired with native state preservation and compaction. ARC Prize’s matched analysis finding—49% fewer tokens and roughly 3.66x lower aggregate elapsed time on jointly solved game/reasoning pairs—suggests that agent infrastructure is becoming as important to practical outcomes as the raw foundation model.

For developers, this argues for benchmarking the complete system they actually deploy. Freeze the exact model ID, reasoning effort, context-management mode, tool set, retry policy, cost accounting and benchmark revision. Report Standard and provider-native harnesses in separate columns. Measure success rate, wall-clock time, total tokens, cost per success and failure modes together.

For readers, the safest summary is simple: GPT-6 Astra did not produce one ARC-AGI-3 number. ARC Prize verified a 62.7% best Standard-harness result and a 99.9% best Provider Adapter result. Both are meaningful, but they describe different evaluation conditions, and the harness label is part of the result.