ARC-AGI-3: learning a game without instructions
ARC-AGI-3 tests how efficiently agents learn unfamiliar games. How its scoring works, what the March 2026 paper reported, and why preview scores differ.
On this page
Open a game without a tutorial. A few colored objects appear, some buttons do nothing, and you do not yet know what counts as winning. Your first job is to work out what changes when you act.
That is the problem ARC-AGI-3 gives an AI agent. The ARC Prize Foundation's March 2026 release replaced the series' static input-output puzzles with interactive, turn-based environments. An agent must discover the rules, infer the goal, and apply what it learns across levels.
The benchmark measures how many actions that takes relative to humans. Finishing eventually is not enough for a high score.
TL;DR ARC-AGI-3 tests learning through interaction. The March 2026 paper reported scores below 1% for its evaluated frontier models. An earlier preview competition used a different set of games, so its 12.58% winning result cannot be compared directly with those launch scores. Later Astra results also show why an evaluation's model and harness must stay attached to its score.
September 2026 update: The figures below are historical results from version 1 of the March paper. See the Astra analysis for later evaluations with substantially higher scores under different configurations.
What changes when a puzzle becomes interactive
ARC-AGI-1 presents a few examples of a transformation between colored grids. The solver infers the rule and applies it to another grid. ARC-AGI-2 keeps that format while requiring more complex combinations of rules.
In ARC-AGI-3, the solver chooses actions and receives new frames. The rules are not supplied as prose, and the goal must be discovered. These are turn-based environments: the state changes when the agent acts, so the task does not require fast reflexes.
Imagine that moving one object opens a passage. A useful next action tests whether the passage stays open, whether the object must remain in place, or whether another object behaves the same way. Repeating unrelated clicks provides less information.
The technical paper groups these demands into exploration, planning, memory, and goal acquisition. General knowledge can still help an agent interpret space and objects. What it cannot assume is that it has already seen the exact task or knows its solution.
Why taking more actions lowers the score
ARC-AGI-3 uses Relative Human Action Efficiency. For each completed level, it compares the human baseline's action count with the agent's count, squares the ratio, and caps the result at 100%.
For example, suppose the human baseline needs 10 actions:
| Agent actions | Credit for that completed level |
|---|---|
| 10 | 100% |
| 20 | 25% |
| 100 | 1% |
These are illustrative calculations, not model results. The benchmark then weights level scores within an environment and averages across environments. Failure to complete levels also lowers the final result.
The paper sets the baseline using the second-best action count from ten human participants per environment. That is a strong reference point, not the average person's first attempt. Its official evaluation also imposes action limits, so an agent cannot always keep exploring until it succeeds.
This is why a score below 1% should not be translated into “solved fewer than 1% of games.” Completion and efficiency both contribute. Conversely, completing a game through a long sequence of guesses may earn little credit.
What the March paper actually reported
Table 2 of the paper's first version gives these Semi-Private results at release:
| Model configuration | Score |
|---|---|
| Gemini 3.1 Pro Preview | 0.37% |
| GPT-5.4 (High) | 0.26% |
| Opus 4.6 (Max) | 0.25% |
| Grok-4.20 (Beta 0309 Reasoning) | 0.00% |
The narrow conclusion is that these evaluated systems struggled to learn and complete unfamiliar environments efficiently. The table does not isolate whether a failure came from visual interpretation, exploration, memory, planning, or the application code around the model.
That surrounding code is the harness. It determines how frames reach the model, which actions are available, what history is preserved, and when a run stops. Changing it can change the result. The later Astra evaluations make that dependence visible by reporting separate Standard and Provider Adapter configurations.
What the earlier preview taught the designers
Before the full release, ARC Prize ran a competition on three public preview games. Its 30-day report names StochasticGoose as the winner at 12.58%, with 18 levels completed.
StochasticGoose used a convolutional neural network to learn which actions were likely to change the frame. This helped it avoid spending as many moves on ineffective actions. Another entrant, Blind Squirrel, built a state graph and used a learned value model; it scored 6.71%.
Both entrants combined learned components with their exploration strategies. They also ran on a different game set from the March benchmark. Dividing 12.58% by a launch-model score does not produce a meaningful performance advantage.
The preview exposed weaknesses in the benchmark too. Some games were too easy to brute-force, and anonymous human play made first-attempt baselines difficult to establish. The Foundation used that feedback to refine the games, scoring, and controlled human evaluation for the full release.
What to inspect in an agent's run
An interactive benchmark becomes more useful when you look past its final number. Does the agent notice that an action had no effect? Does it record a discovered rule? When a strategy fails, does it revise the rule or repeat the same move?
Those questions also apply to a tool-using agent navigating a website or debugging a repository. The environment is different, but successful multi-step work still depends on using the previous result to choose the next action.
ARC-AGI-3 provides controlled environments for studying that behavior. Its launch scores establish a historical baseline. The useful ongoing comparison is between complete, documented runs: the same task set, known action limits, and a clear account of what the model and its harness retained between steps.
Sources
- ARC Prize Foundation: ARC-AGI-3 launch
- ARC Prize Foundation: ARC-AGI-3 technical paper, version 1
- ARC Prize Foundation: Preview competition, 30-day learnings
- ARC Prize Foundation: Astra evaluation
Common questions
What is ARC-AGI-3?
ARC-AGI-3 is an interactive benchmark from the ARC Prize Foundation. Agents encounter unfamiliar turn-based games and must infer their mechanics and goals from actions and feedback, without task-specific instructions.
How does ARC-AGI-3 score an agent?
It compares the actions an agent needs to complete each level with a human baseline, then squares the efficiency ratio and caps credit at 100%. A completed level taking ten times the baseline actions earns 1% before the scores are aggregated across levels and environments.
What did the ARC-AGI-3 launch paper report?
Table 2 of the March 2026 paper's first version reports 0.37% for Gemini 3.1 Pro Preview, 0.26% for GPT-5.4 High, 0.25% for Opus 4.6 Max, and 0.00% for Grok-4.20 Beta. These are historical Semi-Private results under the paper's evaluation setup, not current model rankings.
What was the 12.58% ARC-AGI-3 preview result?
StochasticGoose won the earlier three-game preview competition with 12.58%, using a convolutional neural network to learn which actions would change a frame. That preview result is not directly comparable with the March 2026 launch benchmark and was not a plain graph-search result.
Does solving ARC-AGI-3 prove AGI?
A strong score shows efficient performance on the benchmark's unfamiliar interactive environments. It does not establish the same ability across open-ended real-world tasks, and the model, harness, dataset, and evaluation conditions remain important.