The ARC Prize

In 2019, AI researcher François Chollet published a benchmark unlike any other: the Abstraction and Reasoning Corpus (ARC-AGI). Each task shows you two or three pairs of small colored grids — an "input" and its matching "output" — and then a new input grid with no output. Your job: figure out the hidden rule and draw the output yourself.

There is no vocabulary to learn, no world knowledge required, no language to translate. A curious ten-year-old can solve most of these puzzles in seconds by simply noticing the pattern: objects got bigger, colors got swapped, the shape got mirrored. And yet, for years, state-of-the-art AI systems — the same ones acing bar exams and coding interviews — scored close to zero.

That gap is the whole point. ARC-AGI was designed so that memorizing the internet doesn't help. Every puzzle is novel, and solving it requires acquiring a new skill on the spot from just a few examples — which is precisely how Chollet chose to define intelligence in the first place.

Below are training examples: small colored grids that turn into other grids by some consistent rule. Study them, then apply what you learned to the test grid.

<p class="hint">{{hint_para}}</p>
<div class="examples" id="examples"></div>
<div class="test-wrap">
  <div class="test-col">
    <div class="label">{{test_input_label}}</div>
    <div class="grid" id="test-input"></div>
  </div>
  <div class="arrow">&rarr;</div>
  <div class="test-col">
    <div class="label">{{your_answer_label}}</div>
    <div class="grid" id="test-answer"></div>
  </div>
</div>
<div class="status" id="status">{{click_hint}}</div>
<div class="btns">
  <button id="check" type="button">{{btn_check}}</button>
  <button id="reveal" type="button">{{btn_reveal}}</button>
  <button id="reset" type="button" class="ghost">{{btn_clear}}</button>
</div>

* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; }
.hint { font-size: .9rem; color: #444; margin: 0 0 .7rem; line-height: 1.45; }
.examples { display: flex; gap: 1.2rem; flex-wrap: wrap; margin-bottom: .9rem; }
.example { display: flex; align-items: center; gap: .5rem; }
.label { font-size: .75rem; font-weight: 700; color: #555; text-transform: uppercase;
          letter-spacing: .03em; margin-bottom: .3rem; }
.grid { display: grid; grid-template-columns: repeat(3, 28px); gap: 2px;
        background: #cdd9e3; padding: 2px; border-radius: 6px; }
.grid.big { grid-template-columns: repeat(3, 36px); }
.cell { width: 28px; height: 28px; border-radius: 3px; background: #eef1f4; cursor: default; }
.grid.big .cell { width: 36px; height: 36px; cursor: pointer; transition: transform .08s; }
.grid.big .cell:hover { transform: scale(1.06); }
.arrow { font-size: 1.3rem; color: #1d3557; margin: 0 .4rem; align-self: center; }
.test-wrap { display: flex; align-items: flex-end; gap: .6rem; flex-wrap: wrap; margin-bottom: .6rem; }
.test-col { display: flex; flex-direction: column; }
.status { font-size: 1rem; font-weight: 600; margin: .5rem 0; min-height: 1.4em; }
.status.ok { color: #0a7d33; }
.status.bad { color: #c92f3c; }
.btns { display: flex; gap: .5rem; flex-wrap: wrap; }
button { font: 600 14px system-ui, sans-serif; padding: .45rem .9rem; border: 1px solid #1d3557;
         background: #1d3557; color: #fff; border-radius: 8px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }

// Code not found

Notice what you just did: nobody told you the rule in words. You looked at two or three examples and induced it — the same jump from "a few instances" to "a general procedure" that every ARC-AGI task demands, and that current AI systems still make unreliably.

Why is a puzzle a child can solve so hard for machines? Because ARC-AGI does not measure skill — it measures the efficiency of acquiring a new skill.

• It's not pattern lookup. Large models trained on huge datasets are extremely good at recognizing patterns they've already seen. ARC-AGI's puzzles (and its private evaluation set) are generated to be unlike anything in common training data, so memorized shortcuts fail.
• Each task is a tiny "programming" problem. Solving one is close to a miniature instance of program synthesis: search over a space of simple grid-transformation programs (reflect, recolor, extend, count objects, complete a symmetry) until one program fits every training pair, then run it on the test input.
• Humans use strong priors. Chollet argues people succeed because we come pre-loaded with "core knowledge" — intuitive physics, objectness, counting, symmetry — and can compose it on the fly. A system without those priors must instead search a much larger space of possible programs, closer to brute enumeration.
• Status: ongoing, unsolved at full efficiency. The ARC Prize 2025 competition offered US $1,000,000 in total prizes, including a $700,000 Grand Prize for any open-sourced approach that reached 85% accuracy on the private evaluation set within a fixed compute budget — a bar meant to approximate typical human performance. As of the prize's own published results, that Grand Prize remains unclaimed; top scores have climbed steeply but efficient, general solutions are still an open problem.

So the "real" complexity is not raw difficulty per puzzle — it's what researchers call sample efficiency: how much new capability a system can build from how little data, which is a very different question from "can a big enough model eventually get it right."

ARC-AGI is a toy grid on the surface, but the capability it isolates — learning a new rule from almost no examples — is the crux of several serious fields:

• Benchmarking progress toward AGI: unlike exams that can be studied for, ARC-AGI's private test set is refreshed precisely to catch overfitting, making it one of the few benchmarks the field treats as a real signal rather than a leaderboard to game.
• Program synthesis and automated reasoning: many top approaches literally search program space — the same machinery behind synthesizing spreadsheet formulas from examples or inferring code from input/output pairs.
• Few-shot and continual learning: robots, assistants, and scientific-discovery agents constantly meet situations with no labeled dataset; techniques that generalize from a handful of demonstrations there are stress-tested here first.
• Human-like abstraction as a design target: Chollet's "core knowledge priors" framing has shaped how researchers think about building in objectness, physics, and symmetry rather than only scaling data.

Whenever you hear a claim that a model has reached "general" intelligence, ARC-AGI is one of the sharpest tools for checking whether that generality survives contact with a puzzle nobody wrote about on the internet.

The ARC Prize turns a deceptively simple idea into one of the sharpest open questions in AI: can a system look at a couple of examples and genuinely figure out something new, the way a child does? Grids of colored squares make the question concrete, checkable, and resistant to memorization.

Billions of dollars have gone into scaling today's models, and scores on ARC-AGI have risen a great deal — yet the $700,000 Grand Prize for an efficient, general solution sits unclaimed. Until it isn't, the gap between recognizing and reasoning — the same gap probed by questions like P vs NP — is still very much open.