Introduction

In 2019, AI researcher François Chollet published a benchmark unlike any other: the Abstraction and Reasoning Corpus (ARC-AGI). Each task shows you two or three pairs of small colored grids — an "input" and its matching "output" — and then a new input grid with no output. Your job: figure out the hidden rule and draw the output yourself.

There is no vocabulary to learn, no world knowledge required, no language to translate. A curious ten-year-old can solve most of these puzzles in seconds by simply noticing the pattern: objects got bigger, colors got swapped, the shape got mirrored. And yet, for years, state-of-the-art AI systems — the same ones acing bar exams and coding interviews — scored close to zero.

That gap is the whole point. ARC-AGI was designed so that memorizing the internet doesn't help. Every puzzle is novel, and solving it requires acquiring a new skill on the spot from just a few examples — which is precisely how Chollet chose to define intelligence in the first place.

Try It

Below are training examples: small colored grids that turn into other grids by some consistent rule. Study them, then apply what you learned to the test grid.

<p class="hint">Each example turns an input grid into an output grid by the <b>same</b> hidden rule. Study both examples, click cells on the answer grid to cycle their color, and reproduce the rule on the test grid.</p>
<div class="examples" id="examples"></div>
<div class="test-wrap">
  <div class="test-col">
    <div class="label">Test input</div>
    <div class="grid" id="test-input"></div>
  </div>
  <div class="arrow">&rarr;</div>
  <div class="test-col">
    <div class="label">Your answer (click cells)</div>
    <div class="grid" id="test-answer"></div>
  </div>
</div>
<div class="status" id="status">Click the cells to change their color, then check.</div>
<div class="btns">
  <button id="check" type="button">Check answer</button>
  <button id="reveal" type="button">Reveal solution</button>
  <button id="reset" type="button" class="ghost">Reset</button>
</div>

* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; }
.hint { font-size: .9rem; color: #444; margin: 0 0 .7rem; line-height: 1.45; }
.examples { display: flex; gap: 1.2rem; flex-wrap: wrap; margin-bottom: .9rem; }
.example { display: flex; align-items: center; gap: .5rem; }
.label { font-size: .75rem; font-weight: 700; color: #555; text-transform: uppercase;
          letter-spacing: .03em; margin-bottom: .3rem; }
.grid { display: grid; grid-template-columns: repeat(3, 28px); gap: 2px;
        background: #cdd9e3; padding: 2px; border-radius: 6px; }
.grid.big { grid-template-columns: repeat(3, 36px); }
.cell { width: 28px; height: 28px; border-radius: 3px; background: #eef1f4; cursor: default; }
.grid.big .cell { width: 36px; height: 36px; cursor: pointer; transition: transform .08s; }
.grid.big .cell:hover { transform: scale(1.06); }
.arrow { font-size: 1.3rem; color: #1d3557; margin: 0 .4rem; align-self: center; }
.test-wrap { display: flex; align-items: flex-end; gap: .6rem; flex-wrap: wrap; margin-bottom: .6rem; }
.test-col { display: flex; flex-direction: column; }
.status { font-size: 1rem; font-weight: 600; margin: .5rem 0; min-height: 1.4em; }
.status.ok { color: #0a7d33; }
.status.bad { color: #c92f3c; }
.btns { display: flex; gap: .5rem; flex-wrap: wrap; }
button { font: 600 14px system-ui, sans-serif; padding: .45rem .9rem; border: 1px solid #1d3557;
         background: #1d3557; color: #fff; border-radius: 8px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }

const fmt = (s, v) => s.replace(/%(\w+)%/g, (_, k) => String(v[k] ?? ''));
// A miniature ARC-AGI-style task: infer a grid transformation rule from a couple of
const N = 3;
// 0 = background (light gray); 1..4 = distinct colors, cycled by clicking a cell.
const PALETTE = ['#eef1f4', '#e63946', '#2a9d8f', '#e9c46a', '#3d5a80'];

// Two training pairs (input grid -> output grid) that share one common rule.
const EX1_IN  = [[0,0,1],[0,1,0],[1,0,0]];
const EX1_OUT = [[1,0,0],[0,1,0],[0,0,1]];
const EX2_IN  = [[2,0,0],[0,0,0],[0,0,3]];
const EX2_OUT = [[0,0,2],[0,0,0],[3,0,0]];

// The test input the player must transform by hand, applying the inferred rule.
const TEST_IN  = [[0,4,0],[4,0,4],[0,0,4]];
const SOLUTION = [[0,4,0],[4,0,4],[4,0,0]]; // the rule is a left-right mirror of the grid; SOLUTION is precomputed for checking

// The player's current answer grid, mutated by clicking cells.
let answer = TEST_IN.map(row => row.slice());

function makeGrid(container, grid, opts){
  container.innerHTML = '';
  container.classList.toggle('big', !!(opts && opts.big));
  grid.forEach((row, r) => row.forEach((v, c) => {
    const d = document.createElement('div');
    d.className = 'cell';
    d.style.background = PALETTE[v] || PALETTE[0];
    if (opts && opts.clickable) {
      d.addEventListener('click', () => {
        answer[r][c] = (answer[r][c] + 1) % PALETTE.length;
        renderAnswer();
        setStatus('Click the cells to change their color, then check.', '');
      });
    }
    container.appendChild(d);
  }));
}

function renderExamples(){
  const wrap = document.getElementById('examples');
  wrap.innerHTML = '';
  [[EX1_IN, EX1_OUT], [EX2_IN, EX2_OUT]].forEach(([inG, outG], i) => {
    const ex = document.createElement('div');
    ex.className = 'example';
    const colIn = document.createElement('div');
    const labIn = document.createElement('div');
    labIn.className = 'label';
    labIn.textContent = fmt('Example %n%', { n: i + 1 });
    const gIn = document.createElement('div');
    gIn.className = 'grid';
    colIn.appendChild(labIn); colIn.appendChild(gIn);
    makeGrid(gIn, inG, {});
    const arrow = document.createElement('div');
    arrow.className = 'arrow';
    arrow.textContent = '→';
    const colOut = document.createElement('div');
    const labOut = document.createElement('div');
    labOut.className = 'label';
    labOut.textContent = ' ';
    const gOut = document.createElement('div');
    gOut.className = 'grid';
    colOut.appendChild(labOut); colOut.appendChild(gOut);
    makeGrid(gOut, outG, {});
    ex.appendChild(colIn); ex.appendChild(arrow); ex.appendChild(colOut);
    wrap.appendChild(ex);
  });
}

function renderTestInput(){
  makeGrid(document.getElementById('test-input'), TEST_IN, { big: true });
}
function renderAnswer(){
  makeGrid(document.getElementById('test-answer'), answer, { big: true, clickable: true });
}
function setStatus(t, cls){
  const s = document.getElementById('status');
  s.textContent = t;
  s.className = 'status ' + cls;
}

// The answer is correct iff it matches SOLUTION cell for cell.
function isCorrect(){
  for (let r = 0; r < N; r++) for (let c = 0; c < N; c++)
    if (answer[r][c] !== SOLUTION[r][c]) return false;
  return true;
}

document.getElementById('check').addEventListener('click', () => {
  if (isCorrect()) setStatus('✓ Correct — you inferred the rule from just two examples!', 'ok');
  else setStatus('✗ Not quite. Compare inputs and outputs again — what moved, and where?', 'bad');
});
document.getElementById('reveal').addEventListener('click', () => {
  answer = SOLUTION.map(row => row.slice());
  renderAnswer();
  setStatus('This is the correct output for the hidden rule.', 'ok');
  console.log('Hidden rule:', 'mirror the grid left-to-right');
});
document.getElementById('reset').addEventListener('click', () => {
  answer = TEST_IN.map(row => row.slice());
  renderAnswer();
  setStatus('Click the cells to change their color, then check.', '');
});

renderExamples();
renderTestInput();
renderAnswer();
Result
Console

Notice what you just did: nobody told you the rule in words. You looked at two or three examples and induced it — the same jump from "a few instances" to "a general procedure" that every ARC-AGI task demands, and that current AI systems still make unreliably.

The Real Complexity

Why is a puzzle a child can solve so hard for machines? Because ARC-AGI does not measure skill — it measures the efficiency of acquiring a new skill.

  • It's not pattern lookup. Large models trained on huge datasets are extremely good at recognizing patterns they've already seen. ARC-AGI's puzzles (and its private evaluation set) are generated to be unlike anything in common training data, so memorized shortcuts fail.
  • Each task is a tiny "programming" problem. Solving one is close to a miniature instance of program synthesis: search over a space of simple grid-transformation programs (reflect, recolor, extend, count objects, complete a symmetry) until one program fits every training pair, then run it on the test input.
  • Humans use strong priors. Chollet argues people succeed because we come pre-loaded with "core knowledge" — intuitive physics, objectness, counting, symmetry — and can compose it on the fly. A system without those priors must instead search a much larger space of possible programs, closer to brute enumeration.
  • Status: ongoing, unsolved at full efficiency. The ARC Prize 2025 competition offered US $1,000,000 in total prizes, including a $700,000 Grand Prize for any open-sourced approach that reached 85% accuracy on the private evaluation set within a fixed compute budget — a bar meant to approximate typical human performance. As of the prize's own published results, that Grand Prize remains unclaimed; top scores have climbed steeply but efficient, general solutions are still an open problem.

So the "real" complexity is not raw difficulty per puzzle — it's what researchers call sample efficiency: how much new capability a system can build from how little data, which is a very different question from "can a big enough model eventually get it right."

Where It Matters

ARC-AGI is a toy grid on the surface, but the capability it isolates — learning a new rule from almost no examples — is the crux of several serious fields:

  • Benchmarking progress toward AGI: unlike exams that can be studied for, ARC-AGI's private test set is refreshed precisely to catch overfitting, making it one of the few benchmarks the field treats as a real signal rather than a leaderboard to game.
  • Program synthesis and automated reasoning: many top approaches literally search program space — the same machinery behind synthesizing spreadsheet formulas from examples or inferring code from input/output pairs.
  • Few-shot and continual learning: robots, assistants, and scientific-discovery agents constantly meet situations with no labeled dataset; techniques that generalize from a handful of demonstrations there are stress-tested here first.
  • Human-like abstraction as a design target: Chollet's "core knowledge priors" framing has shaped how researchers think about building in objectness, physics, and symmetry rather than only scaling data.

Whenever you hear a claim that a model has reached "general" intelligence, ARC-AGI is one of the sharpest tools for checking whether that generality survives contact with a puzzle nobody wrote about on the internet.

Conclusion

The ARC Prize turns a deceptively simple idea into one of the sharpest open questions in AI: can a system look at a couple of examples and genuinely figure out something new, the way a child does? Grids of colored squares make the question concrete, checkable, and resistant to memorization.

Billions of dollars have gone into scaling today's models, and scores on ARC-AGI have risen a great deal — yet the $700,000 Grand Prize for an efficient, general solution sits unclaimed. Until it isn't, the gap between recognizing and reasoning — the same gap probed by questions like P vs NP — is still very much open.

Share this article

Pick a channel — or use your device's native share sheet.

Comments

Loading comments...

https://www.kipuhub.com/en/article/arc-prize-agi/Content licensed under CC BY-NC 4.0.