Introduction

When you ask a language model a hard arithmetic question — say, "If a store sells 17 apples at $0.85 each and takes a 12% discount, what is the total?" — one thing separates a reliable answer from a wrong one: whether the model shows its work.

The idea is called chain-of-thought (CoT) prompting, and it is one of the most striking discoveries in modern AI. Popularized by a 2022 Google Brain paper by Jason Wei and colleagues, it showed that simply asking a model to reason step-by-step — or including a few worked examples in the prompt — dramatically boosts accuracy on problems involving arithmetic, logic, and symbolic reasoning.

The intuition is old and human: scratch paper exists because our working memory is finite. A language model's "working memory" is the context window, and intermediate tokens are the scratch paper. Each reasoning step produces a token that the next step can attend to — the model is literally computing by writing.

Try It: One-Shot vs Step-by-Step

The demo below simulates how a language model approaches a multi-step arithmetic problem in two modes: One-Shot (blurt the answer directly) and Step-by-Step (chain-of-thought). Click a mode, then hit Run to see each approach unfold.

<!-- Chain-of-thought demo: one-shot vs step-by-step arithmetic solving -->
<div class="cot-app">
  <p class="hint">Choose a mode — One-Shot (answer directly) or Step-by-Step (chain-of-thought) — then hit Run. Watch how intermediate steps change what the model gets right.</p>

  <div class="problem-box">
    <label class="problem-label">Problem</label>
    <div class="problem-text" id="problem-text"></div>
  </div>

  <div class="mode-btns">
    <button id="btn-oneshot" type="button" class="mode-btn active" data-mode="oneshot">One-Shot</button>
    <button id="btn-cot" type="button" class="mode-btn" data-mode="cot">Step-by-Step</button>
  </div>

  <div class="action-row">
    <button id="btn-run" type="button" class="run-btn">Run</button>
    <button id="btn-new" type="button" class="ghost-btn">New Problem</button>
  </div>

  <div class="output-box" id="output-box">
    <div class="output-placeholder" id="output-placeholder">Choose a mode and press Run…</div>
    <div id="steps-container" class="steps-container" style="display:none"></div>
  </div>

  <div class="result-row" id="result-row" style="display:none">
    <span class="result-label">Answer:</span>
    <span class="result-value" id="result-value"></span>
    <span class="result-check" id="result-check"></span>
  </div>
</div>

/* Styles for the chain-of-thought demo */
* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; margin: 0; color: #222; }

.cot-app { max-width: 560px; margin: 0 auto; padding: .5rem 0; }

.hint { font-size: .88rem; color: #555; margin: 0 0 .9rem; line-height: 1.5; }

.problem-box { background: #f0f4f8; border: 1px solid #d1dce8; border-radius: 10px;
               padding: .7rem 1rem; margin-bottom: .85rem; }
.problem-label { font-size: .75rem; font-weight: 700; text-transform: uppercase;
                 letter-spacing: .04em; color: #6b8296; display: block; margin-bottom: .3rem; }
.problem-text { font-size: .97rem; color: #1d3557; line-height: 1.5; font-weight: 500; }

.mode-btns { display: flex; gap: .5rem; margin-bottom: .75rem; }
.mode-btn { flex: 1; padding: .45rem .6rem; font: 600 .88rem system-ui, sans-serif;
            border: 2px solid #1d3557; border-radius: 8px; cursor: pointer;
            background: #fff; color: #1d3557; transition: all .15s; }
.mode-btn.active { background: #1d3557; color: #fff; }

.action-row { display: flex; gap: .5rem; margin-bottom: .85rem; }
.run-btn { flex: 2; font: 700 .95rem system-ui, sans-serif; padding: .5rem 1rem;
           background: #2a7d4f; color: #fff; border: none; border-radius: 8px; cursor: pointer; }
.run-btn:hover { background: #235f3c; }
.ghost-btn { flex: 1; font: 600 .88rem system-ui, sans-serif; padding: .5rem .8rem;
             background: #fff; color: #1d3557; border: 1px solid #1d3557; border-radius: 8px; cursor: pointer; }

.output-box { min-height: 140px; background: #fafbfc; border: 1px solid #dce3ea;
              border-radius: 10px; padding: .75rem 1rem; margin-bottom: .7rem; position: relative; }
.output-placeholder { color: #aab; font-size: .9rem; font-style: italic; }

.steps-container { display: flex; flex-direction: column; gap: .5rem; }
.step { display: flex; gap: .6rem; align-items: flex-start; opacity: 0;
        animation: fadeIn .3s forwards; }
.step-num { min-width: 22px; height: 22px; border-radius: 50%; background: #1d3557;
            color: #fff; font: 700 .75rem system-ui; display: flex; align-items: center;
            justify-content: center; margin-top: .1rem; flex-shrink: 0; }
.step-text { font-size: .9rem; line-height: 1.5; color: #1d3557; }
.step.direct .step-num { background: #888; }
.step.direct .step-text { color: #444; }

.result-row { display: flex; align-items: center; gap: .6rem; padding: .5rem .8rem;
              border-radius: 8px; background: #eef6f0; border: 1px solid #b8dcc5; }
.result-label { font-size: .85rem; color: #555; }
.result-value { font: 700 1rem system-ui; color: #1d3557; }
.result-check { font-size: 1rem; }
.result-row.wrong { background: #fdf0f0; border-color: #e8b8b8; }

@keyframes fadeIn { to { opacity: 1; } }

// Chain-of-thought demo logic
// One-shot mode deliberately introduces errors to show the failure mode; CoT always follows correct steps.

const PROBLEMS = [
  { en: '17 × 6 + 45', es: '17 × 6 + 45', answer: 147 },
  { en: '(8 + 14) × 3 − 19', es: '(8 + 14) × 3 − 19', answer: 47 },
  { en: '120 ÷ 4 + 7 × 5', es: '120 ÷ 4 + 7 × 5', answer: 65 },
  { en: '(25 − 7) × (3 + 4)', es: '(25 − 7) × (3 + 4)', answer: 126 },
  { en: '3² + 4² − 10', es: '3² + 4² − 10', answer: 15 },
  { en: '(100 − 37) × 2 + 14', es: '(100 − 37) × 2 + 14', answer: 140 }
];

// One-shot mode: only 2 fake steps (no real sub-calculations)
const ONE_SHOT_STEPS = {
  en: [
    (p) => '→ ' + p.en,
    (p) => '= ' + (p.answer + Math.floor((Math.random() - 0.4) * 20))
  ],
  es: [
    (p) => '→ ' + p.es,
    (p) => '= ' + (p.answer + Math.floor((Math.random() - 0.4) * 20))
  ]
};

// CoT mode: real intermediate steps per problem
const COT_STEPS = {
  '17 × 6 + 45': {
    en: ['Step 1: Multiply first  →  17 × 6 = 102', 'Step 2: Add the constant  →  102 + 45 = 147', 'Answer: 147'],
    es: ['Paso 1: Primero la multiplicación  →  17 × 6 = 102', 'Paso 2: Sumar la constante  →  102 + 45 = 147', 'Respuesta: 147']
  },
  '(8 + 14) × 3 − 19': {
    en: ['Step 1: Parentheses  →  8 + 14 = 22', 'Step 2: Multiply  →  22 × 3 = 66', 'Step 3: Subtract  →  66 − 19 = 47', 'Answer: 47'],
    es: ['Paso 1: Paréntesis  →  8 + 14 = 22', 'Paso 2: Multiplicar  →  22 × 3 = 66', 'Paso 3: Restar  →  66 − 19 = 47', 'Respuesta: 47']
  },
  '120 ÷ 4 + 7 × 5': {
    en: ['Step 1: Division first  →  120 ÷ 4 = 30', 'Step 2: Multiplication  →  7 × 5 = 35', 'Step 3: Add results  →  30 + 35 = 65', 'Answer: 65'],
    es: ['Paso 1: División primero  →  120 ÷ 4 = 30', 'Paso 2: Multiplicación  →  7 × 5 = 35', 'Paso 3: Sumar resultados  →  30 + 35 = 65', 'Respuesta: 65']
  },
  '(25 − 7) × (3 + 4)': {
    en: ['Step 1: First parenthesis  →  25 − 7 = 18', 'Step 2: Second parenthesis  →  3 + 4 = 7', 'Step 3: Multiply  →  18 × 7 = 126', 'Answer: 126'],
    es: ['Paso 1: Primer paréntesis  →  25 − 7 = 18', 'Paso 2: Segundo paréntesis  →  3 + 4 = 7', 'Paso 3: Multiplicar  →  18 × 7 = 126', 'Respuesta: 126']
  },
  '3² + 4² − 10': {
    en: ['Step 1: Square 3  →  3² = 9', 'Step 2: Square 4  →  4² = 16', 'Step 3: Add, then subtract  →  9 + 16 − 10 = 15', 'Answer: 15'],
    es: ['Paso 1: Cuadrado de 3  →  3² = 9', 'Paso 2: Cuadrado de 4  →  4² = 16', 'Paso 3: Sumar y restar  →  9 + 16 − 10 = 15', 'Respuesta: 15']
  },
  '(100 − 37) × 2 + 14': {
    en: ['Step 1: Parenthesis  →  100 − 37 = 63', 'Step 2: Multiply  →  63 × 2 = 126', 'Step 3: Add  →  126 + 14 = 140', 'Answer: 140'],
    es: ['Paso 1: Paréntesis  →  100 − 37 = 63', 'Paso 2: Multiplicar  →  63 × 2 = 126', 'Paso 3: Sumar  →  126 + 14 = 140', 'Respuesta: 140']
  }
};

// Detect page language to pick the right step strings
const lang = (document.documentElement.lang || 'en').startsWith('es') ? 'es' : 'en';

let currentProblem = PROBLEMS[Math.floor(Math.random() * PROBLEMS.length)];
let currentMode = 'oneshot';
let running = false;

const problemEl = document.getElementById('problem-text');
const outputBox = document.getElementById('output-box');
const placeholder = document.getElementById('output-placeholder');
const stepsContainer = document.getElementById('steps-container');
const resultRow = document.getElementById('result-row');
const resultValue = document.getElementById('result-value');
const resultCheck = document.getElementById('result-check');

function showProblem() {
  problemEl.textContent = lang === 'es' ? currentProblem.es : currentProblem.en;
}

function resetOutput() {
  placeholder.style.display = 'block';
  stepsContainer.style.display = 'none';
  stepsContainer.innerHTML = '';
  resultRow.style.display = 'none';
  resultRow.className = 'result-row';
}

function addStep(text, isDirect, delay) {
  return new Promise(resolve => {
    setTimeout(() => {
      const div = document.createElement('div');
      div.className = 'step' + (isDirect ? ' direct' : '');
      const num = document.createElement('div');
      num.className = 'step-num';
      num.textContent = isDirect ? '?' : String(stepsContainer.children.length + 1);
      const txt = document.createElement('div');
      txt.className = 'step-text';
      txt.textContent = text;
      div.appendChild(num);
      div.appendChild(txt);
      stepsContainer.appendChild(div);
      resolve();
    }, delay);
  });
}

// One-shot runner: skips real sub-steps, may produce wrong answer
async function runOneShot() {
  placeholder.style.display = 'none';
  stepsContainer.style.display = 'flex';
  const pText = lang === 'es' ? currentProblem.es : currentProblem.en;
  // Label making clear no work is shown
  const intro = lang === 'es' ? '(no intermediate steps — direct output)' : '(no intermediate steps — direct output)';
  await addStep(intro, true, 0);
  await addStep('→ ' + pText, true, 400);
  // Deliberately off by a random amount to show the failure mode
  const wrongOffset = [3, -5, 7, -8, 11, -4, 6, -9][Math.floor(Math.random() * 8)];
  const guessed = currentProblem.answer + wrongOffset;
  await addStep('= ' + guessed, true, 900);
  const isCorrect = guessed === currentProblem.answer;
  showResult(guessed, isCorrect, 1400);
}

// CoT runner: replays pre-computed correct intermediate steps with a reveal delay
async function runCoT() {
  placeholder.style.display = 'none';
  stepsContainer.style.display = 'flex';
  const key = lang === 'es' ? currentProblem.es : currentProblem.en;
  const cotSteps = (COT_STEPS[key] || COT_STEPS[currentProblem.en]);
  const steps = cotSteps ? cotSteps[lang] : cotSteps.en;
  for (let i = 0; i < steps.length - 1; i++) {
    await addStep(steps[i], false, i * 500);
  }
  // Final answer step — always correct in CoT mode
  const lastStep = steps[steps.length - 1];
  const delay = (steps.length - 1) * 500;
  await addStep(lastStep, false, delay);
  showResult(currentProblem.answer, true, delay + 500);
}

function showResult(value, isCorrect, delay) {
  setTimeout(() => {
    resultRow.style.display = 'flex';
    resultRow.className = 'result-row' + (isCorrect ? '' : ' wrong');
    resultValue.textContent = value;
    resultCheck.textContent = isCorrect ? '✓ Correct' : '✗ Wrong';
    running = false;
  }, delay);
}

document.getElementById('btn-run').addEventListener('click', async () => {
  if (running) return;
  running = true;
  resetOutput();
  if (currentMode === 'oneshot') {
    await runOneShot();
  } else {
    await runCoT();
  }
});

document.getElementById('btn-new').addEventListener('click', () => {
  const idx = Math.floor(Math.random() * PROBLEMS.length);
  currentProblem = PROBLEMS[idx];
  showProblem();
  resetOutput();
  running = false;
});

document.querySelectorAll('.mode-btn').forEach(btn => {
  btn.addEventListener('click', () => {
    currentMode = btn.dataset.mode;
    document.querySelectorAll('.mode-btn').forEach(b => b.classList.remove('active'));
    btn.classList.add('active');
    resetOutput();
    running = false;
  });
});

showProblem();
Result
Console

Notice the pattern: one-shot sometimes gets lucky on easy problems, but as the problem grows harder the chain-of-thought mode makes each sub-result visible and checkable. The intermediate tokens are not decoration — they are the computation. Without them, the model must compress the entire reasoning into a single forward pass through the output head.

The Real Complexity

Why does writing intermediate steps help so much? The answer is about computational depth.

A transformer block is a fixed-depth circuit: each token passes through the same number of layers regardless of problem difficulty. If you ask for the answer in one step, every sub-calculation must be compressed into that single output position — the model cannot reuse earlier results because they were never written down.

Chain-of-thought breaks that ceiling. Each intermediate token becomes a new input to the next generation step, effectively adding more sequential computation layers. The total depth of the computation is no longer bounded by the transformer's fixed number of layers; it is bounded by the length of the reasoning trace.

Formally, CoT lets a constant-depth transformer simulate a much deeper — potentially polynomial-depth — circuit, because each token generation is a new forward pass. Research by Merrill & Sabharwal (2023) showed that with CoT, transformers can solve problems in TC1\text{TC}^1 that they cannot solve in TC0\text{TC}^0 without scratch space. In plain terms: hard problems need more reasoning steps, and those steps must be visible.

Unlike NP-complete problems, which are hard because verification requires exponential search, CoT's gains are about depth not search — and that makes them practically achievable just by changing the prompt.

Where It Matters

Chain-of-thought is not an academic curiosity — it is the engine inside nearly every AI system that needs to be reliably correct:

  • Math and science tutoring: step-by-step solutions let students follow and catch errors, not just copy a number.
  • Code generation: writing pseudocode or a plan before generating code mirrors CoT exactly — the model debugs its own logic before committing to syntax.
  • Medical and legal reasoning: high-stakes decisions require an auditable trace; a bare answer with no justification is not trusted or safe.
  • Multi-hop question answering: answering "Which country has a higher GDP per capita, the one that borders France to the east and also borders Switzerland?" requires chaining facts — one-shot answers routinely fail.
  • Transformers and attention: the attention mechanism that reads the context is the same mechanism that reads the CoT trace — understanding one deepens the other.

The flip side: CoT costs tokens. Longer reasoning traces mean higher latency and more compute. Current research explores compressed scratchpads, process reward models that score reasoning quality, and tree-of-thought extensions that branch and prune candidate reasoning paths.

Conclusion

Chain-of-thought prompting revealed something profound: the tokens a model generates are not just its answer — they are its working memory. By making intermediate steps explicit, a fixed-depth architecture gains the ability to tackle problems that otherwise exceed its reach.

The practical upshot is elegant: if you want a language model to reason reliably, give it room to think out loud. The scratchpad is not a crutch — it is the computation. And the next time an AI gives you a confident wrong answer on a multi-step problem, ask it to show its work. You may be surprised how much the answer changes.

Share this article

Pick a channel — or use your device's native share sheet.

Comments

Loading comments...

https://www.kipuhub.com/en/article/chain-of-thought/Content licensed under CC BY-NC 4.0.