A prompting technique that asks a language model to spell out its reasoning step by step before giving the final answer.
People using prompts for multi-step tasks like arithmetic, logic puzzles, and code generation read this.
01What it is and how it works
CoT works by inserting an explicit cue—often a short sentence like "Let's think step by step"—into the prompt. The model then generates a series of intermediate statements that form a logical chain, before producing the final output. Each intermediate token is treated as part of the answer, so the model can correct itself along the way. This extra reasoning layer has been shown to improve accuracy on multi‑step tasks such as arithmetic, logic puzzles, and code generation.
It tells the model to think out loud, list each step, then answer.
02What to do about it
Start using CoT in any prompt that requires more than one inference step. Step 1: Add a clear cue (e.g., "Think through the problem step by step.") at the end of the user query. Step 2: If the model stops early, increase the max token limit or add a reminder like "Continue the reasoning." Step 3: Review the intermediate steps for completeness before trusting the final answer. Step 4: Log the full response so you can later compare accuracy with non‑CoT runs.
03How it is measured or noticed
You can spot CoT output by looking for multiple short sentences that each contain a piece of the solution. In logs, the token count will be higher than a plain answer. To measure impact, run an A/B test: one set of prompts with the CoT cue, another without. Compare final‑answer correctness and note any reduction in hallucinations. Tools that extract reasoning steps (e.g., regex for "Step \d+:") make automated scoring easier.
04Common mistakes
- Assuming a model will generate a chain of thought without an explicit cue.
- Using a cue that is too vague, such as just "Explain" without asking for steps.
- Evaluating only the final sentence and ignoring errors in the intermediate steps.
- Setting a max token limit that cuts the reasoning short.
05Limits
CoT adds latency because the model produces more tokens. It works best with models that have strong reasoning abilities; older or smaller models may produce shallow or irrelevant steps. The technique does not help with pure factual recall where no reasoning is needed. It is often confused with self‑consistency (sampling multiple CoT chains and voting) or with few‑shot prompting, which are separate strategies.
06Worked example
"Q: A train travels 60 miles per hour for 3 hours, then 40 miles per hour for 2 hours. What is the total distance?
A: Let's think step by step.
Step 1: Distance in the first segment = 60 mph × 3 h = 180 miles.
Step 2: Distance in the second segment = 40 mph × 2 h = 80 miles.
Step 3: Total distance = 180 miles + 80 miles = 260 miles.
Answer: 260 miles."
Frequently asked questions
How is Chain of Thought different from simply asking the model to give an answer?
Usually, Chain of Thought is a prompting technique that asks the model to spell out its reasoning step by step, while a plain prompt expects a direct answer without intermediate explanation.
Should I always use Chain of Thought for multi‑step problems?
It depends on the complexity of the task; for problems that need more than one inference step, Chain of Thought often improves accuracy, but for simple factual queries it can add unnecessary latency.
How do I actually add a Chain of Thought cue to my prompt?
Usually, you insert a short phrase such as “Let’s think step by step” or “Reason through the problem” before the question, which signals the model to generate a stepwise explanation.
Does Chain of Thought still work with the latest model versions?
Yes, most modern language models still respond to Chain of Thought cues, although the magnitude of improvement can vary with model size and training data.
What are the risks if I forget to include the Chain of Thought cue?
Usually, the model will skip the reasoning process and give a direct answer, which may be less accurate for complex tasks; you’ll notice the output lacks the intermediate steps that help verify the solution.
How long does it take for Chain of Thought output to appear compared to a normal answer?
It usually adds latency proportional to the number of reasoning steps, so you may see a few extra seconds or more tokens before the final answer is produced.
Asked out loud
spoken, not typedThe same term in the words somebody uses speaking to an assistant rather than typing into a box — written from the situation, which is why each one carries the situation it came from.
Yes, I can break the problem down into individual operations so you can see each part of the calculation and spot any errors quickly.
Usually, I’ll list the reasoning steps one by one, which lets you narrate the process clearly while you stay engaged in the conversation.
It depends, but I can provide a step‑by‑step trace of the logic, letting you compare each stage to your work and identify the mistake.