Let the model think: reasoning and verification
Use step-by-step reasoning where it helps, separate thinking from the answer, and check the steps.
- Explain when chain-of-thought prompting helps
- Separate reasoning from the final answer for reliable parsing
- Verify reasoning steps instead of trusting them
Chain-of-thought prompting (Wei et al., 2022) asks a model to work through intermediate steps before answering. On multi-step problems - math, logic, planning, comparing options - this often improves accuracy, because each generated step becomes context for the next.
Practical patterns:
- Ask for reasoning in
<thinking>tags and the result in<answer>tags, so code can extract just the answer. - Many current models have built-in extended thinking; with them, a high-level instruction (“think carefully about edge cases”) often beats a hand-written step list.
- Self-consistency (Wang et al., 2022): sample several reasoning paths and take the majority answer.
Reasoning text is not a guarantee: steps can contain mistakes, and the explanation may not reflect how the answer was really produced. Check what matters - especially arithmetic, which tools do better.
Try it
Does it need step-by-step reasoning?
Decide whether asking for reasoning is likely to help.
“Which of three phone plans is cheapest for 14 GB a month with roaming in two countries?”
“Translate “Good morning” into French”
“Does this contract clause conflict with clause 7?”
“Extract the email address from this signature”
“Plan a database migration with zero downtime”
import re
reply = "<thinking>14 GB costs 25 on plan B...</thinking><answer>Plan B</answer>"
print(re.search(r"<answer>(.*?)</answer>", reply, re.DOTALL).group(1))Plan B
Key takeaways
Ask for step-by-step reasoning on multi-step problems; skip it for simple ones.
Keep reasoning and answer in separate tags.
Verify important steps - reasoning text can be wrong or unfaithful.
Lesson quiz
5 questions · pass with 4 correct · up to 50 XP
Passing this quiz completes the lesson and keeps your streak going. Questions you miss come back in review sessions later.
Practice: write Python
Write Python in the editor and run it against sample inputs. Python runs locally in your browser using a WebAssembly runtime.
Extract the answer from a reasoning reply
Replies are separated by lines containing only ===. For each reply, print the text inside <answer>…</answer> (stripped; it may span lines - join its lines with spaces), or no answer tag if it’s missing.
- Three replies
Python runs in a sandboxed browser worker with a 60 second time limit. Its runtime loads from the Pyodide CDN; your code stays in this browser.
Check the arithmetic in a reasoning trace
Each input line is a reasoning step. For every line containing A OP B = C (integers, OP one of + - * /), print line N: ok or line N: wrong, A OP B is X (for /, X may be a decimal - print it with :g). Skip lines without an equation.
- A phone-plan comparison
Python runs in a sandboxed browser worker with a 60 second time limit. Its runtime loads from the Pyodide CDN; your code stays in this browser.
Questions about this lesson
Stuck? Ask. Figured something out? Share it. Explaining is one of the best ways to learn.
Loading posts…