How Reasoning Models Work: Training Paths, Traces, and Cost

Reasoning models are language models that generate intermediate tokens — visible or hidden — before a final answer. That sentence hides most of the engineering. A model can produce such a trace because it was prompted to, because it was fine-tuned on traces, because it was optimized with a final-answer reward, or because several of those paths were combined. The trace is not a transparent readout of a separate reasoning module, and a correct answer is not a certificate that the trace is sound.

The one-sentence version: A reasoning model is a language model that has been set up — by prompting, training on supervised traces, process supervision, outcome-based reinforcement learning, distillation, or a combination — to spend tokens before answering. The visible trace is evidence about behaviour, not proof of the computation that produced the answer.

1. Three labels, not one ladder

Base, instruction-tuned, and reasoning are useful labels, but they are not a strict ladder. A base model can follow in-context examples. An instruction-tuned model can be a poor reasoner. A reasoning model can start from either a base checkpoint or an instruction-tuned one.

Label	Usual meaning	Typical training signal	What it does not imply
Base model	A transformer trained on broad text with next-token prediction. It can do in-context learning and pattern completion.	Next-token cross-entropy on a large corpus.	That it cannot reason, or that it cannot produce a chain of thought when prompted.
Instruction-tuned model	A base model further trained to follow instructions and chat formats.	Supervised fine-tuning on demonstrations, optionally followed by preference optimization.	That it was trained for long reasoning, or that every instruction-tuned model uses the same recipe.
Reasoning model	A model trained or prompted to emit intermediate tokens before answering, often at greater length.	One or more of: long chain-of-thought SFT, process supervision, outcome-based RL, distillation, or prompting only.	That a single objective produced it, or that its visible trace is faithful.

Two corrections to common shorthand matter for everything that follows.

First, reasoning training does not always start from an instruction-tuned model. DeepSeek-R1-Zero applies reinforcement learning directly to a base model without a supervised fine-tuning stage (DeepSeek-AI, 2025, §2.3). The old shorthand “instruction-tuned plus chain of thought” describes one common path, not the definition.

Second, outcome-based RL does not usually reward “faithful intermediate steps.” R1-Zero’s rule-based reward combines final-answer accuracy with a format reward; the content of the trace is not directly supervised (DeepSeek-AI, 2025, §2.2). A correct final answer can therefore be optimized without any step-level guarantee.

2. The training paths behind reasoning behaviour

The paths below are mechanisms, not a canonical pipeline. Real models often combine them, and some labs do not disclose the mix.

2.1 Prompting: change the context, not the weights

Chain-of-thought (CoT) prompting supplies examples of intermediate steps, or a trigger phrase, and lets the model continue. Wei et al. used eight CoT exemplars and reported state-of-the-art GSM8K accuracy with a 540B-parameter model, with gains across arithmetic, commonsense, and symbolic tasks (Wei et al., 2022, §2 and §3.4). Kojima et al. showed that the single phrase “Let’s think step by step” raised accuracy on MultiArith from 17.7% to 78.7% and on GSM8K from 10.4% to 40.7% with text-davinci-002 (Kojima et al., 2022, §3).

Prompting is real, but it is not training. The weights do not change. The gain depends on the model and task, and the generated text is still a sample from the model’s existing policy. Prompting can reveal a capability; it does not establish where that capability came from.

Empirically, CoT prompting produces measurable gains in the cited settings: Wei et al. show large GSM8K gains with a 540B-parameter model using eight CoT exemplars (§2.1), and Kojima et al. show zero-shot CoT gains on text-davinci-002 (§2.1). These examples demonstrate that prompting can improve accuracy in specific model/benchmark settings; they do not establish a general comparison of prompting gains across models with and without trained reasoning. The boundary between instruction-tuned and reasoning models also blurs: many reasoning-enhanced checkpoints are instruction-tuned models further fine-tuned on CoT trajectories (the DeepSeek-R1 cold-start stage is one documented example, §2.3).

2.2 Supervised reasoning data: imitate traces

Supervised fine-tuning (SFT) on reasoning data trains the model to predict the tokens of a provided trace, usually with a question, a trace, and an answer. The traces can be human-written, generated by a stronger teacher, or generated by the model itself and filtered.

Three documented variants:

• STaR generates rationales, retries with the correct answer when the answer is wrong, and fine-tunes on the rationales that ultimately produced correct answers (Zelikman et al., 2022, §3).
• s1 fine-tunes on a curated set of 1,000 question-trace pairs and adds budget forcing at test time (Muennighoff et al., 2025, §2 and §3).
• DeepSeek-R1 distillation fine-tunes open base models on 800,000 samples generated by R1, using SFT only for the distilled models (DeepSeek-AI, 2025, §3 and Appendix F).

What this path optimizes is the likelihood of the provided tokens. It can teach format, useful decompositions, and domain patterns. It does not directly optimize final-answer correctness, and it will imitate flaws in the data. Filtering traces by final answer helps, but it does not make every intermediate step true or faithful.

2.3 Process supervision: score the steps

Process supervision supplies feedback at each intermediate step rather than only at the final answer. Lightman et al. compared process-supervised and outcome-supervised reward models on the MATH dataset, reported that process supervision outperformed outcome supervision in their setup, and released PRM800K with 800,000 step-level human labels (Lightman et al., 2023, §2.6 and §4.1).

Uesato et al. found a more nuanced tradeoff on GSM8K: pure outcome-based supervision produced similar final-answer error rates with less label supervision, but process-based supervision or a learned process reward model was needed to reduce reasoning errors among final-answer-correct solutions (Uesato et al., 2022, §3). They reported final-answer error falling from 16.8% to 12.7% and reasoning error among final-answer-correct solutions falling from 14.0% to 3.4% in their best setting.

Process supervision is expensive, and its reward model is a proxy that can be wrong or gamed. It can be used to rerank candidate solutions or to train a policy. It is not the same thing as outcome-based RL.

2.4 Outcome-based RL: sample, score the final answer, update

Outcome-based RL samples one or more traces, scores the final result, and updates the policy to make higher-scoring traces more likely. DeepSeek-R1-Zero is a clear documented case: it uses Group Relative Policy Optimization on a base model, with rule-based accuracy rewards for math, code, and logic plus a format reward, and no neural reward model for the reasoning tasks (DeepSeek-AI, 2025, §2.1 and §2.2).

The reward is a scalar on the final answer and format. The model is free to discover the intermediate text. That freedom is the point, and it is also the risk: the objective does not directly reward a true or faithful step.

A final-answer verifier used for reranking is a different mechanism. Cobbe et al. trained a verifier to rank many candidate solutions and select the highest-ranked one on GSM8K (Cobbe et al., 2021, §4.2). The signal is still outcome-based, but it is used for selection rather than a policy update. Keep the signal and the algorithm separate.

2.5 Hybrid training and distillation

DeepSeek-R1 is not R1-Zero plus a cosmetic patch. Its pipeline uses cold-start data with a conversational thinking style, RL training, rejection sampling followed by another SFT stage, and a second RL stage for helpfulness and harmlessness (DeepSeek-AI, 2025, §3). The cold-start stage was added partly because R1-Zero had poor readability and language mixing. Distilled models then use SFT on teacher-generated data rather than the full RL pipeline.

There is no universal recipe

The examples above are existence proofs, not a canonical sequence. Different labs use different mixes of prompting, SFT, process supervision, outcome RL, rejection sampling, and distillation, and some do not disclose the mix. A model that emits long traces may have been trained with any of these paths. A model that does not expose a trace may still have generated one internally. Do not infer the training method from the visible behaviour alone.

3. What outcome-based RL does and does not establish

Outcome-based RL has a genuine result behind it: in verifiable domains, optimizing final-answer rewards can improve final-answer accuracy. R1-Zero is the clearest public example (DeepSeek-AI, 2025, §2.3).

It does not establish three stronger claims.

It does not establish faithful traces. The reward is on the final answer and format, not on the truth of each step. A policy can learn a trace that produces correct answers while containing false intermediate claims, omitted factors, or post-hoc rationalisation.

It does not establish that RL creates reasoning from nothing. Yue et al. evaluated RL with verifiable rewards across model families and benchmarks. They found that RLVR-trained models outperform their base models at small pass@k, but base models achieve higher pass@k at large k in their setup, and they conclude that the tested RLVR methods do not elicit fundamentally new reasoning patterns; distillation, by contrast, can expand the boundary (Yue et al., 2025, §3 and §4.2). That is evidence against a universal “RL teaches reasoning from scratch” story, not proof that no RL method can ever do so.

It does not remove reward hacking. DeepSeek explicitly avoided neural reward models for its reasoning RL because it observed that such models are susceptible to reward hacking at scale (DeepSeek-AI, 2025, §2.2). Chen et al. found that when RL increased how often models used a hint, the models did not become more likely to verbalise that hint (Chen et al., 2025, §5). A policy can get better at scoring while getting worse at telling you why.

The practical consequence is to evaluate capability, faithfulness, and cost as separate axes. A higher benchmark score is not a faithfulness measurement.

4. A traced problem: intermediate tokens, final-answer checking, and cost

The trace below is a constructed illustration, not a logged model output. It is written to contain a plausible flaw. The token counts are real for the exact strings shown, using tiktoken’s cl100k_base encoding; another tokenizer or chat template will give different counts.

Prompt (29 tokens):

A hall has 12 rows of chairs. Each row has 4 chairs. 7 chairs are removed for repairs. How many chairs remain?

Reasoning trace (72 tokens):

Let me work through this. There are 12 rows, and each row has 4 chairs. 12 x 4 = 50. Two chairs in the front row are broken, so 50 - 2 = 48. Then 7 chairs are removed for repairs: 48 - 7 = 41. The answer is 41.

Final answer (1 token):

41

4.1 Step check

The independent computation is 12 × 4 = 48, then 48 − 7 = 41. The extracted answer is 41, so a final-answer checker passes. The process check is less kind.

Step	Trace claim	Verdict
1	12 rows with 4 chairs each	Supported by the prompt.
2	12 x 4 = 50	False. The correct product is 48.
3	Two chairs in the front row are broken	Unsupported. The prompt says 7 chairs are removed; it does not mention broken chairs or a front row.
4	50 - 2 = 48	Arithmetic is valid, but it depends on the false total and the invented premise.
5	48 - 7 = 41	Valid given 48.
6	Final answer 41	Matches the independent computation.

This is the central distinction. The final-answer check is a one-bit check: does the extracted answer match the expected answer? It cannot see that step 2 was wrong or that step 3 invented a fact. The two errors happen to compensate: 50 - 2 = 48, and then 48 - 7 = 41. The answer is right for the wrong reasons.

The trace is plausible because it has the shape of reasoning: quantities, operations, a correction, and a conclusion. Plausibility is a human judgement about form. Unsupported steps (those that invent premises not in the problem, or make incorrect claims) are not the same as sound steps. A trace containing unsupported steps can still produce a correct answer if errors compensate, but the steps are not an audit trail of the computation that produced the answer.

The converse also happens. Constructed illustration. A discount problem might ask for the final price after 25% off and then 10% tax. A trace that computes 10% of the original price instead of 10% of the discounted price gets $68 instead of $66. The steps look reasonable, but the answer is wrong. A plausible trace can fail in either direction.

4.2 Cost

Assume, only for this arithmetic, that input tokens cost $3 per million and output tokens cost $15 per million, and that reasoning tokens are billed at the output rate. These are hypothetical prices, not a vendor quote. Check your provider’s current price and billing policy before using the numbers operationally.

For the traced call:

input cost  = 29 × 3 / 1,000,000  = 0.000087
output cost = (72 + 1) × 15 / 1,000,000 = 0.001095
total       = 0.001182

The direct-answer baseline uses the same 29-token prompt and a 1-token answer:

input cost  = 29 × 3 / 1,000,000 = 0.000087
output cost = 1 × 15 / 1,000,000 = 0.000015
total       = 0.000102

The traced call costs about 11.6 times the direct-answer call: $0.00118 versus $0.00010. At 1,000 calls, that is roughly $1.18 versus $0.10, before caching, batching, or retries.

Reasoning tokens R	Output tokens R + 1	Output cost at $15/M	Total cost including input	Multiple of direct answer
0	1	$0.000015	$0.000102	1.0×
72	73	$0.001095	$0.001182	11.6×
500	501	$0.007515	$0.007602	74.5×
2,000	2,001	$0.030015	$0.030102	295×

Under a fixed price, cost grows linearly in output tokens. Latency does not have to grow linearly: batching, hardware, and the serving stack change throughput. But each reasoning token is another decode step and another entry in the KV cache. Long traces increase both the bill and the memory footprint, and the effect is easy to underestimate because the final answer is short. Measure end-to-end latency and cost under your actual batch size and context length.

Budget the trace, not just the answer

A reasoning feature can look cheap per call and expensive per thousand calls. Cap the reasoning budget, log prompt and completion tokens separately, and compare the cost against the value of the extra accuracy. If you sample multiple traces, multiply the cost before selection; the rejected traces are still billed.

5. Trace faithfulness: what primary research shows

Three properties are often conflated:

• Correctness: the final answer matches the expected answer.
• Faithfulness: the stated trace reflects the process that actually produced the answer.
• Plausibility: a human finds the trace reasonable.

Correctness and faithfulness are independent. A trace can be correct but unfaithful, faithful but wrong because of an honest mistake, correct and faithful, or wrong and unfaithful. Plausibility can appear in any of those cells.

	Faithful trace	Unfaithful trace
Correct answer	The ideal case: the answer is right and the trace is an accurate account.	Correct by luck, compensating errors, or post-hoc rationalisation. The trace is not an audit trail.
Wrong answer	An honest mistake: the trace reflects the process, but the process is wrong.	Wrong and misleading: neither the answer nor the account can be trusted.

The research does not support a single verdict on CoT. It supports calibrated distrust.

Intervening on the trace changes answers. Lanham et al. tested early answering, adding mistakes, paraphrasing, and replacing the trace with filler tokens. Models varied widely in how strongly they conditioned on the trace, and larger, more capable models produced less faithful reasoning on most tasks they studied. They also concluded that CoT can be faithful when the model size and task are carefully chosen (Lanham et al., 2023, §2.3–2.6 and §3). So “always faithful” and “never faithful” are both wrong.

Biasing features can be omitted from explanations. Turpin et al. added features such as reordering multiple-choice options so the answer was always “(A).” Models often produced explanations that did not mention the bias, and when biased toward incorrect answers they rationalised those answers; accuracy dropped by as much as 36% across 13 BIG-Bench Hard tasks with GPT-3.5 and Claude 1.0 (Turpin et al., 2023, §3.2). The explanation was plausible, but it was not a complete account of the cause.

Intermediate tokens can carry computation without carrying meaning. Pfau et al. showed that transformers can use meaningless filler tokens such as “……” in place of a chain of thought to solve two algorithmic tasks, and that learning to use filler tokens is difficult and requires dense supervision (Pfau et al., 2024, §1 and §4). The extra tokens can provide computation independent of their semantic content. A trace that looks like a sequence of steps may therefore be doing something other than what the words suggest.

Reasoning models often do not verbalise the cues they use. Chen et al. evaluated CoT faithfulness across six reasoning hints. In most settings and models, the CoT revealed use of a hint in at least 1% of the examples where the hint was used, but the reveal rate was often below 20%. Outcome-based RL initially improved faithfulness but plateaued without saturating, and when RL increased hint use, the propensity to verbalise the hint did not increase (Chen et al., 2025, §3–5). Monitoring traces is promising, but it is not sufficient to rule out undesired behaviour.

Outcome rewards do not directly supervise the trace. R1-Zero’s reward is final-answer accuracy plus format (DeepSeek-AI, 2025, §2.2). That design can produce strong final-answer performance without making each intermediate claim true. The trace is a learned artefact of a policy optimized for the answer, not a transcript of a verified proof.

Claims to avoid

Do not say “the trace is the model’s true reasoning,” “a correct answer proves the trace,” “all CoT is unfaithful,” or “outcome RL trains faithful intermediate steps.” Do not reduce unfaithfulness to a different random seed: a greedy or deterministic trace can also be post-hoc, omit a causal factor, or contain a fabricated step. The evidence supports neither blind trust nor blanket dismissal.

6. Checks that catch different failures

No single check covers correctness, faithfulness, and cost. Use the check that matches the failure mode.

Check	What it catches	What it misses
Final-answer check	A wrong or unparseable final answer.	Every intermediate error that happens to produce the right answer.
Unit tests or a compiler	Wrong code behaviour on covered cases.	Uncovered inputs, specification errors, and natural-language claims.
Calculator or symbolic solver	Arithmetic and algebra errors.	Premises that were never in the problem, and unit or modelling errors.
Process verifier	Invalid or unsupported steps, if the verifier is reliable.	Verifier blind spots and reward hacking; it is a learned proxy, not a proof.
Retrieval or citation check	Unsupported factual claims.	Arithmetic and logical validity.
Human audit	Subtle conceptual errors and misleading framing.	Scale, cost, and consistency across many calls.
Multiple samples or self-consistency	Some variance and unstable answers.	Shared blind spots: many samples can agree on the same wrong trace.

The practical rule is defence in depth. A final-answer checker is necessary for a scored task, but it is not a process check. A process verifier is useful, but it is not a formal proof. A human audit is stronger on meaning, but it does not scale to every call. Combine checks and state which one produced the evidence.

7. Exit task: classify a trace and compute its cost

Use the tank problem and the trace below. The trace is another constructed illustration, not a model transcript.

Problem (28 tokens):

A tank holds 500 liters. It is 40% full. 30 liters are added. How many liters are in the tank?

Trace (42 tokens):

40% of 500 is 250. Then 250 plus 30 is 280. But 50 liters evaporated, so 280 minus 50 is 230. The answer is 230.

Final answer (1 token):

230

Exit task: correctness, faithfulness, and cost

1. 01Compute the correct final answer independently. Then extract the trace's final answer and state whether a final-answer checker passes.

   Show a hint

   Compute 40% of 500 first, then add 30.

2. 02Identify the first invalid arithmetic step and any claim that the prompt does not support.

   Show a hint

   Check the percentage calculation before checking the compensation.

3. 03Classify the trace using the correctness and faithfulness matrix. Justify the classification in two sentences.

   Show a hint

   A correct answer can coexist with a flawed trace if the errors compensate.

4. 04Using 28 prompt tokens, 42 reasoning tokens, 1 answer token, and hypothetical prices of $3 per million input tokens and $15 per million output tokens, compute the trace cost, the direct-answer baseline, and the ratio.

   Show a hint

   Assume reasoning tokens are billed at the output rate. Compute input and output separately.

5. 05Name one external check that would catch the flaw and one failure that the same check would still miss.

   Show a hint

   A calculator checks arithmetic; it does not check whether a premise belongs in the problem.

Check your work

1. Correct answer: 500 × 0.40 = 200, then 200 + 30 = 230. The trace’s final answer is 230, so the final-answer checker passes.
2. First invalid step: “40% of 500 is 250” is false; 40% of 500 is 200. The claim that 50 liters evaporated is unsupported by the prompt. The errors compensate: 250 − 50 = 200, then 200 + 30 = 230.
3. Classification: correct final answer, unsound trace. The answer is right, but the stated process contains a false arithmetic claim (40% of 500 = 250) and an invented premise (50 liters evaporated), so the trace is not an audit trail. This is a constructed example; unfaithfulness is an empirical property measured by intervention studies (Lanham et al., Turpin et al., Chen et al.), not by inspection of a single trace.
4. Cost: input 28 × 3 / 1,000,000 = 0.000084; output (42 + 1) × 15 / 1,000,000 = 0.000645; total 0.000729. Direct baseline: 0.000084 + 1 × 15 / 1,000,000 = 0.000099. Ratio: 0.000729 / 0.000099 ≈ 7.36. At 1,000 calls: about $0.73 versus $0.10.
5. Check: a calculator or step-level verifier catches the percentage error; it can still miss an unsupported premise, a misleading omission, or a verifier’s own blind spot. A human audit catches more meaning but costs more and is less consistent.

8. Key takeaways

Key takeaways

• Base, instruction-tuned, and reasoning are labels, not a strict ladder. Reasoning training can start from a base model or an instruction-tuned one.
• Prompting, supervised traces, process supervision, outcome-based RL, and distillation are different mechanisms. Public examples are existence proofs, not a universal recipe.
• Outcome-based RL optimizes a final-answer signal. It can improve accuracy without making the intermediate trace true or faithful.
• A final-answer check is a one-bit check. A plausible trace can pass it while containing false steps, invented premises, or compensating errors.
• Trace faithfulness is a causal claim, not a style judgement. Use interventions, process checks, external tools, and human review according to the failure mode.
• Reasoning tokens are generated tokens. When billed at the output rate, cost grows with trace length; KV-cache use grows regardless.

9. Sources and claim-to-source mapping

Source	Exact section	What it supports	Scope and limits
Wei et al. (2022), “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models” (arXiv:2201.11903)	§2, §3.4	CoT prompting with exemplars improves arithmetic, commonsense, and symbolic tasks; eight exemplars with a 540B model reached state-of-the-art GSM8K accuracy.	Prompting only; gains are scale- and task-dependent; the paper does not claim a universal training recipe.
Kojima et al. (2022), “Large Language Models are Zero-Shot Reasoners” (arXiv:2205.11916)	§3	“Let’s think step by step” raised MultiArith from 17.7% to 78.7% and GSM8K from 10.4% to 40.7% with text-davinci-002.	Zero-shot prompting on specific benchmarks; model-dependent.
Ouyang et al. (2022), “Training language models to follow instructions with human feedback” (arXiv:2203.02155)	§3.1	InstructGPT used supervised fine-tuning, a reward model trained on human rankings, and PPO; a 1.3B InstructGPT model was preferred to 175B GPT-3 on the authors’ prompt distribution.	One instruction-tuning recipe, not the definition of instruction tuning; not a reasoning-training paper.
Lightman et al. (2023), “Let’s Verify Step by Step” (arXiv:2305.20050)	§2.6, §4.1	Process supervision outperformed outcome supervision in the authors’ MATH setup; the process-supervised model solved 78% of a representative MATH subset; PRM800K contains 800,000 step-level labels.	MATH and the authors’ reward-model setup; process supervision is expensive and its reward model is a proxy.
Uesato et al. (2022), “Solving math word problems with process- and outcome-based feedback” (arXiv:2211.14275)	§3	Pure outcome supervision produced similar final-answer error with less label supervision, but process-based supervision reduced reasoning errors among final-answer-correct solutions.	GSM8K; specific models and annotation setup; results do not imply that outcomes are sufficient for faithful reasoning.
Cobbe et al. (2021), “Training Verifiers to Solve Math Word Problems” (arXiv:2110.14168)	§4.2	An outcome-based verifier can rerank many candidate solutions and improve GSM8K performance.	Verification and reranking, not policy optimization; specific dataset and model scale.
DeepSeek-AI (2025), “DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning” (arXiv:2501.12948)	§2.1–2.3, §3	R1-Zero uses GRPO on a base model with rule-based accuracy and format rewards, no neural reward model, and no SFT before RL; R1 adds cold-start data, RL, rejection-sampling SFT, and a second RL stage; distilled models use SFT only.	One lab’s recipe; R1-Zero had readability and language-mixing issues; the reward does not directly supervise trace faithfulness.
Muennighoff et al. (2025), “s1: Simple test-time scaling” (arXiv:2501.19393)	§2, §3	A curated set of 1,000 question-trace pairs plus budget forcing can produce strong reasoning performance after SFT.	SFT and test-time intervention on a specific model; not an RL result and not a universal data recipe.
Zelikman et al. (2022), “STaR: Bootstrapping Reasoning With Reasoning” (arXiv:2203.14465)	§3	STaR generates rationales, rationalises with the correct answer when needed, and fine-tunes on rationales that produced correct answers.	Self-generated SFT on specific tasks; filtering by final answer does not guarantee faithful steps.
Lanham et al. (2023), “Measuring Faithfulness in Chain-of-Thought Reasoning” (arXiv:2307.13702)	§2.3–2.6, §3	Interventions on the trace change answers; models vary in how strongly they condition on the trace; larger models were less faithful on most studied tasks; CoT can be faithful in some settings.	Specific models and tasks; faithfulness is operationalised through interventions, not a universal metric.
Turpin et al. (2023), “Language Models Don’t Always Say What They Think” (arXiv:2305.04388)	§3.2	Biasing features can change answers while being omitted from explanations; accuracy dropped by as much as 36% across 13 BIG-Bench Hard tasks with GPT-3.5 and Claude 1.0.	Few-shot settings and specific biases; not a claim that every explanation is unfaithful.
Pfau et al. (2024), “Let’s Think Dot by Dot” (arXiv:2404.15758)	§1, §4	Filler tokens can improve performance on two algorithmic tasks; intermediate tokens need not encode the computation in a human-interpretable form.	Synthetic algorithmic tasks; learning to use filler tokens required dense supervision; not a general claim about all CoT.
Chen et al. (2025), “Reasoning Models Don’t Always Say What They Think” (arXiv:2505.05410)	§3–5	Across six hints, CoT reveal rates were often below 20%; outcome-based RL initially improved faithfulness but plateaued; reward hacking increased hint use without increasing verbalisation.	Specific reasoning models and hint settings; monitoring is promising but not sufficient.
Yue et al. (2025), “Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?” (arXiv:2504.13837)	§3, §4.2	RLVR improved pass@1 at small k, but base models achieved higher pass@k at large k in the tested setup; distillation expanded the reasoning boundary.	Math, code, and visual benchmarks; specific RL algorithms; not proof that no future RL method can create new capability.

Verification gaps stated:

• The two traced examples are constructed illustrations, not sampled model outputs. The token counts were computed with tiktoken’s cl100k_base on the exact strings shown; other tokenizers and chat templates will differ.
• The prices are hypothetical and are used only for arithmetic. They are not a vendor quote. Latency and KV-cache costs are discussed qualitatively and were not measured here.
• The cited papers measure specific models, datasets, and training setups. Their numbers are not universal benchmarks, and none establishes a single recipe for reasoning models.
• Some training details are undisclosed. The paths in section 2 are a map of documented mechanisms, not a claim about how any particular commercial model was built.
• Faithfulness is operationalised differently across the cited papers. Compare the intervention or metric before comparing headline numbers.

Related lessons

• LLM Internals & Architecture · Inference foundationsHow LLM inference works9 min
• LLM Internals & Architecture · Inference foundationsUnderstanding the KV Cache12 min
• LLM Internals & Architecture · Transformer architecture and model inputsTokenisation and Embeddings: From Text to Tensors12 min