Thinking

How extended-thinking tokens relate to next-token prediction and to the output the user sees.

Grounds harness decisions on effort levels, session splitting and fan-out. Not an Anthropic spec: the published API model plus what the client observes. Cache effect of a thinking change: Prompt caching.

One stream, no second mechanism

Thinking and visible output come from one autoregressive loop: one forward pass per token, sample, append, repeat. A thinking block is a span between special tokens that post-training (RL against outcome rewards) teaches the model to use for pre-answer deliberation. The API splits spans into thinking and text blocks; the model sees one sequence.

Consequence	Detail
Thinking tokens are output tokens	Billed as output, counted against `max_tokens`, capped by the effort or budget parameter. Slashing the budget only caps how long the pre-answer span runs.
The answer attends to the thinking	Via ordinary attention, like any context; nothing summarizes or distills the scratchpad into the answer.

Why more thinking tokens buy capability

A transformer spends fixed compute per token: one pass through the layers. Between steps only the KV cache carries state, holding only what was emitted or read. Tokens are the only medium for multi-step computation: step 3's result must exist as tokens before step 7 can use it.

Serial computation scales linearly with tokens: a 10k-token span is ~10k extra forward passes with attendable intermediate products, a 123-token span 123. Long dependency chains (reconciling documents, eliminating hypotheses in order, tracking case splits) must be written out link by link.

Effort levels do only this. A smaller budget removes serial steps, not per-token intelligence, so the model commits earlier.

Failure mode

Greedy commitment: the first salient reading of an ambiguous instruction wins, because the arbitration steps that would reject it were never generated. It looks like anchoring and literal-minimal reading, not forgetting: distant context is attended but no longer weighed against alternatives.

Interleaved thinking in agent loops

An agent turn is many calls: think → tool call → tool result → think → tool call → … Choosing the next call rarely needs a long chain, so reasoning slices into short inter-tool blocks. The "Inference Gap" analysis measured this: a low per-call median over agent traffic is mostly slicing, not necessarily a cut.

A chain interrupted by a tool call must survive the round trip as written context; harnesses manage thinking aggressively:

Prior-turn thinking may be stripped or pruned on later calls (cache management). Only visible text, files and tool results reliably persist.
Some model tiers redact or encrypt thinking client-side and never persist it in transcripts.

Rule

Externalize reasoning that must survive: plans, assumptions tables, ledgers, findings files. A conclusion on disk survives budget caps and pruning; one only in a thinking span is gone once pruned.

Serial depth vs parallel width

Thinking is serial compute; the two ways to buy computation are not interchangeable.

Lever	Buys	Use when
Serial: more thinking per attempt	A longer dependency chain.	The answer needs the chain: design decisions, root-cause elimination.
Parallel: more attempts	n independent shallow chains.	A cheap verifier selects among candidates (verification asymmetry: recognizing a correct finding is cheaper than producing one). Review fan-out with refuters works; n parallel shallow designs without a selection gate are n wrong answers.

Faithfulness caveat

Thinking text is trained for outcome, not honest narration: the medium of the computation, not necessarily a faithful account of it. The model may reach an answer for reasons the scratchpad never states. Treat visible thinking as a capability mechanism, not an audit log.

Summary table

Property	Implication
Thinking tokens = output tokens, same loop	Budget caps are token caps on the pre-answer span.
No inter-step state except emitted tokens	Serial compute scales with tokens generated.
Answer attends to scratchpad via normal attention	Nothing survives that isn't written down.
Agent loops slice thinking across tool calls	Per-call thinking is naturally short; chains must survive as artifacts.
Harnesses prune prior thinking from context	Externalize conclusions to files, not scratchpad.
Parallel attempts ≠ serial depth	Fan out only what a verifier can select.