A reasoning LLM (OpenAI o1/o3-style) doesn't just emit one token stream — it explores a tree of partial "thoughts" the way a search algorithm explores a state space. Each node here is a reasoning step; expanding a node asks the model to propose b next thoughts, and a value function estimates how promising each one is:
value(node) ≈ P(solution reachable | thought so far)
frontier ← top-k nodes by value (beam width k)
expand each frontier node → b children
keep only the best k of (frontier ∪ children)
- Tree-of-Thought (best-first): at every depth, only the k highest-value nodes survive (a beam) — this is self-consistency and ToT in miniature: multiple candidate reasoning paths are generated, scored, and pruned in parallel, letting the search backtrack away from a low-value branch instead of committing to it.
- Chain-of-Thought (single path): the same tree with beam width forced to 1 — one thought leads greedily to the next with no branching or backtracking, matching how a standard (non-reasoning) LLM generates a single linear rationale.
- Branching factor b — how many candidate next-thoughts are sampled per expansion.
- Beam width k — how many parallel branches Tree-of-Thought keeps alive; k=1 degenerates exactly into Chain-of-Thought.
- Node color encodes its value estimate (green = high-value / likely on the winning path, gray = pruned once a stronger sibling out-scores it).
The bottom sparkline is not in the 3D original: it plots the winning branch's value estimate and the cumulative pruned-node count after every search step, so you can see the beam's quality converge (or thrash) over the run, not just at the current instant. Drag the tree canvas to pan and scroll/pinch to zoom — useful once branching factor and depth make the tree wider than the panel.
Real systems (ReAct, Tree-of-Thoughts prompting, OpenAI's o1/o3 reasoning models) use exactly this shape — generate candidate reasoning steps, score them (self-evaluation, a reward model, or majority voting), and search the resulting tree instead of decoding one greedy path — which is why they solve multi-step problems that trip up single-pass Chain-of-Thought.