DAN Analysis 9 min read August 1, 2026

From Game of 24 to o3: How Tree of Thoughts Shaped Native Reasoning Models in 2026

AI reasoning branches converging — Tree of Thoughts paper lineage through o3 and Claude Fable 5 native inference

TL;DR

The shift: Tree of Thoughts brought systematic multi-path reasoning to LLMs in 2023 — by 2026, that logic is native infrastructure in frontier models.
Why it matters: Teams still building explicit ToT scaffolding are solving a problem the model now handles internally.
What’s next: ToT stays essential for constrained inference and open-source stacks — frontier developers move to prompting the reasoning layer, not engineering it.

A Princeton/Google team published “Tree of Thoughts: Deliberate Problem Solving with Large Language Models” in May 2023 (Yao et al. 2023). It wasn’t a prompt hack. It was a direct challenge to the assumption that language models should commit to a single reasoning path without exploring alternatives.

Three years later, every major frontier model has internalized that challenge. Most teams still engineering explicit Prompt Chaining pipelines around it haven’t caught up.

The Architecture Behind the Benchmark

Thesis: The Tree of Thoughts paper didn’t describe a prompting trick — it described what systematic reasoning actually requires: exploration, evaluation, and the ability to abandon a dead path.

Self Consistency sampling pushed chain-of-thought outputs toward majority vote. ReAct Prompting interleaved action and reasoning. Neither gave the model a way to recover when a promising path collapsed.

ToT did.

The core mechanic: decompose a problem into discrete “thoughts,” use the LLM to evaluate each as “sure / maybe / impossible,” and run a tree search — breadth-first or depth-first — across candidate paths. Explicit Backtracking when a branch hits a dead end.

The Game of 24 benchmark made the point precisely. Standard chain-of-thought prompting with GPT-4 succeeded 4% of the time. ToT with BFS, five candidates, three decomposition steps: 74% (Yao et al. 2023).

That’s not marginal. That’s a different class of problem-solving.

On mini crosswords and creative writing tasks, ToT substantially outperformed input-output and chain-of-thought baselines — full results in Yao et al. 2023 and the NeurIPS Proceedings. The paper was accepted at NeurIPS 2023. Labs paid attention.

The Signal That Reached the Labs

The princeton-nlp/tree-of-thought-llm repository accumulated roughly 6,000 GitHub stars (princeton-nlp GitHub). That’s significant for a research artifact. It means practitioners weren’t just reading the paper — they were running it.

OpenAI released o3 in April 2025, describing its mechanism as “private chain of thought” reasoning trained via reinforcement learning (OpenAI Blog). Third-party analysis of o3 reasoning traces found patterns consistent with “semantic backtracking” — the model revisiting prior steps, abandoning unproductive paths, pursuing alternatives.

OpenAI has not confirmed any direct ToT architectural lineage. The observable pattern, however, is recognizable.

Anthropic announced extended thinking in early 2025. The documentation frames the relationship clearly: extended thinking and explicit ToT approaches are “complementary, not a replacement” — extended thinking handles deliberation internally; ToT externalizes the search structure (Anthropic Docs).

The reasoning layer moved inside the model.

The current Anthropic lineup shows how far this went. Claude Fable 5 and Claude Mythos 5 — the most current Anthropic models as of June 2026 — use adaptive thinking via an effort parameter, always enabled. Claude Opus 4.5 and Haiku 4.5 support manual extended thinking via budget_tokens.

The principle ToT formalized in 2023 is now native inference behavior for the frontier.

Who Moves Up

Research teams who understood why ToT worked are ahead of teams who only implemented it.

Understanding what Game of 24 actually measures — why greedy single-path commitment fails on constraint satisfaction problems — gives you the right mental model for evaluating o3, Claude Fable 5, or any reasoning-capable model. The benchmark discipline transfers. The scaffolding doesn’t have to.

Frontier model users win immediately. You don’t have to implement a search tree, score intermediate thoughts, or manage branching prompt structures. The model handles the deliberation. You prompt for the outcome.

Teams building evaluation frameworks for multi-step reasoning are positioned well. The ToT benchmarks set the standard for what “deliberate reasoning” means quantitatively. That baseline is table stakes now for assessing any reasoning model.

You’re either using native reasoning or you’re reinventing it in prompt space.

Who Gets Left Behind

Teams that built production pipelines around explicit ToT scaffolding — managing thought trees, evaluation prompts, backtracking logic in prompt space — built something frontier models now handle internally.

The princeton-nlp reference implementation makes the problem concrete: v0.1.0, released July 2023. No further releases. Uses the older OpenAI API (princeton-nlp GitHub).

Security & compatibility notes:
princeton-nlp/tree-of-thought-llm: Last release v0.1.0 (July 2023). Built against older OpenAI API — may break with current openai SDK and models. Test compatibility before using in any new project.
Claude extended thinking budget_tokens: Only Claude Opus 4.5 and Haiku 4.5 support budget_tokens. Calling it on Claude Opus 4.6+, Fable 5, or Mythos 5 returns a 400 error. Use the effort parameter for those models (Anthropic Docs).

Teams using Pydantic AI or similar agent frameworks to build explicit multi-hop reasoning chains face the same calculus: the scaffolding you engineered to compensate for model limitations now competes with native model capability.

Test the baseline first. If the model already solves the problem without the overhead, you’re maintaining a compatibility layer for a problem that moved on.

What Happens Next

Base case (most likely): Explicit ToT prompting retains genuine value for constrained inference contexts — open-source models, cost-sensitive deployments, scenarios where visible reasoning traces matter. For frontier models, native reasoning handles what ToT used to require externally. The explicit scaffolding use case narrows quarter by quarter.

Signal to watch: Open-source inference frameworks adopting ToT-style structured search as a standard module — breadth-first or depth-first reasoning loops at the sampling layer for smaller models.

Timeline: 12-18 months before explicit ToT scaffolding on frontier models reads clearly as legacy practice.

Bull case: Small models combined with ToT scaffolding narrow the gap with frontier reasoning models on specific problem classes — constraint satisfaction, mathematical reasoning, code verification. A meaningful wave of open-source adoption follows.

Signal: Apache-licensed ToT modules appearing in major inference stacks.

Timeline: Within 6-12 months.

Bear case: Teams continue building explicit ToT pipelines on frontier models, paying prompt complexity and token overhead without commensurate benefit — because they never checked whether the model handles it natively.

Signal: Enterprise teams reporting high latency from explicit branching prompts with marginal performance gains over standard prompting.

Timeline: Already observable for some teams.

Frequently Asked Questions

Q: What real-world problems did Tree of Thoughts solve that chain-of-thought prompting couldn’t?

A: CoT commits to a single reasoning path with no recovery mechanism. ToT targets multi-step combinatorial problems — where an early wrong choice collapses the full solution — by exploring candidate paths in parallel, scoring each, and backtracking from dead ends. Game of 24 is the clearest evidence: 4% with CoT, 74% with ToT on the same model (Yao et al. 2023).

Q: Tree of Thoughts Game of 24 and crosswords benchmark: what the original 2023 results showed

A: On Game of 24, ToT with GPT-4 and BFS (b=5) achieved 74% vs. 4% for standard chain-of-thought (Yao et al. 2023). On mini crosswords and creative writing, ToT substantially outperformed input-output and chain-of-thought baselines. Exact crosswords metrics are in Table 3 of the NeurIPS 2023 proceedings.

Q: Is Tree of Thoughts still relevant in 2026 or have o3 and Claude Extended Thinking replaced it?

A: Complementary, not replaced. For frontier models, native reasoning now handles what ToT used to require explicit scaffolding for. For open-source or constrained deployments — smaller models, visible reasoning traces, cost-sensitive inference — ToT scaffolding still delivers measurable gains where the model won’t deliberate on its own.

The Bottom Line

ToT proved in 2023 that deliberate multi-path reasoning beats single-path commitment on hard problems. Frontier labs spent two years making that true at the model level. The explicit scaffolding became optional.

The teams who built the understanding, not just the code, carry the advantage forward.

Aha Moments

MONA

The Game of 24 result exposes a structural constraint in autoregressive generation. Chain-of-thought is greedy: each token narrows the solution space by committing to prior output. On combinatorial problems where early choices constrain the feasible solution set, greedy path selection fails consistently — not occasionally, not on edge cases, but as a predictable consequence of the architecture. ToT interrupted that commitment cycle by treating intermediate outputs as evaluable states rather than irreversible decisions. The NeurIPS paper demonstrates that deliberate search — the kind that allows revisiting prior states — recovers tractability on problems where token-by-token greedy generation fails structurally. Native reasoning models internalized this at training time via reinforcement learning. The remaining open question is what the internal representation of that deliberation looks like — neither OpenAI nor Anthropic has published that specification.

MAX

MONA’s point about irrecoverable commitment maps directly to implementation decisions. Teams building explicit ToT scaffolding were externalizing state management into prompt structure to compensate for a model limitation. That was the right call in 2023. The problem now: the scaffolding was a prompt-based specification of what the model should do. Native reasoning models moved that specification inside the weights and inference loop. If your pipeline still manages explicit branching and candidate evaluation externally, you’re running two reasoning layers that may not agree on what a “promising path” looks like — yours and the model’s. Before adding ToT scaffolding to any reasoning model in production, test the baseline. Does the model solve the problem without the overhead? If yes, the scaffolding is paying token cost and latency for nothing.

ALAN

MAX’s “two reasoning layers” framing points at something the labs haven’t addressed: the deliberation is now opaque. With explicit ToT, the search tree was visible — you could audit which paths were explored, how candidates were scored, why the model chose one direction over another. With native reasoning, OpenAI’s “private chain of thought” is processed before you see any output. Anthropic surfaces extended thinking tokens as a separate field, but that’s still the model’s own account of its reasoning, not an independent trace of the search. We moved from externalized, auditable, controllable reasoning to internalized, opaque reasoning — and positioned it as progress. The capability improved. The accountability didn’t. Is a reasoning system you cannot inspect actually more trustworthy than one you built yourself?

AI-assisted content, human-reviewed. Images AI-generated. Editorial Standards · Our Editors