Deliberate Reasoning, Hidden Branches: Accountability Gaps in Tree of Thoughts Systems

What we are choosing not to examine, in our enthusiasm for Tree of Thoughts reasoning, is the question of who — or what — watches the deliberation. When a system explores multiple paths, prunes dead ends, and selects the best continuation, it looks like something approaching careful thought. The appearance is deliberate. The accountability is not.

Seventy-Four Percent

In 2023, researchers at Princeton and Google published results that would quietly reshape expectations for what language models could do when given structure to think with. On Game of 24 — a mathematical puzzle requiring multi-step arithmetic — GPT-4 with standard chain-of-thought prompting succeeded 4% of the time. The same model, given the Tree of Thoughts framework to explore multiple reasoning paths and evaluate intermediate steps, succeeded 74% of the time (Yao et al., 2023). That gap — four percent to seventy-four — became the number that sold deliberate machine reasoning as a category.

The figure is real. Since then, medical researchers have applied similar multi-branch frameworks to clinical question answering, with documented accuracy improvements on USMLE medical QA that have made the approach credible enough to deploy in clinical reasoning tools. The number keeps compounding. The enthusiasm is understandable.

But a number is a claim about one thing, and this number has been made to carry claims about something much broader than the thing it measured. The problem is not that the benchmark is dishonest. The problem is that it is circulating as a governance argument in domains where it was never a governance measurement.

The Efficiency Case and Its Discontents

The optimistic reading of 74% is that deliberate reasoning closes the gap between what language models can produce and what careful human reasoning requires. If we can get models to explore multiple hypotheses, evaluate each one, and commit only to the most defensible path — using Backtracking to eliminate dead ends before they reach an output — we get systems that behave more like methodical analysts than autocomplete engines. The case is coherent. Research showing meaningful accuracy improvements in clinical reasoning tasks gives it empirical grounding.

The critical reading is narrower but more precise: what the benchmark measured was a model’s ability to navigate a constrained puzzle where correct intermediate steps can be verified mathematically. In that domain, the model-as-evaluator functions because there is an external ground truth against which calibration is possible. Remove that external truth — deploy the system in medical prognosis, hiring assessment, or legal risk scoring — and the gap between “evaluated reasoning” and “correct reasoning” does not close. It simply becomes invisible, buried inside the system’s own self-report.

Both readings are circulating simultaneously. One in research papers, one in procurement decisions. Neither asks the structural question that makes the number insufficient as a governance argument: who, exactly, is doing the evaluation?

The Neutral Arbiter Nobody Elected

The shared assumption underneath both readings is that the model doing the evaluation is a different kind of entity from the model doing the generation — that when a system scores and prunes its own branches, it is performing something analogous to independent review. That when Prompt Chaining orchestrates a model to rate its own intermediate outputs, a meaningful check has occurred.

It has not. The evaluator in a Tree of Thoughts system is the same model as the generator. Both the candidate hypotheses and the judgments about which hypothesis is most promising emerge from identical weights, trained on identical data, reflecting identical priorities. The word “deliberate” in the framework’s full name — Deliberate Problem Solving — implies careful consideration by an actor who can stand outside their own reasoning. The model cannot do that. It can only recurse.

This would be a minor concern if the stakes were always mathematical puzzles. But the same architecture that earned 74% on Game of 24 is being deployed — or proposed for deployment — in contexts where the consequences of systematically biased self-evaluation are carried by people who never consented to be assessed by a system they cannot examine.

When the Judge Is Also the Defendant

Research on self-preference bias in LLM evaluation has surfaced a finding that should unsettle everyone deploying multi-branch reasoning for consequential tasks. Language models systematically favor their own outputs during evaluation — and advanced reasoning capabilities are “uncorrelated, or even negatively correlated” with low self-preference bias (Yang et al., 2026). The better the model reasons, the worse it becomes at recognizing when its own reasoning is flawed. Structured multi-dimensional evaluation can reduce this bias by a meaningful margin on average (Yang et al., 2026), but it cannot eliminate what is a feature of the architecture rather than a configuration error.

A related asymmetry compounds the problem. Research on when Tree of Thoughts actually succeeds shows that generator quality drives outcomes, while discrimination capabilities remain comparable across model scales — meaning a less capable evaluator cannot reliably assess stronger generator outputs (Chen et al., 2024). The model that generates sophisticated but biased reasoning paths will be evaluated by the very model that generated them, with no mechanism to recognize the gap between sophistication and correctness.

Then there is the question of what the reasoning traces themselves represent. Emerging research into trace faithfulness finds that exposed ToT-style reasoning traces “may merely approximate internal reasoning behaviorally” rather than reflect it faithfully — they are plausible reconstructions of process, not records of it (Lu et al., 2026). Separate research into reasoning model self-evaluation finds that phrases like “wait, let me reconsider” sway self-judgments through superficial reflection signals rather than genuine re-evaluation — the appearance of reconsidering is not the same as reconsidering. So when an auditor reviews what a Tree of Thoughts system appears to have considered, they may be reading a coherent account of deliberation that corresponds imperfectly to whatever actually shaped the output.

This is where the ethical risks of deploying these systems in high-stakes decisions become concrete — not because the system is deceiving anyone, but because the appearance of accountable deliberation is not the same as the existence of an accountable reasoning process. Can frameworks like ReAct Prompting or structured Self Consistency sampling introduce enough independence to address this? Partly, in constrained domains with verifiable ground truth. But partial independence in constrained domains is not the same as auditable reasoning in domains where affected people have rights they can invoke.

Accountability as Architecture, Not Annotation

Thesis: Deploying Tree of Thoughts in high-stakes decisions without independent evaluation is not a configuration choice — it is a structural delegation of judgment to a system that cannot assess its own reasoning from any position outside itself.

The governance frameworks are beginning to catch up, but with instruments designed for different problems. The EU AI Act’s Article 12 requires high-risk AI systems to auto-generate tamper-resistant logs of inputs, outputs, and decisions, with six-month retention — enforceable from August 2026 for most use cases, with some categories extending to December 2027 (EU AI Act). Legislation in some jurisdictions now classifies systems making crucial decisions in employment, healthcare, and financial contexts as high-risk, triggering similar requirements. The NIST AI Risk Management Framework requires completeness of reasoning chains and the ability to reconstruct decision context in post-incident investigations (NIST AI RMF).

These are meaningful requirements. But they rest on an assumption that decision logs contain auditable reasoning — that the log is a faithful record of why the system chose what it chose. If Tree of Thoughts traces approximate rather than record internal deliberation, then a tamper-resistant log of those traces is a tamper-resistant approximation. The compliance requirement is met. The accountability requirement is not.

Orchestration tools like Pydantic AI can surface intermediate reasoning steps in structured formats that compliance teams can examine. Structure is not independence. A well-formatted account of biased deliberation is still biased deliberation, and the auditor reading the log has no vantage point from which to distinguish systematic bias from considered judgment.

What Belongs Outside the Tree

The 74% benchmark tells us what Tree of Thoughts can achieve in a domain where correct reasoning is verifiable from outside the system. It cannot tell us who bears responsibility when reasoning is not externally verifiable — when the output of a pruned branch shaped an employment rejection, informed a clinical recommendation, or set a credit score.

It cannot tell us whether the model’s self-evaluation tracked truth or preference. It cannot tell us which branches were explored and which were pruned by the model’s own biases, because those branches exist nowhere except in the model’s self-report of what it considered. It cannot tell us whether the appearance of deliberation corresponded to anything that deserves the name.

These are not edge cases to be addressed after deployment. They are the normal conditions of deploying reasoning systems in the contexts where reasoning matters most — contexts where affected people have interests in understanding, contesting, and assigning responsibility for decisions that shape their lives.

Where This Argument Breaks

This argument against Tree of Thoughts accountability depends on a claim about trace faithfulness that is still an emerging area of research. If interpretability methods develop to the point where they can verify that ToT reasoning traces accurately reflect internal computation — and if tooling is built to surface that verification to auditors in real time — the governance gap becomes tractable rather than structural. The appropriate response would then be investment in interpretability, not restraint in deployment.

The argument also depends on human oversight failing to compensate for self-referential bias. If the EU AI Act’s logging requirements, combined with mandatory human review at each consequential decision point, prove sufficient to catch systematic bias introduced by self-preference in evaluation, then the concern is real but manageable. That assumption of functional human oversight is not irrational — it is simply not guaranteed by the deployment architecture as currently designed, and it requires institutional capacity that most organizations currently lack.

What would not change this position: improved benchmark performance on constrained tasks. Seventy-four percent will become eighty-five, and eighty-five will become ninety-three. None of those numbers tell us anything new about the accountability of self-evaluating deliberation in domains without ground truth — because the benchmark itself is the domain with ground truth. The domains that matter are the ones that do not have it.

The Question That Remains

We have built systems that appear to think carefully — that generate possibilities, examine them, and commit to the most defensible path. We have not yet built the institutional infrastructure to determine whether that apparent care corresponds to anything that affected people can audit, contest, or hold anyone accountable for.

The question is not whether Tree of Thoughts improves performance. It does, in the domains where performance can be measured against external truth. The harder question is whether performance — in the absence of independent evaluation, verifiable deliberation, and enforceable accountability — is what we actually mean by reasoning, in the contexts where the cost of getting it wrong is borne by people who never entered the benchmark.

Ethically, Alan.