DAN Analysis 9 min read September 14, 2026

GLM 4.5, Qwen3, and Claude on BFCL v3: What Function Calling Benchmarks Miss in 2026

Function calling benchmark split showing different model leaders on schema accuracy versus agentic reliability metrics in

TL;DR

The shift: BFCL v3 and tau-bench now produce different winners, exposing a structural divide between schema accuracy and agentic reliability
Why it matters: The benchmark you use to select a model determines which production failures you’ll own
What’s next: BFCL V4 (live since July 2025) already reweights toward agentic tasks — and only 13 models have been submitted so far

GLM 4.5 sits at #1 on BFCL v3 with a 0.778 score, per llm-stats.com. Claude Sonnet 4.5 isn’t in the top 19 tracked models. Switch to tau-bench Retail and Sonnet 4.5 takes #1 at 0.862 — while GLM 4.5 falls to #6.

That’s not a ranking quirk. That’s two benchmarks measuring two fundamentally different engineering problems.

The benchmark you cite to justify your model selection is quietly determining which production failures you’ll spend next quarter explaining.

The Two-Leaderboard Problem

Thesis: The BFCL v3–tau-bench split isn’t a measurement artifact — it’s the field correctly identifying that schema accuracy and agentic reliability are separate optimization targets, and that excelling at one does not guarantee the other.

BFCL v3 tests Tool Use in Prompts precision under controlled conditions: extract the right arguments, match the correct function signature, adhere to the Tool Calling Schema. Single-turn. Synthetic tasks. No user in the loop.

tau-bench tests something closer to what your production agent actually does. Sustained policy adherence across multi-turn conversations, real user inputs, tool sequences spanning dozens of exchanges.

Different test. Different winner. Different engineering problem.

Teams selecting agentic models by BFCL v3 rank alone are calibrating on the wrong instrument. The Prompt Testing And Evaluation dimension that actually predicts production performance — multi-turn reliability under adversarial conditions — doesn’t appear on the single-turn leaderboard.

GLM 4.5, Qwen3, and Claude: What the Scores Actually Show

GLM 4.5, Zhipu AI’s mixture-of-experts model (355B total / 32B active parameters, Z.AI Docs), leads BFCL v3 at 0.778 as of June 2026. Its lighter sibling, GLM-4.5-Air (106B total / 12B active), holds #2 at 0.764. Thinking mode is enabled by default via the thinking.type parameter.

Qwen3-Next-80B-A3B-Thinking ties for #4 at 0.720. The Qwen team recommends Hermes-style tool use formatting for this model family (Qwen Docs).

Claude Sonnet 4.5, released September 2025, hasn’t been submitted to the BFCL v3 tracked cohort. That’s a deliberate posture, not an omission.

On tau-bench Retail — 25 evaluated models, multi-turn agentic scenarios — Sonnet 4.5 scores 0.862 (#1), per llm-stats.com. On tau-bench Airline (23 models), 0.700 and #1. GLM 4.5 scores 0.797 on tau-bench Retail, falling to #6. Native Parallel Tool Calling capability and a SWE-bench Verified score of 77.2% solo (Anthropic Blog) help explain why Sonnet 4.5’s performance profile looks so different across benchmark types.

The number the market is underweighting: BFCL V4 launched July 17, 2025. The new scoring formula weights Agentic tasks at 40%, Multi-Turn at 30%, Live at 10% (Berkeley BFCL). As of June 2026, only 13 models have submitted — Qwen3.7 Max leads at 0.750. No GLM 4.5. No Claude.

That vacuum tells its own story.

The Teams That Get This Right

Schema-intensive, single-turn pipelines — Structured Output Prompting for classification, data extraction, API routing — have a clear choice. GLM 4.5 and Qwen3 variants deliver the best verified JSON Schema adherence available at their tier. Teams using Instructor or applying Constrained Decoding techniques to enforce output contracts will find these models perform precisely where the benchmark says they do.

Agentic builders have an equally clear signal. Multi-turn agents that need to maintain policy across user interruptions, handle tool failures gracefully, and stay coherent across long exchanges point directly at Sonnet 4.5’s tau-bench results. At $3/$15 per million tokens input/output (Anthropic Blog), the cost is calculable. The reliability margin is harder to put a number on — until something breaks at 2 a.m.

The teams winning in 2026 matched the benchmark to the workload first, then selected the model. Not the reverse.

The Teams That Get Left Behind

Any team that equates BFCL v3 rank with production readiness.

Datadog’s State of AI Engineering shows a 2% error rate across all LLM call spans as of March 2026, down from 5% in February. Progress — but the residual failures concentrate in complex, multi-step tool interactions. Exactly the domain BFCL v3 doesn’t test.

Multi-agent systems fail at rates between 41% and 86.7% in production, across 14 failure modes in the MAST taxonomy (Augment Code). Prompt Injection — OWASP’s #1 LLM risk — dominates the agentic attack surface. Schema drift from version upgrades. State management failures. Policy drift across turns.

The n8n incident is instructive. An upgrade from v2.4.7 to v2.6.3 broke Vector Store tool JSON schema compatibility, causing both OpenAI and Anthropic APIs to reject requests. The affected model’s BFCL rank was irrelevant. Real-world schema drift doesn’t surface in controlled benchmark conditions.

Teams spending cycles on Prompt Optimization against static test sets while ignoring injection surfaces and schema versioning are solving 2024’s failure mode in 2026’s production environment.

What Happens Next

Base case (most likely): BFCL V4 gradually becomes the primary selection standard as submission volume grows. Qwen3 variants maintain top positions in schema-accuracy tasks. Sonnet 4.5 and successors hold the agentic reliability lead. The two-benchmark split persists — teams that understand the difference self-select correctly; those that don’t pick the wrong model for their workload and discover the gap in production. Signal to watch: GLM 4.5 successors (GLM-4.7 and GLM-5.2, both shipping as of June 2026 per MarkTechPost) submit to BFCL V4 and tau-bench simultaneously, making a cross-benchmark comparison possible. Timeline: Q3–Q4 2026.

Bull case: tau-bench and MCP-Bench — Accenture Labs’ 28-server, 250-tool production-realistic eval — converge into a unified agentic readiness standard. Labs begin optimizing for sustained reliability over discrete accuracy. BFCL V4’s agentic weighting is the early signal pointing in this direction. Signal: A model in the top five on BFCL V4 also ranks top three on tau-bench. Timeline: 2027.

Bear case: Labs game BFCL V4 the way they gamed V3. Synthetic task over-fitting, benchmark contamination, scores that don’t transfer to real workloads. Enterprise teams start building private eval suites because public leaderboards no longer predict outcomes. Signal: V4 top-three models underperform MCP-Bench by a wide margin after broader submission. Timeline: Visible by end of 2026 if V4 cohort diversity stays low.

Frequently Asked Questions

Q: How does Claude Sonnet 4.5 score against GLM 4.5 Thinking on the BFCL v3 function calling leaderboard? A: GLM 4.5 leads the BFCL v3 tracked cohort at 0.778 (#1 of 19 models, llm-stats.com). Claude Sonnet 4.5 has not been submitted to that leaderboard. On tau-bench Retail, the rankings invert: Sonnet 4.5 scores 0.862 (#1); GLM 4.5 scores 0.797 (#6).

Q: What are documented failure modes when LLM tool calling breaks in production agentic systems? A: The MAST taxonomy identifies 14 multi-agent failure modes. Top categories include schema drift from version upgrades, state management failures, policy drift across turns, and prompt injection attacks. Datadog’s March 2026 data shows rate limiting accounts for roughly a third of LLM call errors in production, down from more than half in February.

Q: Where is LLM function calling headed in 2026 after BFCL v3 and tau-bench agentic results? A: BFCL V4 (live since July 2025) shifts scoring toward agentic and multi-turn tasks at 70% combined weight. Qwen3.7 Max leads the early cohort. The direction is toward production-realistic evaluation. Schema accuracy alone is no longer a sufficient model selection signal for agentic systems.

The Bottom Line

Two benchmarks. Two winners. One measures what the benchmark committee thought mattered when the benchmark launched. The other measures what’s actually breaking in production systems today.

You’re either picking the right benchmark for your workload or you’re picking the wrong model for your system.

Sources

BFCL Changelog: gorilla/berkeley-function-call-leaderboard/CHANGELOG.md - V3 launch date (September 2024) and V4 launch date (July 2025)
Berkeley BFCL: Berkeley Function Calling Leaderboard V4 - V4 scoring formula (Agentic 40%, Multi-Turn 30%, Live 10%)
llm-stats.com: BFCL-v3 Leaderboard - GLM 4.5, Qwen3, and full model rankings as of June 2026
llm-stats.com: TAU-bench Retail Leaderboard - Claude Sonnet 4.5 and GLM 4.5 multi-turn agentic scores
Anthropic Blog: Introducing Claude Sonnet 4.5 - Pricing, SWE-bench Verified score, parallel tool execution
Z.AI Docs: GLM-4.5 Developer Documentation - GLM-4.5 architecture, parameter counts, and thinking mode
Datadog: State of AI Engineering (March 2026) - Production LLM error rates and rate limiting breakdown
Augment Code: Why Multi-Agent LLM Systems Fail - MAST taxonomy and production multi-agent failure rates

Aha Moments

MONA

BFCL v3 and tau-bench test genuinely distinct properties. BFCL v3 measures schema extraction fidelity under controlled, single-turn conditions. tau-bench measures policy adherence under stochastic multi-turn pressure with real user inputs. The ranking divergence isn’t surprising once you examine what each benchmark operationalizes — the optimization that makes a model excellent at argument extraction doesn’t directly transfer to maintaining state coherence across extended tool-use chains. These are separate targets. The benchmark split is the field correctly diagnosing two different engineering problems, not a measurement quality failure. The real question is whether the evaluation community can formalize a composite that captures both dimensions before production deployments expose the gap through incidents rather than leaderboard data.

MAX

MONA’s distinction holds — and it has a direct consequence for how teams write context specifications. The brief for a schema-extraction pipeline looks nothing like the brief for a multi-turn agent. One needs exact contract enforcement on the output format. The other needs explicit policy memory, error recovery paths, and turn-boundary handling designed into the context architecture. The models that top each leaderboard succeeded because they were optimized for fundamentally different context structures. Selecting a model without first specifying what your context architecture actually requires — which tools, how many turns, what failure recovery looks like — isn’t model selection. It’s a guess with a benchmark number attached to make it look principled.

ALAN

Both of you are right about the measurement gap — but notice what neither benchmark makes visible. The MAST taxonomy names the failure modes. The leaderboard scores name the performance leaders. What neither surfaces is who bears the cost when agentic tool-calling fails at scale: misrouted transactions, injection-poisoned chains, or state drift accumulating silently until a user is harmed. The governance layer for agentic tool use is almost entirely absent from the benchmark framing. When the gap between controlled evaluation and production deployment is the site of the harm, does the benchmark carry any obligation to make that gap visible — or does accountability disappear into the space between what was measured and what happened?

AI-assisted content, human-reviewed. Images AI-generated. Editorial Standards · Our Editors