Tool Calling Schema

A tool calling schema is a JSON-based definition that tells a language model which functions it can call, what parameters each function accepts, and what data types those parameters require.

What It Is

When you ask an AI assistant to check the weather or book a calendar event, something has to tell the model what capabilities exist, what information each capability needs, and in what format. That is the job of a tool calling schema.

A tool calling schema is a machine-readable declaration—written in JSON Schema—that describes a set of functions the model is permitted to call. It lists each function’s name, a short description of what it does, and the parameters the function expects: their names, types (string, integer, boolean, array), and which are required. The model reads this declaration as part of its prompt and uses it to construct valid function calls when generating a response.

Think of it as a contract between the application and the model. The application hands the model a menu of available actions; the model orders from that menu by name, spelling out the arguments exactly as the schema specifies.

The schema lives inside the prompt—either injected automatically by the API or embedded explicitly by the developer. Modern APIs accept tool definitions in a structured format and handle schema injection internally. When the model decides a function call is the right response, it outputs a structured call object that the application receives and executes.

Three components make up every tool calling schema:

• Function name: the identifier the model uses to call the function (e.g., get_current_weather)
• Description: a plain-language explanation of what the function does—this is what the model reads to decide when to call it
• Parameters object: a JSON Schema block describing accepted inputs, their types, and which are required

The description field deserves special attention. The model does not see your code—it only sees the description. A vague or missing description means the model has to infer when to call the function, and it will often guess wrong. In the context of how LLMs parse function calling schemas, the description is the primary signal the model uses to match user intent to a function.

How It’s Used in Practice

The most common encounter with tool calling schemas is in chat-based application development. When building a feature where users can ask questions and the app needs to fetch live data—order status, product prices, database records—the developer defines functions that retrieve that data, writes a schema for each, and passes them to the model via the API’s tools parameter. The model decides when to call a function, returns a structured call object with the chosen function name and arguments, and the application executes it and feeds the result back into the conversation.

Libraries like Instructor and BAML add a layer above the raw schema, generating it automatically from Python type annotations or function signatures. This reduces hand-written JSON, but the underlying structure the model reads is identical.

Pro Tip: Put the real work into the description field, not just the parameter names. If you have two similar functions—say, search_products and search_inventory—write descriptions that make the distinction explicit and concrete. The model picks based on descriptions, and subtle overlaps in wording reliably produce the wrong call in edge cases.

When to Use / When Not

Common Misconception

Myth: The tool calling schema teaches the model how to execute code—so the model is actually running the function when it calls it.

Reality: The model never executes anything. The schema tells it what functions exist and what structure a valid call must have. The model outputs a call object; your application code receives that object and runs the actual function. The model’s role ends at generating the call.

One Sentence to Remember

A tool calling schema is the JSON contract that turns a model’s text output into a reliable, parseable function call—it defines the vocabulary the application and the model share so each side knows what the other expects.

FAQ

Q: Can I pass multiple tool schemas in one API call? A: Yes. Most LLM APIs accept an array of tool definitions. The model picks which function to call based on the descriptions. Keep the set to roughly 20 or fewer tools per call to avoid degraded selection accuracy.

Q: What happens if the model calls a function that is not in the schema? A: A well-implemented API constrains model output to declared function names, so unlisted functions cannot be called. The schema enforced at inference time is one reason structured tool calls are more reliable than asking the model to format calls in plain text.

Q: Do I need to write JSON Schema syntax by hand? A: Not always. Libraries like Instructor, BAML, and Pydantic can generate the JSON Schema from annotated Python classes or typed function signatures. The API still receives JSON Schema under the hood—the library just spares you from writing it directly.