Tool Use in Prompts

What tool use in prompts is

Most AI applications need to do more than answer questions — they need to look up live prices, query databases, send emails, or call external APIs. Without tool use, you would have to parse free-form text and guess what the model intended. Tool use in prompts solves this by giving the model an explicit list of callable functions, and by making it respond with a structured call rather than prose.

The mechanism works like this: before the model sees the user’s message, the application inserts a block of tool schemas alongside the system prompt. Each schema has three parts — a name (the function identifier), a description (a plain-language explanation of what the function does and when to call it), and a parameters block (a JSON Schema object defining what arguments the function accepts, their types, and which are required). The model reads these schemas as part of the prompt. When the user’s request matches a tool’s purpose, the model responds not with text but with a structured output: a function name paired with argument values that conform to the parameter schema.

Think of it like a set of pre-printed request forms at a help desk. The desk clerk (the application) keeps a stack of approved forms: “Check Account Balance,” “Schedule Appointment,” “Update Contact Record.” A visitor (the user) describes what they need in plain language. The desk agent (the model) decides which form applies, fills in the correct values, and hands it back. The clerk stamps it and takes the action. The agent never opened a drawer — they only wrote on the form.

This separation matters. The model does not execute anything. It produces a structured representation of intent: which function to call and with which arguments. The calling code is what runs the function, handles errors, and decides whether to return the result to the model for further reasoning. Understanding how a model parses these schemas — which parts of the description it reads most closely, how parameter names influence argument generation, and where schema complexity hurts accuracy — is exactly the question the parent article addresses.

How tool use in prompts is used in practice

The most common encounter is AI chat assistants connected to real-time data. Search assistants, customer support bots that check order status, personal assistants that query your calendar — these work by adding tool schemas to the system prompt. When you ask “Has my package shipped?”, the model outputs a call to get_shipment_status with your order number as the argument. The application runs the query, gets a result, and passes it back to the model, which writes the human-readable reply.

A second use case is workflow automation: applications that chain several tool calls in sequence. The model might call search_knowledge_base, then summarize_text, then send_email. The application acts as the loop — executing each tool, returning the result, letting the model decide the next step.

Tool descriptions are read by the model the same way any other text in the prompt is. Write them as you would write an API docstring for a new developer: say what the function does, what each parameter means, what the function returns, and what constraints apply. Vague descriptions like “processes the request” produce unreliable calls. The parameter name matters too — a field named user_id signals different expectations to the model than one named customer_identifier, even when both carry identical descriptions.

The tool-use reading path: mechanism, limits, then the stakes

Start with how LLMs parse function calling schemas — it fixes what tool use actually is at the API level, a structured JSON request rather than magic, which is the misunderstanding behind most of what follows. Read the hard limits of LLM tool calling right after: it catalogs what breaks between the demo and production — context consumed by schema definitions, parameters filled with confidently wrong values — before you’ve built anything to blame it on.

When you’re ready to build, the tool-description and pipeline guide treats the description field as the model’s actual decision spec, not documentation, and covers the message-ordering rules that throw errors if you get them wrong. For the model-selection question that guide doesn’t answer, the BFCL v3 benchmark comparison shows why the leaderboard you check determines which failures you inherit. Close with the accountability questions raised when LLMs call external APIs — once the pipeline works, this is the read for before you widen what it’s allowed to touch.

How tool use differs from prompt injection and prompt optimization

Two neighbours get folded into this topic, and each conflation hides a different problem.

Tool use is the capability that lets a model act; prompt injection is one way that capability gets hijacked into acting on an attacker’s behalf. Turning on tool calling doesn’t create new injection surface at the model level — the model was already reading attacker-controlled text — but it converts a text-generation mistake into a taken action, which is why injection defenses and tool-scoping decisions have to be designed together, not bolted on in sequence.

Tool use is also not the same problem as prompt optimization. Automated optimizers tune a prompt against a text-quality or task-success score; they were never built to judge whether a model picked the right tool with the right arguments. That judgment needs function-calling-specific benchmarks — the BFCL-versus-tau-bench split above — not a general eval number.

When tool use in prompts fits — and when it doesn't

Common questions about tool use in prompts

Q: Should I expose every tool to the model in every prompt, or scope by task? A: Scope them. Loading dozens of tool schemas into one context burns tokens and gives the model more chances to pick the wrong one; the tool-description guide recommends scoping tools to the current task and constraining choice with tool_choice subsets instead of loading everything at once.

Q: Which function-calling benchmark should I trust when picking a model for my agent? A: No single one — BFCL v3 and tau-bench disagree on the winner because they measure different things: schema accuracy versus multi-step agentic reliability. Pick the benchmark that matches your actual failure mode, not the one with the highest score for your candidate model.

Q: Does adding tool use increase my exposure to prompt injection? A: It raises the stakes without changing the surface. A model already reads attacker-controlled text before it ever calls a tool, so the injection risk exists either way — tool use just turns a hijacked response into a hijacked action, which is why scoping and permission limits matter more once tools are live.

Q: Who is accountable when a tool call does the wrong thing in production? A: Usually nobody cleanly — the accountability gap around LLMs calling external APIs is structural, not a policy oversight: once a model chooses among tools autonomously, tracing the decision back to a responsible human gets mathematically harder, not just organizationally harder.

Part of the prompt ops and security theme · closest neighbour: prompt injection.