OpenAI Function Calling Spec: Making AI Actually Do Things

The model is smart. It can write essays, solve math, explain quantum physics. But it can't check the weather. Can't query your database. Can't send an email.

Until you give it tools.

OpenAI's function calling spec is the most widely adopted standard for giving AI models the ability to act in the real world. Anthropic, Google, Mistral, open-source frameworks — everyone supports it or something compatible with it.

It's not complicated. It's a JSON schema that says "here are the functions you can call, here's what they expect." But the details — tool_choice, parallel calls, streaming, error handling — are where the production nuances live.

How Function Calling Works

The flow is simple:

1. You send a message + tool definitions to the model
2. Model decides if it needs a tool
3. If yes: model returns a tool_call (function name + arguments)
4. You execute the function locally
5. You send the result back to the model
6. Model incorporates the result and responds to the user

The model never executes functions. It just generates the function name and JSON arguments. You run the code.

┌──────┐ tools + message ┌───────┐

│ Your │ ──────────────────→ │ Model │

│ Code │ │ │

│ │ tool_call │ │

│ │ ←────────────────── │ │

│ │ │ │

│ │ [execute locally] │ │

│ │ │ │

│ │ tool result │ │

│ │ ──────────────────→ │ │

│ │ │ │

│ │ final response │ │

│ │ ←────────────────── │ │

└──────┘ └───────┘

Defining Tools: The JSON Schema

Each tool is defined with a name, description, and parameter schema:

python

import openai

client = openai.OpenAI()
tools = [
{
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a location. Use when the user asks about weather, temperature, or conditions.",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "City and country, e.g., 'Mumbai, India'"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "Temperature unit"
                    }
                },
                "required": ["location"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "search_products",
            "description": "Search the product catalog. Use when user asks about products, prices, or availability.",
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {
                        "type": "string",
                        "description": "Search query"
                    },
                    "category": {
                        "type": "string",
                        "enum": ["electronics", "clothing", "books", "food"],
                        "description": "Filter by category"
                    },
                    "max_price": {
                        "type": "number",
                        "description": "Maximum price filter"
                    },
                    "in_stock": {
                        "type": "boolean",
                        "description": "Only show in-stock items"
                    }
                },
                "required": ["query"]
            }
        }
    }

]

Schema Tips That Matter

The description fields are critical. The model reads them to decide when and how to use the tool:

Field Purpose Impact

python

Bad — model doesn't know when to use it

"description": "Gets data"

Good — model knows exactly when to use it

"description": "Get current weather conditions for a specific city. Use when the user asks about weather, temperature, rain, or forecasts. Returns temperature, humidity, and conditions."

🔧

The Complete Request/Response Cycle

python

Step 1: Send message with tools

response = client.chat.completions.create(
model="gpt-4o",
    messages=[
        {"role": "system", "content": "You are a helpful shopping assistant."},
        {"role": "user", "content": "What's the weather in Mumbai and find me cheap headphones?"}
    ],
    tools=tools,
    tool_choice="auto"  # Let the model decide

)

Step 2: Check if model wants to call tools

message = response.choices[0].message

if message.tool_calls:

print(f"Model wants to call {len(message.tool_calls)} tool(s):")
    for tc in message.tool_calls:
        print(f"  - {tc.function.name}({tc.function.arguments})")

Output:

Model wants to call 2 tool(s):

- get_weather({"location": "Mumbai, India", "unit": "celsius"})

- search_products({"query": "headphones", "max_price": 2000, "in_stock": true})

python

Step 3: Execute functions and send results back

import json

Add the assistant message (with tool_calls) to conversation

messages = [
{"role": "system", "content": "You are a helpful shopping assistant."},
    {"role": "user", "content": "What's the weather in Mumbai and find me cheap headphones?"},
    message,  # Assistant message with tool_calls

]

Execute each tool call and add results

for tool_call in message.tool_calls:

if tool_call.function.name == "get_weather":
        args = json.loads(tool_call.function.arguments)
        result = get_weather(**args)  # Your function
    elif tool_call.function.name == "search_products":
        args = json.loads(tool_call.function.arguments)
        result = search_products(**args)  # Your function

messages.append({
        "role": "tool",
        "tool_call_id": tool_call.id,  # Must match the tool_call ID
        "content": json.dumps(result)
    })

Step 4: Get final response with tool results incorporated

final_response = client.chat.completions.create(
model="gpt-4o",
    messages=messages,
    tools=tools

)

print(final_response.choices[0].message.content)

"It's 32°C and sunny in Mumbai! As for headphones, here are some options under ₹2000..."

📊

tool_choice: Controlling When Tools Are Called

The tool_choice parameter controls whether and which tools the model uses:

Value Behavior When to Use

python

Force the model to always use a specific tool

response = client.chat.completions.create(
model="gpt-4o",
    messages=[{"role": "user", "content": "Tell me about Paris"}],
    tools=tools,
    tool_choice={
        "type": "function",
        "function": {"name": "get_weather"}
    }
    # Model WILL call get_weather, even if the user didn't ask about weather

)

When to Use Each

python

Chatbot with optional tools → "auto"

tool_choice = "auto"

Data extraction pipeline (always extract) → "required"

tool_choice = "required"

Follow-up after tool results (just generate text) → "none"

tool_choice = "none"

Router that MUST classify → force specific tool

tool_choice = {"type": "function", "function": {"name": "classify_intent"}}

Parallel Tool Calls

Models can request multiple tool calls in a single response. This is huge for latency:

python

User: "Compare weather in Mumbai, Delhi, and Bangalore"

Model returns 3 parallel tool calls:

tool_call_1: get_weather({"location": "Mumbai, India"})

tool_call_2: get_weather({"location": "Delhi, India"})

tool_call_3: get_weather({"location": "Bangalore, India"})

You can execute all 3 in parallel!

import asyncio

async def execute_parallel(tool_calls):

tasks = []
    for tc in tool_calls:
        args = json.loads(tc.function.arguments)
        tasks.append(async_get_weather(**args))

results = await asyncio.gather(*tasks)
    return results

This turns 3 sequential API calls into 1 parallel batch

Disabling Parallel Calls

Sometimes you want sequential execution (e.g., tool 2 depends on tool 1's result):

python

response = client.chat.completions.create(
model="gpt-4o",
    messages=messages,
    tools=tools,
    parallel_tool_calls=False  # Force sequential tool calls

)

Streaming with Tool Calls

When streaming, tool calls arrive in chunks:

python

stream = client.chat.completions.create(
model="gpt-4o",
    messages=[{"role": "user", "content": "What's the weather in Mumbai?"}],
    tools=tools,
    stream=True

)

tool_calls = {}

for chunk in stream:

delta = chunk.choices[0].delta

if delta.tool_calls:
        for tc in delta.tool_calls:
            idx = tc.index
            if idx not in tool_calls:
                tool_calls[idx] = {"id": tc.id, "name": tc.function.name, "arguments": ""}
            if tc.function.arguments:
                tool_calls[idx]["arguments"] += tc.function.arguments

tool_calls[0]["arguments"] now has the complete JSON string

Arrives in chunks: '{"lo' → 'cation' → '": "Mu' → 'mbai, ' → 'India"}'

🔬

Error Handling: When Tools Fail

What happens when your tool throws an error? Send it back as the tool result — the model can handle it gracefully:

python

for tool_call in message.tool_calls:

try:
        args = json.loads(tool_call.function.arguments)
        result = execute_tool(tool_call.function.name, args)
        content = json.dumps(result)
    except Exception as e:
        # Send error as tool result — model will explain to user
        content = json.dumps({
            "error": str(e),
            "message": "Tool execution failed"
        })

messages.append({
        "role": "tool",
        "tool_call_id": tool_call.id,
        "content": content
    })

Model sees the error and responds:

"I wasn't able to check the weather right now — the service seems to be down.

Can I help with something else?"

🛡️

Cross-Provider Compatibility

The OpenAI function calling format has become the de facto standard:

Provider Compatibility Notes

OpenAI Native The original spec

python

LiteLLM: same function calling code, any provider

from litellm import completion

Works with OpenAI

response = completion(model="gpt-4o", messages=messages, tools=tools)

Same code works with Anthropic

response = completion(model="claude-sonnet-4-5-20250514", messages=messages, tools=tools)

And Mistral

response = completion(model="mistral/mistral-large", messages=messages, tools=tools)

📦

Best Practices

1. Keep Tool Descriptions Precise

python

❌ Vague — model guesses

"description": "Get info"

✅ Specific — model knows exactly when and how

"description": "Get current stock price and daily change for a US-listed ticker symbol. Returns price in USD, daily change percentage, and volume. Use when user asks about stock prices, market data, or portfolio values."

1. Limit Tool Count

python

❌ 50 tools — model gets confused, picks wrong ones

tools = [tool_1, tool_2, ..., tool_50]

✅ 5-10 relevant tools — model picks correctly

tools = select_relevant_tools(user_message, all_tools)  # Pre-filter

1. Validate Arguments

python

The model generates JSON, but it can be wrong

args = json.loads(tool_call.function.arguments)

Always validate before executing

if tool_call.function.name == "delete_user":

if "user_id" not in args:
        raise ValueError("user_id required")
    if not isinstance(args["user_id"], int):
        raise ValueError("user_id must be integer")
    if args["user_id"] <= 0:
        raise ValueError("user_id must be positive")

1. Use Structured Output for Reliability

python

For critical tool calls, use strict mode

tools = [{
"type": "function",
    "function": {
        "name": "extract_order",
        "description": "Extract order details from user message",
        "parameters": {
            "type": "object",
            "properties": {
                "product": {"type": "string"},
                "quantity": {"type": "integer"},
                "shipping": {"type": "string", "enum": ["standard", "express"]}
            },
            "required": ["product", "quantity", "shipping"],
            "additionalProperties": False  # Strict — no extra fields
        },
        "strict": True  # Guaranteed to match schema
    }

}]

🚀

Practical Takeaways

Function calling = model generates JSON, you execute the code — the model never runs functions

Descriptions are everything — they determine when and how the model uses each tool

tool_choice controls behavior — auto, none, required, or force a specific tool

Parallel tool calls save latency — execute independent calls concurrently

Send errors back as tool results — the model handles failure gracefully

The OpenAI format is the de facto standard — every major provider supports it or something compatible

🔄

What's Next?

Episode 80: Structured Output / JSON Mode — Function calling gives models tools. But what about when you need the model's response to be valid JSON? Guaranteed. Every time. No "here's a JSON block in markdown." We'll explore JSON mode, strict schemas, and grammar-based constrained generation.

← Previous

Ep 78: MCP (Model Context Protocol)

Next →

Ep 80: Structured Output / JSON Mode

Next: Episode 80 — Structured Output / JSON Mode

Your JSON.parse() explodes. The model wrapped valid JSON inside markdown. This breaks every pipeline. Here's the fix.

This is part of a 98-episode series covering AI engineering from tokens to production deployment.