The OpenAI API Spec: The Standard Everyone Follows

Google has Gemini. Anthropic has Claude. Meta has Llama. Mistral, Cohere, DeepSeek, Qwen — dozens of model providers.

Every single one of them supports the OpenAI API format.

Not because OpenAI is the best. Not because they're required to. But because OpenAI shipped first, developers adopted it, and now it's the de facto standard. Like how USB-C won — not by being technically perfect, but by being everywhere.

If you understand the OpenAI chat completions API, you understand how to talk to virtually every LLM on the planet. Let's dissect it.

The Chat Completions Endpoint

Everything centers on one endpoint:

POST https://api.openai.com/v1/chat/completions

That's it. One endpoint. You send messages, you get a completion. Every feature — tools, streaming, JSON mode, vision, structured outputs — layers on top of this single endpoint.

python

import openai

client = openai.OpenAI()
response = client.chat.completions.create(
model="gpt-4o",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is the capital of France?"}
    ]

)

print(response.choices[0].message.content)

"The capital of France is Paris."

The Messages Format

The core of the API is the messages array — an ordered list of messages that represent the conversation. Each message has a role and content:

The Four Roles

Role Purpose Who writes it

python

messages = [
# System: Developer instructions (invisible to user)
    {
        "role": "system",
        "content": "You are a concise, technical AI assistant. "
                   "Answer in 2-3 sentences max."
    },
# User: What the human typed
    {
        "role": "user",
        "content": "Explain transformers in AI"
    },
# Assistant: What the model responded (from a previous turn)
    {
        "role": "assistant",
        "content": "Transformers use self-attention to process all tokens "
                   "in parallel, unlike RNNs which go sequentially."
    },
# User: Follow-up
    {
        "role": "user",
        "content": "What's self-attention?"
    }

]

Why Messages, Not a Single Prompt?

The old completions API (v1) took a single text string. The chat completions API uses structured messages because:

Multi-turn conversations — each message has a clear role and boundary

System prompts — developers can set behavior without users seeing/overriding it

Tool results — tool outputs go in tool messages, keeping the flow clean

History management — easy to truncate old messages to fit the context window

🔧

The Request Object

Here's the full anatomy of a chat completions request:

python

response = client.chat.completions.create(
# Required
    model="gpt-4o",                    # Which model
    messages=[...],                     # Conversation history
# Common optional
    temperature=0.7,                    # Randomness (0 = deterministic, 2 = creative)
    max_tokens=1000,                    # Max output length
    top_p=1.0,                          # Nucleus sampling (alternative to temperature)
# Tool calling
    tools=[...],                        # Available tools
    tool_choice="auto",                 # "auto", "none", or force specific tool
# Output format
    response_format={"type": "json_object"},  # Force JSON output
# Streaming
    stream=True,                        # Get tokens as they generate
# Advanced
    stop=["\n\n", "END"],               # Stop sequences
    n=1,                                # Number of completions to generate
    seed=42,                            # Reproducibility (best-effort)
    frequency_penalty=0.0,              # Penalize repeated tokens
    presence_penalty=0.0,               # Penalize already-used tokens
    logprobs=True,                      # Return token probabilities
    top_logprobs=5,                     # Number of top tokens to return
    user="user_123"                     # End-user ID for abuse tracking

)

Key Parameters Explained

Parameter Range What it does

Pro tip: Use temperature OR top_p, not both. They're alternative ways to control randomness. Most people use temperature.

📊

The Response Object

python

{

"id": "chatcmpl-abc123",
    "object": "chat.completion",
    "created": 1700000000,
    "model": "gpt-4o-2024-08-06",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "The capital of France is Paris."
            },
            "finish_reason": "stop"  # Why it stopped
        }
    ],
    "usage": {
        "prompt_tokens": 25,        # Tokens you sent
        "completion_tokens": 8,     # Tokens generated
        "total_tokens": 33          # Total
    }

}

finish_reason Values

Reason Meaning

Always check finish_reason! If it's length, the model's response was truncated. You're missing content.

Why This Format Won

The OpenAI API format became the standard because of three things:

1. First Mover Advantage

OpenAI shipped ChatGPT in November 2022 and the API in March 2023. By the time competitors shipped their APIs, thousands of tools, libraries, and tutorials already used the OpenAI format.

1. Simplicity

The API is genuinely well-designed. One endpoint, four message roles, a clean JSON format. Compare this to older ML serving APIs that needed custom gRPC schemas, protobuf definitions, and model-specific preprocessing.

1. Network Effects

Once LangChain, LlamaIndex, and other frameworks built around the OpenAI format, every new model provider had a choice: invent your own format and force developers to learn it, or just support the OpenAI format and get instant compatibility.

Everyone chose compatibility:

Provider Own API OpenAI-Compatible API

Anthropic Yes (Messages API) Community adapters

Some providers (Together, Groq, Ollama) don't even have their own format. They went straight to OpenAI compatibility. That's how dominant this spec became.

Using the Format with Any Provider

Because the format is standardized, switching providers is often just changing the base URL and API key:

python

from openai import OpenAI

OpenAI

client = OpenAI(
api_key="sk-...",
    base_url="https://api.openai.com/v1"

)

Groq (same code, different URL)

client = OpenAI(
api_key="gsk_...",
    base_url="https://api.groq.com/openai/v1"

)

Ollama (local, same code)

client = OpenAI(
api_key="ollama",  # Not actually needed
    base_url="http://localhost:11434/v1"

)

Together AI

client = OpenAI(
api_key="tog_...",
    base_url="https://api.together.xyz/v1"

)

Same code works with all of them!

response = client.chat.completions.create(
model="gpt-4o",  # Change to provider's model name
    messages=[{"role": "user", "content": "Hello!"}]

)

This is incredibly powerful. Your application code stays the same. You only change the configuration.

🔬

Multimodal Messages

The messages format extends naturally to images, audio, and other modalities:

python

Text + Image (Vision)

messages = [
{
        "role": "user",
        "content": [
            {"type": "text", "text": "What's in this image?"},
            {
                "type": "image_url",
                "image_url": {
                    "url": "https://example.com/photo.jpg",
                    "detail": "high"  # "low", "high", or "auto"
                }
            }
        ]
    }

]

Note: content can be a string OR an array of content parts

String: {"content": "Hello"}

Array: {"content": [{"type": "text", "text": "Hello"}, {"type": "image_url", ...}]}

🛡️

The Completions vs Chat Completions API

You'll occasionally see references to the old v1/completions endpoint. Here's the difference:

Completions (Legacy) Chat Completions (Current)

The old completions API treated the model like a text-in, text-out function. The chat completions API treats it like a conversational partner with roles and tool capabilities. Every modern model uses chat completions.

📦

Authentication

Simple bearer token authentication:

bash

curl https://api.openai.com/v1/chat/completions \

-H "Authorization: Bearer sk-..." \

-H "Content-Type: application/json" \

-d '{

"model": "gpt-4o",
    "messages": [{"role": "user", "content": "Hello!"}]

}'

For organization-level access:

bash

-H "OpenAI-Organization: org-..."

No OAuth, no API key exchange flows, no complex authentication. Just a bearer token. This simplicity is another reason the API won.

🚀

Practical Takeaways

One endpoint does everything — POST /v1/chat/completions is the only endpoint you need for 95% of tasks

Four roles define all conversations — system, user, assistant, tool

The format is the industry standard — learn it once, use it with every provider

Always check finish_reason — length means your output was truncated

Temperature controls creativity — 0 for deterministic, 0.7 for balanced, 1+ for creative

Switching providers = changing base_url — the OpenAI SDK works with almost any provider

🔄

What's Next?

Episode 37: Streaming — Why does ChatGPT show you words one at a time? It's not a fancy UI trick. The model genuinely generates tokens sequentially, and streaming delivers them as they're produced. SSE, chunked transfer, and why streaming makes everything feel faster.

← Previous

Ep 35: Agent Failure Modes

Next →

Ep 37: Streaming

Next: Episode 37 — Streaming

Watch ChatGPT's words appear one by one, like someone typing in real time. It's not a UX trick — it's how generation works.

This is part of a 98-episode series covering AI engineering from tokens to production deployment.