What Is an Agent: When AI Stops Answering and Starts Doing

ChatGPT answers your questions. An agent does your work.

That's the difference. And it's massive.

When you ask ChatGPT "what's the weather in Mumbai?" — it guesses. It might hallucinate. It doesn't actually check the weather. But an agent? It calls a weather API, gets the real data, reads it, and gives you the answer. Then it decides if you also need an umbrella recommendation and checks tomorrow's forecast too.

The jump from "chatbot" to "agent" is like the jump from a Google search box to a personal assistant who actually does things. And right now, this is where the entire AI industry is headed.

The Three Ingredients of an Agent

Every AI agent — from a simple code assistant to a fully autonomous researcher — is built from the same three components:

1. An LLM (The Brain)

The large language model is the reasoning engine. It reads the situation, thinks about what to do, and decides the next action. GPT-4, Claude, Gemini — any of them can be the brain.

But here's what makes it different from a chatbot: the LLM doesn't just generate text. It generates actions.

1. Tools (The Hands)

Tools are functions the agent can call. Search the web. Run code. Query a database. Send an email. Read a file. Call an API.

Without tools, an LLM is just a very expensive autocomplete. With tools, it can interact with the real world.

1. A Loop (The Persistence)

This is the ingredient most people miss. A chatbot does one turn: you ask, it answers. An agent runs a loop:

while task_not_complete:

1. Think about the current state
    2. Decide what to do next
    3. Execute an action (call a tool)
    4. Observe the result
    5. Update understanding
    6. Repeat

The loop is what turns a single LLM call into sustained, goal-directed behavior. The agent keeps going until it finishes the job — or runs out of budget.

A Concrete Example

Let's say you tell an agent: "Find the cheapest flight from Delhi to Bangalore next Friday and book it."

Here's what happens inside the loop:

Step Think Act Observe

Five iterations of the loop. Five decisions. The LLM didn't just answer — it reasoned, acted, observed, and adapted.

🔧

Chatbot vs Agent

People confuse these constantly. Here's the sharp distinction:

Chatbot Agent

Input User message User goal

Output Text response Completed task

A chatbot is like asking a friend for directions. An agent is like hiring a driver who takes you there.

📊

The Autonomy Spectrum

Not all agents are created equal. There's a spectrum from "barely autonomous" to "fully autonomous":

Level 0 — Chatbot: No tools, no loop. Just text in, text out.

Level 1 — Tool-augmented LLM: Can call tools but only when explicitly asked. "Search for X" → calls search → returns results. One tool call per turn.

Level 2 — Simple agent: Has a loop. Can chain multiple tool calls. "Research topic X and write a summary" → searches → reads pages → writes summary. But follows a fairly predictable path.

Level 3 — Autonomous agent: Makes complex decisions. Handles errors. Backtracks when something doesn't work. Can plan multi-step strategies. Most coding agents (Cursor, Claude Code, Codex) are here.

Level 4 — Fully autonomous: Runs for hours or days with minimal human input. Manages its own resources and priorities. We're not quite here yet for most use cases, but this is where things are heading.

What Makes Agents Hard

If it's just LLM + tools + loop, why isn't every app an agent? Because the devil is in the details:

Reliability

An LLM might decide to call the wrong tool. Or pass the wrong arguments. Or misinterpret a tool's output. In a single-turn chatbot, a bad answer is annoying. In an agent loop, a bad decision compounds — the agent goes down the wrong path, wastes money, and produces garbage.

Cost

Every iteration of the loop is at least one LLM call. A complex task might take 20-50 iterations. At $10-15 per million output tokens, an agent can burn through dollars in minutes. Imagine an infinite loop bug where the agent keeps calling tools forever — that's a real production concern.

Safety

A chatbot can only say wrong things. An agent can do wrong things. Delete files. Send emails to the wrong person. Execute bad code. Book the wrong flight. The stakes are higher because the agent has real-world capabilities.

Evaluation

How do you test something that makes dynamic decisions? A chatbot can be evaluated on answer quality. An agent needs to be evaluated on task completion, efficiency, cost, safety, and reliability — all at once.

Real-World Agent Examples

Agents are already everywhere, even if you don't call them that:

Agent Brain Tools Loop Type

The pattern is always the same: LLM + tools + loop. The difference is in which tools are available and how sophisticated the loop is.

🔬

Building Your First Agent (Pseudocode)

Here's what a minimal agent looks like:

python

import openai

tools = {
"search_web": search_web_function,
    "read_page": read_page_function,
    "write_file": write_file_function,

}

messages = [
{"role": "system", "content": "You are a research agent. Use tools to complete tasks."},
    {"role": "user", "content": "Research the latest AI agent frameworks and write a summary."}

]

The Loop

for step in range(20): # Max 20 steps to prevent infinite loops

response = openai.chat.completions.create(
        model="gpt-4o",
        messages=messages,
        tools=tool_definitions,
    )

message = response.choices[0].message
    messages.append(message)

# Check if the agent wants to call a tool
    if message.tool_calls:
        for tool_call in message.tool_calls:
            result = tools[tool_call.function.name](
                **json.loads(tool_call.function.arguments)
            )
            messages.append({
                "role": "tool",
                "tool_call_id": tool_call.id,
                "content": str(result)
            })
    else:
        # No tool calls = agent is done, it's giving a final answer
        print(message.content)
        break

That's a working agent in ~30 lines. The LLM decides when to call tools, which tools to call, and when it's done. The loop keeps going until the agent either finishes or hits the step limit.

🛡️

The Agent Pattern — Why Now?

Agents aren't a new idea. The concept of autonomous AI systems dates back decades. But three things converged in 2024-2025 that made them practical:

Models got good enough — GPT-4 and Claude 3+ can reliably follow complex instructions, use tools correctly, and reason about multi-step plans

Tool calling became standard — Every major API now supports structured function calling, making it easy to connect LLMs to external tools

Context windows got huge — 128K-1M token windows mean agents can maintain context across many loop iterations without forgetting what they're doing

Before these three things, agents were a research curiosity. Now they're a production reality.

📦

Practical Takeaways

An agent = LLM + tools + loop — that's the recipe, everything else is details

The loop is what separates agents from chatbots — sustained, multi-step execution toward a goal

Autonomy is a spectrum — from simple tool-augmented LLMs to fully autonomous systems

Agents can DO things, not just SAY things — this raises the stakes on reliability and safety

Cost control is critical — always set max iterations, token budgets, and spending limits

Every major AI product is becoming an agent — this is the direction of the entire industry

🚀

What's Next?

Episode 30: Tool Calling — How does an LLM actually "call" a function? It can't run code. It doesn't have hands. So how does tool calling work under the hood? The answer involves JSON schemas, structured output, and a clever trick that makes text generation feel like programming.

← Previous

Ep 28: Production RAG

Next →

Ep 30: Tool Calling

Next: Episode 30 — Tool Calling

An LLM can't run code, access the internet, or call APIs. So how does it 'use' tools? The answer is surprisingly elegant.

This is part of a 98-episode series covering AI engineering from tokens to production deployment.