Why good AI agents fail in production: The missing infrastructure layer

It was 6 AM when the first alert fired. Then the second. Then the third.

The on-call engineer opened her laptop to find 3 unrelated failures from a single AI agent. The agent handled support tickets, processed billing adjustments, and answered customer questions. It ran on LangChain. It had passed every test in staging, and it worked—until it didn't.

I've watched teams pour months into prompt engineering and model selection only to lose a weekend cleaning up an incident that had nothing to do with either. The failures that follow are not hypothetical. They're the kind of thing that happens the first time an agent meets production without the infrastructure needed to support it properly. And I've yet to meet a framework that ships that infrastructure—the identity boundaries, safety enforcement, and operational governance that production deployment demands.

43 duplicate support tickets

The agent—an AI system that reasons about goals, selects tools, and takes multistep actions autonomously—was monitoring a support queue. A customer submitted a case. The agent called the ticketing API to create a record,  and the API accepted the request and created the ticket. But then the response timed out.

From the agent's perspective, the call failed. LangChain did what any well-designed framework does on a failed tool call: it retried. The retry created a second ticket. The response timed out again. Another retry. Another ticket.

By 6 AM there were 43 duplicate tickets in the system. Each triggered an automated acknowledgment email—which meant 43 messages landing in customer inboxes. The support team spent the morning manually closing duplicates and sending apologies instead of doing their actual work.

I want to be precise about what went wrong here, because it matters. LangChain handled this correctly. The framework detected a failed response and retried the call. That is exactly what the retry logic is designed to do. The problem wasn't the retry, it was that nothing in the infrastructure tracked whether the original request had succeeded. There was no idempotency envelope (a mechanism that ensures a repeated request produces the same result as the original, preventing duplicates). No deduplication layer. No platform-level awareness that this was a retry of an already-completed operation.

For any decision maker doing the math: the cost here was not just an embarrassing morning. It was support-staff hours diverted to cleanup, customer trust eroded by robotic duplicate emails, and a system that couldn't be trusted until someone manually audited every open ticket. For any developer reading the retry logic, the code was correct. The gap lay beneath it.

The framework did its job. The infrastructure around it just didn't exist.

$4000 charged to the wrong account

Infrastructure gaps can be loud—43 tickets loud— but it can also be silent. The same AI agent processed billing adjustments. It had access to a billing API provisioned with a service account during deployment. The service account could reach multiple billing accounts because that was the fastest way to get the agent running in development. No one scoped the credentials before promoting the agent to production because no one had built an identity boundary—the infrastructure layer that controls which resources a given workload can access based on verified identity, not just configuration.

The agent needed to process a charge against Account A. Its prompt included the account identifier. The model selected Account B—a plausible error for any large language model (LLM). The billing API accepted the call and $4,000 was charged to the wrong customer.

This wasn't a model failure. LLMs produce plausible outputs, but sometimes they're wrong. The likely cause was the same billing API with the wrong account parameter—a plausible selection error.

In traditional software, the potential impact of a misrouted API call is limited by 2 layers: credential scoping constrains which services a workload can reach, and the downstream API enforces which parameters are valid for a given caller. A developer would never ship a web application where a single set of credentials could write to every customer's billing record, but that's exactly what happened here.

The agent had been provisioned with credentials broad enough to reach any account in the system, and nothing in the stack—neither the platform nor the downstream API—constrained that scope. No framework currently ships credential scoping. None provide cryptographic workload identity or enforce which services an agent can reach based on infrastructure-level policy rather than prompt content.

Nobody noticed until the following Monday, when someone reviewed the invoice. I've  seen variations of this failure at every scale—the details change, but the root cause is always the same. The agent had credentials it shouldn't have held, and nothing in the stack enforced the boundary.

A refund policy the company never wrote

Unenforced boundaries caused a $4,000 error that night, but the third failure cost something harder to quantify. A customer asked the agent about the return window for a product purchase. The model generated a confident, clear response: the refund policy allows returns within 90 days. The actual policy was 30 days. The response was articulate, helpful, and completely wrong.

LangChain passed the model's output to the customer-facing response channel. Again, the framework did exactly what it was built to do—route the model's output to the next step in the chain. There was no guardrail at the inference boundary—the point where a model's output enters the real world,  whether sent to a customer, written to a database, or used to trigger an action). No validation layer compared the output against documented policy. No mechanism checked whether the agent's response contradicted known facts before it reached the customer. The output went from model to customer in a straight line, with nothing in between.

The customer returned a $280 product on day 47, citing the agent's response. The team honored the return. The cost was not just the refund itself, it was the legal exposure—an AI agent had made an unauthorized contractual representation to a customer, and the company had no audit trail showing what controls were in place to prevent it.

The billing error has a dollar amount. The duplicate tickets have a count. A fabricated policy sent to a customer is harder to quantify, and harder to contain. It's a liability that compounds—every response the agent sends without validation is another potential contractual commitment your legal team never approved. Every organization running an agent that communicates with customers faces this risk, and most of them face it without knowing.

The agent was fine

3 failures in a single night from an agent that worked perfectly in development.

Look at the pattern. In every case the framework performed correctly. LangChain orchestrated the tool calls, it routed the outputs, it retried on failures. The agent loop—reasoning, planning, tool calling, orchestration—worked as designed.

Every failure happened because production infrastructure was absent. The first failure needed idempotency and deduplication at the platform level. The second needed an identity boundary with scoped credentials and cryptographic workload identity. The third needed guardrails at the inference boundary—output validation against documented fact before the response reached a customer.

These are not framework concerns. Identity, tool governance (controlling which tools an agent can access and under what conditions), observability (tracking every action an agent takes so you can audit and debug), and safety enforcement are platform concerns—none of which any framework was designed to solve.

I've talked to teams running agents on LangChain, CrewAI, LangGraph, and custom frameworks. The framework choice varies. The gap doesn't. They all hit the same wall: the distance between "works in development" and "runs in production" isn'ta framework problem, it's an infrastructure problem, and it costs real money, real customer trust, and real engineering time while it remains unsolved.

What closes the gap

That infrastructure problem has a specific shape. Agent frameworks solve the agent loop. They were never designed to solve production infrastructure, and they shouldn't have to. I wouldn't expect a web framework to ship its own Transport Layer Security (TLS) certificate authority. These are different categories of engineering.

Red Hat AI provides the production infrastructure that frameworks don't. Red Hat OpenShift AI acts as the foundation. The operating principle is BYOA—Bring your own agent, where the platform operationalizes any agent runtime, whether LangChain, CrewAI, LangGraph or custom code, without requiring changes to the agent. Your framework investment is preserved. Your agent code doesn't change. The platform injects the identity, safety, governance, and observability layers underneath.

The gap you already have

The agent that broke at 6 AM wasn't a bad agent. It was a good agent running without a floor beneath it. Every framework gives you the reasoning loop, the tool calls, the orchestration. None give you the infrastructure that decides whether those capabilities are safe to run in the real world. That distinction—between what the framework handles and what the platform must handle—is the production gap. It isn't a gap your framework was designed to close.

If you have agents in development right now, I bet they're standing on the same missing floor. Not because you built them wrong, but because the floor doesn't ship with the framework. Our next article introduces the specific pieces that floor is made of and what it takes to put them in place.

Get started

Ready to see what production-ready agent infrastructure looks like? Start here:

• Try OpenShift AI free in the Developer Sandbox: a preconfigured environment for experimenting with AI workloads
• Explore the Red Hat AI interactive demos: hands-on introductions to Red Hat AI
• Start with the BYO Agent starter kits: pre-configured templates for LangGraph, CrewAI, LlamaIndex, Langflow, Google ADK and more
• Read Red Hat AI documentation: full platform documentation
• Operationalizing BYOA on Red Hat AI: The OpenClaw edition: a hands-on article about deploying a real agent on the platform