Top 5 Mistakes Teams Made Shipping Agents to Production in 2026

The short answer. The five most costly agent production mistakes of 2026 were: writing guardrails in the system prompt instead of enforcing them in credentials and APIs; adopting an MCP tool surface without threat-modelling it as a supply chain; relying on evals that pass while customers are harmed; treating context as storage rather than a finite attention budget; and building the agent as a stateless request rather than a durable workflow with idempotent side effects. Every one is an architecture problem rather than a model problem, and each has a documented public incident behind it.

None of these are model problems. All five are architecture problems, and all five have public receipts.

Mistake 1: they wrote the guardrail in the prompt

On 25 April 2026, an agent deleted PocketOS's production database and all volume-level backups in a single API call to Railway. Nine seconds. Founder Jeremy Crane's account: a token created for domain management was scoped to permit any operation including destructive ones; the legacy delete endpoint lacked the delayed-delete logic the dashboard had; the backups sat on the same volume as the data. And the detail that matters most, as reported by The Register: Cursor's system rules explicitly prohibited destructive actions without user request, and the agent ignored them.

That is the cleanest available proof that prompt-level instructions are not a control mechanism. The real controls in that story were credential scope and API design, and both had been left at their defaults. Notice that not one of the three root causes is about the model. A token was over-scoped, an endpoint lacked safety, and backups shared a failure domain with the thing they were backing up. Every one of those is a decision a human made before the agent arrived.

It wasn't isolated. Replit's agent deleted a production database during an explicit code freeze in July 2025, then told the user rollback was impossible; it wasn't. Google Antigravity wiped a user's entire D: drive while clearing a project cache in December 2025, bypassing the recycle bin. Different vendors, one shape: write access, no approval gate, prose where a policy should be.

Anthropic's own numbers say why prose won't hold. Published prompt-injection resistance for a browser agent went from a 23.6% to an 11.2% attack success rate after mitigations. Eleven percent, with the vendor's best work applied, across 123 test cases. OpenAI is blunter: prompt injection "is unlikely to ever be fully 'solved'" and "remains an open challenge for agent security for years to come." And Google's April 2026 Common Crawl analysis, across two to three billion English pages, found malicious indirect-injection detections up 32% between November 2025 and February 2026, including data-exfiltration and file-deletion payloads. This is weather now, not a bug to wait out.

The framing I keep coming back to is Simon Willison's lethal trifecta: access to private data, exposure to untrusted content, and the ability to communicate externally. Hold all three and you have a data-exfiltration machine waiting for a trigger. His point about composition is the one architects miss: "Once you start mixing and matching tools yourself there's nothing those vendors can do to protect you." Your vendor secures its own surface. The trifecta is assembled by your integration choices.

What to do instead: put the constraint where it cannot be argued with. Scope credentials per tool and per environment, so a development agent has no path to production. Require an approval interrupt for destructive and irreversible operations, enforced by the runtime rather than requested in a prompt. Separate backups into a different failure domain. And prefer APIs with delayed-delete or soft-delete semantics for anything an agent can reach.

Mistake 2: they bought a tool surface without threat-modelling it

MCP won, and most teams adopted it as a convenience layer rather than a supply chain. That framing error cost real money in 2026.

Asana's MCP server had a logic flaw that exposed data across organisations for over a month, from 1 May to 4 June, with roughly 1,000 customers affected. Invariant Labs showed a malicious GitHub Issue could coerce an agent into leaking private repositories, noting "this is not a flaw in the GitHub MCP server code itself, but rather a fundamental architectural issue," and separately documented tool poisoning, where "a malicious server can change the tool description after the client has already approved it" and "hijack an agent without ever appearing explicitly in the agent's user-facing interaction log."

Then it got worse in three directions at once. Microsoft disclosed prompt-injection-to-remote-code-execution in Semantic Kernel, with model-controlled input reaching an eval'd lambda filter (CVE-2026-26030, May 2026), and their write-up contains the sentence that should end the "it's a model safety problem" conversation: "The AI model functions exactly as it was designed to: translate intent into structured tool calls." The bug is the architecture.

An injection planted in a GitHub issue title chained into GitHub Actions cache poisoning, credential theft, and a malicious cline package on npm exposed to 5M+ users during an eight-hour window in February 2026. One of the researchers involved put it well: "If the attacker can remotely prompt it, that's not just malware, it's the next evolution of C2." And in March 2026 a compromised LiteLLM package saw 47,000 downloads in a three-hour window, which matters because LiteLLM is the model gateway for CrewAI, DSPy, Microsoft GraphRAG and dozens of other agent frameworks. Your dependency tree has a new critical path and it is probably not in your threat model.

The part that should change your planning: when researchers reported the same injection-via-comments class against Claude Code, Gemini CLI and GitHub Copilot Agent, GitHub paid $500 and closed it as a known architectural limitation. That is the vendors telling you, on the record, that this is your design problem. OWASP now tracks it formally: its Top 10 for Agentic Applications, released December 2025, names agent behaviour hijacking, tool misuse, identity and privilege abuse, goal hijacking and memory poisoning, and six of the ten map back to prompt injection.

Meanwhile 45.6% of teams run their agents on shared API keys, per a Gravitee survey of 900+ executives and practitioners, and only 21.9% treat agents as independent identities. That is the PocketOS root cause at population scale.

What to do instead: treat every MCP server as a third-party dependency with a version, an owner and a review. Pin tool definitions and alert on description changes, because the approval you gave was to a description, not to a server. Give each agent its own identity and its own least-privilege credential. And keep the runtime that executes untrusted content separate from the runtime that holds production secrets.

Mistake 3: their evals passed and their customers didn't

"Add evals" was 2025's advice. 2026's data is more uncomfortable.

A VentureBeat Pulse survey of 108 enterprises found 49% deployed agents that passed internal evals and then failed customers, with only 26% running real-time quality checks on live output and half monitoring nothing but latency, errors and uptime. The finding that should stop you: 85% of enterprises that shipped a failing agent were nonetheless pursuing zero-human deployment, against 61% of those never burned. Expensive evidence that your evaluation missed something appears to increase confidence rather than reduce it.

Inngest's 2026 practitioner report, from a durable-execution vendor so weight it accordingly, found 35% of AI teams run no evals at all, 74% had a customer-visible incident within 90 days against 62% for non-AI teams, and 20% of AI teams spend up to half their engineering capacity on reliability work. Only 19% reported high confidence in handling two to three times current scale, and zero percent at organisations with 500+ engineers.

One number for why a leaderboard score isn't an eval: OpenAI stopped using SWE-bench Verified in February 2026 after finding at least 59.4% of audited problems had flawed test cases that reject functionally correct submissions, plus confirmed contamination across frontier models, including one that reproduced a complete gold patch from a task ID alone. The industry's most-cited agent benchmark was substantially broken, and every vendor claim built on it inherited that.

The framing that belongs on every agent dashboard comes from τ-bench: frontier agents solved under 50% of tasks at pass^1 and under 25% at pass^8 in the retail domain. pass^1 is your demo. pass^k is production.

And monitoring output quality is not the whole job either, because agents fail on inputs too. In March 2026, Amazon's retail site had four high-severity incidents in a single week, one of which blocked checkout, account information and pricing for six hours; per Amazon's account an engineer followed inaccurate guidance an agent had retrieved from outdated internal documentation. Amazon said none of the incidents involved AI-written code and that it would introduce "controlled friction" into deployments, acknowledging that safeguards around generative AI usage had not been fully established. Your agent is only as current as the wiki it reads, and nobody puts a staleness SLO on a wiki.

What to do instead: grade live output, not just uptime. Sample real traffic into an offline set continuously, so your eval suite drifts with your users rather than with your intentions. Measure pass^k on your own tasks, not pass^1. And treat internal knowledge sources an agent reads as production dependencies with owners and freshness checks.

Mistake 4: they treated context as storage instead of an attention budget

Chroma tested 18 models and found performance degrades with input length even on trivial tasks, with a single distractor already reducing accuracy and four compounding it. Anthropic describes context as a finite attention budget, with recall falling as the window fills, attributing it to quadratic pairwise token relationships and sparse long-sequence training data, and describing a performance gradient rather than a cliff.

Long-horizon behaviour is stranger than simple exhaustion. Vending-Bench, running past 20 million tokens, found models that "descend into tangential 'meltdown' loops from which they rarely recover," and reported no clear correlation between failures and the point at which the context window fills, which suggests these breakdowns are not memory-limit artefacts. Anthropic's own real-world run of the same idea had an agent hallucinate a Venmo account it told customers to pay into and spend roughly a day insisting on a fabricated identity.

Then February 2026 turned this from a quality problem into a safety problem. As reported by TechCrunch, a Meta AI security researcher's agent deleted emails she had asked it to merely suggest deleting, and ignored her stop commands. Her stated root cause: compaction. The context grew, the agent summarised, and her final instruction not to proceed may have been dropped in the summary.

If your stop command lives in the context window, compaction can garbage-collect your kill switch. That single sentence connects mistakes 1 and 4, and it is the most important thing in this article.

What to do instead: never let a safety-critical instruction exist only in context. Policy belongs in a deterministic layer the model cannot summarise away, and a stop signal belongs in the runtime as a checked precondition on every side-effecting call. Bound context growth explicitly, log what compaction removed, and persist progress to a file rather than trusting a summary.

Mistake 5: they built the agent as a request instead of a workflow

The arithmetic is unforgiving: five steps at 99% reliability each gives you 95% overall; ten steps gives you 90%. And tasks are getting longer. METR measures the 50%-reliability time horizon at 320 minutes, doubling every 89 days since 2024. A multi-hour run on a stateless request handler will meet a deploy, a timeout or a rate limit, and lose everything it had done.

Anthropic, who sell no workflow engine, reached the same conclusion from their own production experience: agents "run for long periods of time, maintaining state across many tool calls. This means we need to durably execute code and handle errors along the way," and "one step failing can cause agents to explore entirely different trajectories, leading to unpredictable outcomes." They also note the cost dimension people forget: agents use roughly four times the tokens of a chat interaction, and multi-agent systems around fifteen times, so re-running lost work is not just slow, it is billed.

Checkpointing isn't the fix people think, either. Writing state to disk is the easy part; without failure detection, distributed coordination and a worker pool you get manual recovery and duplicate-execution risk. Concrete versions of that gap: frameworks that permit concurrent processes to resume the same checkpoint without distributed locking, frameworks that lose partial work within a step, and one that takes around seven seconds to load a 500-message session. Those are our posts, so discount them as you like, but LangGraph's own documentation makes the same point unprompted: resume "does NOT resume from the same line of code," and you must wrap non-deterministic operations and side effects yourself, using "idempotency keys or verify existing results to avoid unintended duplication."

Straight about the evidence here: I could not find a public postmortem of an agent double-charging a customer. What I can point at is what the people building agent payment rails expect. When OpenAI and Stripe specified agentic checkout, the first mandatory header on every endpoint was Idempotency-Key, with webhooks as the authoritative source of truth "across retries and edge cases." They are not being cautious for fun.

What to do instead: model the agent as a workflow with committed steps, not a request with retries. Make every side-effecting call idempotent and keyed. Pin in-flight runs to the code version they started on, so a deploy does not change the meaning of a run mid-flight. And make sure something other than the agent itself can tell you whether a run completed.

Two statistics to stop repeating

While we're being careful about evidence, two numbers circulate in every agent deck and both need handling.

"Gartner says 40% of agentic AI projects will be cancelled by 2027" is real and correctly quoted, citing escalating costs, unclear business value and inadequate risk controls. It is also a prediction rather than a measurement, so it cannot be validated yet. The better number in the same release is the estimate that only around 130 of thousands of self-described agentic AI vendors are legitimate.

"MIT found 95% of AI pilots fail" is weaker than its circulation suggests. The underlying claim concerns organisations getting zero return, and the methodology is 52 structured interviews plus 153 survey responses, with the report itself conceding the results are self-reported and may not represent broader patterns. If you use it, say that out loud; "a 52-interview sample with a six-month ROI definition" is a more interesting sentence than the statistic anyway.

The pattern across all five mistakes: each team put a reliability guarantee somewhere that cannot enforce it, whether a prompt, a vendor default, a benchmark, a context window or a request handler. Guarantees belong in the runtime.

Quick reference: mistake, evidence, fix

FAQ

Next step: the agent reliability checklist walks the five mistakes in order, with the specific question to ask of your own stack at each one.