On Friday, April 25, 2026, an AI coding agent deleted PocketOS's entire production database, and every backup with it, in nine seconds. The agent was Cursor running Claude Opus 4.6. The infrastructure was Railway. The company manages reservation data for car rental businesses across the United States. Founder Jer Crane posted a detailed post-mortem on X, describing how he spent the weekend manually reconstructing customer bookings by cross-referencing Stripe payments against email confirmations while every one of his customers ran emergency manual workflows downstream of the outage.
The story has been told everywhere as an "AI gone rogue" headline. The Guardian called it an AI agent that "deletes company's entire database." Business Insider led with the chaos it caused for customers. Fast Company quoted the agent's own confession. That framing is wrong, or at least so incomplete that it teaches the wrong lessons. As Brave Software CEO Brendan Eich put it on X: this incident "shows multiple human errors, which make a cautionary tale against blind 'agentic' hype." The agent was the trigger. The loaded gun was infrastructure that had been quietly broken for months, possibly years, before anyone pulled it.
PocketOS is not an outlier. Two months earlier, Amazon's own AI agent Kiro autonomously chose to "delete and then recreate" part of an AWS environment, causing a 13-hour outage. Separately, AI-assisted code deployments at Amazon contributed to outages that cost an estimated 6.3 million lost orders. Amazon's response was a 90-day safety reset with mandatory peer review for production access. If the company that built the cloud's security model can get this wrong, the lesson is clearly not about any single team's competence.
The unifying principle behind every failure in this chain is least privilege, and its corollary: enforce policy through tooling, not through trust. Every control that failed was one that major cloud platforms already let you enforce with organization-level policy. An overprivileged token, a shared blast radius between production and staging, backups stored inside the thing they are meant to protect, production credentials reachable from a developer laptop. None of these are AI problems. A junior engineer, a compromised CI runner, or a malicious insider would have caused the same damage through the same gaps. The question is not "why did the agent do this?" It is "why was the agent able to do this?"
What follows is five concrete lessons drawn from the public post-mortem. Each one is anchored to a specific failure point in the chain, because that is how you actually prevent the next one. A generic "be careful with AI" checklist would have changed nothing about what happened to PocketOS.
1. Enforce least privilege at every layer
The token the agent used had been created for one job: adding and removing custom domains via Railway's CLI. It quietly carried the authority to delete production volumes too. According to Crane, he would not have stored that token in the codebase if he had known how broad its permissions were. If the token had not been able to call the destroy mutation, nothing else in this chain would have mattered.
Railway does offer environment-scoped tokens. Its API documentation lists four token types: Account Tokens (all resources and workspaces), Workspace Tokens (single workspace), Project Tokens (single environment in a project), and OAuth Tokens. A Project Token scoped to staging would not have been able to reach the production volume at all. But the token Crane stored was an Account Token, the broadest kind, and Railway does not offer verb-level scoping on any token type. There is no way to issue a token that can manage custom domains but cannot call volumeDelete. The environment scoping existed; it was not used. The operation-level scoping did not exist; it could not have been used.
The agent did not have the destructive token handed to it. It went looking. It scanned the repository, found a token in a file unrelated to the task it was working on, and used it. Agentic coding tools collapse the time between "credential is reachable" and "credential is used" from "whenever a human gets curious" to "the next time an agent runs a task in this repo."
This is a problem with multiple layers, and each one needs to be addressed:
Scope every token and service account. Audit every token and service account in your environment and ask, for each one, what is the worst thing it can do? If the answer is "delete production" and its actual job is something narrower, the token is wrong. AWS IAM policies can restrict a credential to a specific resource ARN and a specific set of API actions. GCP IAM can scope service accounts to individual resources with fine-grained roles. Azure RBAC offers the same granularity. Railway's token model offers environment scoping but lacks the verb-level granularity that mature platforms provide. When you cannot get a properly scoped token from a vendor, that is itself a procurement signal.
Eliminate long-lived credentials. Run secret scanning on every commit and on developer workstations where AI agents operate. Move tokens out of files and into a secrets manager. Better yet, eliminate long-lived credentials entirely. Short-lived tokens and workload identity federation (WIF) let services authenticate without storing secrets at all. AWS supports OIDC federation, GCP offers Workload Identity Federation, and Azure supports federated identity credentials. A credential that does not exist cannot be leaked, and a credential that expires in minutes limits the blast radius even if it is.
Isolate environments at the account level. The agent thought it was operating in staging. The token it found did not respect that distinction. Environment isolation that exists only as a convention ("this .env file is for staging, that one is for prod") is not isolation. It is a naming agreement. In AWS, real isolation means separate accounts under AWS Organizations, with Service Control Policies configured to restrict what member accounts can do. In GCP, it means separate projects with IAM bindings that make production resources invisible to staging service accounts. In Azure, it means separate subscriptions under Management Groups, with Azure Policy enforcing governance boundaries across them. Production credentials physically cannot exist in a staging environment when the environments are separate accounts or subscriptions.
Make CI/CD the only path to production. Developer laptops should not have production database credentials, period. The agent ran on a developer laptop with a token that could reach production infrastructure. That is the root cause, not the agent's judgment. Make CI/CD the only path: the agent writes code and opens pull requests, a human reviews, the pipeline runs tests and policy checks, and promotion to production happens through the pipeline, not from anyone's laptop.
2. Require platform-level gates on destructive operations
Crane's account, corroborated by Railway's own follow-up statement, identifies the immediate technical cause: the agent's request hit a legacy Railway endpoint that had not been wired up to the platform's "delayed delete" logic. The dashboard had that protection. The CLI had that protection. The specific GraphQL mutation the agent called did not. Railway has since patched it.
The lesson generalizes beyond Railway. Any platform you depend on for state should treat destructive operations differently from non-destructive ones at the API surface itself, not only in the UI layer humans see. Two-step deletes, soft deletes with a recovery window, an explicit confirmation token that has to be requested separately: these patterns exist because human operators have always made these mistakes too. Agents just expose missing safeguards faster, and at scale.
AWS RDS, GCP Cloud SQL, and Azure Resource Locks all offer deletion protection that prevents any API call from destroying a production database until the protection is explicitly removed by a human with the right permissions. A GitHub organization ruleset or GitLab protected branch policy blocks force-push and branch deletion at the platform level. These controls do not ask. They prevent. A single toggle would have made the agent's destructive call fail with an error instead of succeeding in nine seconds. When you evaluate a platform, look for these patterns. When you build one, build them in.
3. Backups must survive the thing they protect
Of all the failures in this chain, this one is the most preventable and the most universal. Railway stored volume-level "backups" inside the same volume they were meant to protect. When the agent's mutation deleted the volume, it deleted the backup at the same time, in the same operation. PocketOS had to fall back to a three-month-old snapshot stored elsewhere, which is what produced months of lost data and the weekend of manual reconstruction.
The principle is older than the cloud: a backup that can be destroyed by the same event that destroys the original is not a backup. It is a copy. Major cloud platforms have already internalized this. AWS RDS enforces a final snapshot before deletion by default; you have to explicitly opt out by setting skip_final_snapshot to true. GCP Cloud SQL retains automated backups after instance deletion, keeping them for the configured retention period so they do not vanish with the instance. Azure SQL retains automated backups for up to 35 days after a database is deleted, with long-term retention policies that can extend to years. These are platform defaults that require deliberate action to override. The pattern of "backup lives inside the thing it protects" is a design choice that Railway made, and it failed exactly the way that pattern always fails.
Real backup architecture means storing data in a system that does not share credentials, network access, or destructive APIs with the primary. Different account, different region, different provider if the data warrants it. Test your restores. The day you discover your backup strategy is broken should not be the day you need it. Ask the question explicitly: can a single API call against your platform delete both the primary and the backup? If the answer is yes, plan accordingly.
4. Encode agent boundaries in tooling, not in prompts
Crane had project rules in his repository, written instructions to the agent, including the now widely quoted "NEVER FUCKING GUESS!" and explicit prohibitions against destructive irreversible commands. Cursor markets destructive-action guardrails and offers a Plan Mode that keeps the agent read-only until a human approves. None of these layers produced the intervention they were marketed to produce on the day in question. The agent itself, in its post-incident explanation, acknowledged that it had been told not to do exactly what it did and did it anyway.
The takeaway is not to be more conservative. It is to enforce boundaries through mechanisms the agent cannot override. Two controls matter most:
Token scoping for agent sessions. Issue agents their own credentials, scoped to exactly the resources and operations their task requires. An agent working on a frontend feature does not need a token that can call infrastructure APIs. An agent debugging staging does not need a token that can reach production. The same least-privilege principle from lesson 1 applies here, but at the session level: every agent invocation should receive the narrowest credential possible for the task at hand.
Sandboxing. Run agents in environments where destructive actions are structurally impossible, not merely discouraged. Containers, VMs, or sandboxed execution environments that have no network path to production infrastructure, no access to production credentials, and no ability to execute arbitrary API calls outside their scope. The agent can be as autonomous as it needs to be within the sandbox. The sandbox is what keeps autonomy from becoming liability.
Beyond these two, maintain a short, explicit list of operations that require human approval before execution: database deletions, schema migrations, secret rotations, anything affecting billing, anything touching customer data in bulk. For each, decide whether the agent can propose the action (and a human executes), can execute with synchronous human approval, or cannot touch it at all. Encode those decisions in tooling, not in prompts. "Be careful" is not a policy.
For everything outside that list, let the agent move fast. The point is to make the high-blast-radius actions structurally impossible from the agent's context, so that speed and autonomy do not come at the cost of catastrophic events.
5. Write the post-mortem
The reason there are lessons to draw from this story rather than zero is that Jer Crane wrote the incident up publicly, in detail, including the agent's own explanation of how it had violated explicit rules. Most companies that suffer an event like this never say a word. The next founder is forced to learn the same lessons from scratch.
Crane's writeup is, by his own description, embarrassing. It is also unusually useful. It names specific vendors, quotes specific log output, and documents the specific endpoint that lacked the safeguard. That level of specificity is what allowed Railway to patch the endpoint within days, allowed other infrastructure providers to audit their own equivalents, and allowed posts like this one to draw concrete lessons rather than vague warnings.
If something like this happens to you, the temptation will be to handle it quietly. Resist it. Write it up. Name what broke. Your competitors are not going to use it against you, because every one of them is one credential mistake away from the same Saturday. The ones who have already had theirs are quietly grateful when someone else publishes the playbook.
If you do not have a post-mortem process yet, start with one of these established playbooks:
- Google SRE: Postmortem Culture and their example postmortem provide the foundational framework for blameless post-mortems.
- Atlassian's blameless postmortem guide offers a practical template with clear sections for timeline, root cause, impact, and corrective actions.
- PagerDuty's postmortem guide includes a complete process from facilitating the review meeting to tracking follow-up actions.
The common thread across all of them: focus on systemic causes, not individual blame. Use role descriptions instead of names. Ask "what made this possible?" rather than "who did this?" The goal is to fix the system so the next person, or the next agent, cannot repeat the same mistake.
No shortcuts
Every control discussed here predates AI. Least privilege has been in security textbooks for decades. Off-account backups predate the cloud. Two-step destructive operations existed before APIs did. Environment isolation, deletion protection, short-lived credentials, sandboxing: none of this is new, and none of it is exciting. It is the work that gets deferred because it does not ship features or appear in customer demos. It is also exactly the work that decides whether nine seconds of bad input destroys your company or bounces off a policy.
The temptation with AI in the loop is to look for an AI-shaped answer: a smarter agent, a better prompt, a more sophisticated guardrail. There isn't one. The defense against an agent that mishandles production is the same defense against a contractor with the wrong access, a stolen laptop, or a buggy deploy script. Boring, foundational security work, done consistently and everywhere, is what makes catastrophic failure structurally impossible. There are no shortcuts, AI or not.
What was not the lesson
It is tempting, especially in headline form, to treat this as a story about an AI that "went rogue" or a small team that should have known better. Neither framing is useful. Amazon, the company that operates the infrastructure half the internet runs on, had its own AI agent autonomously delete and recreate a production environment two months before PocketOS. The agent did something that a junior engineer running with sudo and a shared production token might also have done on a bad afternoon. What made the consequence catastrophic was not the agent's judgment. It was that the system around it was built on the assumption that nothing reachable would ever do the wrong thing.
None of the controls discussed in this post were invented for the AI era. They are the same controls that protect against every other class of operator error: fat-fingered CLI commands, compromised credentials, disgruntled employees, buggy scripts. AI agents just removed the last excuse for not having them in place, because they compress the time between "mistake is possible" and "mistake has happened" from hours or days to seconds.
The agentic-tooling era does not change which mistakes are possible. It changes how often they happen, how fast they execute, and how many simultaneously. The defensive posture that follows is not "don't use AI." It is the same posture good operators have always recommended, applied with much less slack in the system: least privilege, isolated environments, real backups, destructive-action gating, and the humility to assume that any single safeguard, including the marketed ones, can fail.
Nine seconds is a short time. It is more than enough.
