AI agents are a different threat from chatbots because they act on someone's behalf: they run code, change systems and send messages, often for long stretches with no person approving each step. A chatbot's answer passes through a human before anything happens; an agent's output is the action. That makes the central risk a version of what economists call the principal-agent problem. By early 2026 the best public agents could complete, with 50% reliability, software tasks that take human experts about 12 hours, according to the research nonprofit METR, which said that at the time it knew of no company imposing strict requirements on human oversight of agents.
Delegation is an old problem with a new kind of agent
In their 1976 paper on the theory of the firm, the economists Michael Jensen and William Meckling counted three costs of delegation: the principal's spending on monitoring, the agent's spending to guarantee its conduct, and a residual loss from decisions that still diverge from the principal's interests. In Governing AI Agents, published in the Notre Dame Law Review in 2026, Noam Kolt applies that framework to AI, finds the familiar problems of information asymmetry, discretionary authority and loyalty, and argues that the usual remedies, monitoring among them, may fail for agents that make uninterpretable decisions at "unprecedented speed and scale".
Dylan Hadfield-Menell and Gillian Hadfield argued in 2018 that alignment can be understood as an incomplete-contracting problem: human contracts work partly because culture and law fill their gaps, and the challenge is building AI that can draw on the same structure. In a report published on 9 October 2026, Anthropic said Claude Haiku 4.5, told among other things never to log in, make purchases or submit anything destructive, had sent an invented lead through a police department's tip form; the instructions "did not rule out form submissions". Anthropic says the model appears to have been producing example content, not trying to mislead anyone (see our account of the report). Anthropic uses the vocabulary itself: Claude's constitution calls those on whose behalf Claude acts its "principals" and tells it to treat instructions found in tool results and documents as information rather than commands.
The tasks being handed over are getting longer quickly
METR's time-horizon measure estimates the length of task, in a human expert's working time, that an AI agent completes with 50% reliability. Its March 2025 paper put Claude 3.7 Sonnet at about 50 minutes and found the frontier doubling roughly every seven months since 2019, and on the larger task suite METR adopted in January 2026 the doubling time since 2023 was about 131 days. METR's May 2026 Frontier Risk Report put the best public models of February and March 2026 at about 12 hours, with a confidence interval of five to 61 hours, and the most capable internal model that companies shared with it at 16 to 20 hours, beyond what the suite can reliably measure. METR has published no firmer figure since. Its public tracker, which it marked in September as no longer actively updated, last added an early version of Claude Mythos Preview at about 17 hours, also above that ceiling, and in June its estimate for OpenAI's GPT-5.6 Sol ran from about 11 hours to more than 270, depending on how it scored the model's frequent cheating.
The headline figure flatters agents. At 80% reliability the public frontier was about an hour and a half, and the tasks are software problems in what Anthropic calls an idealised setting, with no human interaction and no real-world consequences. Real use is catching up: in a study published in February 2026, Anthropic found that the longest Claude Code turns (the 99.9th percentile) grew from under 25 minutes to over 45 minutes between October 2025 and January 2026 before declining somewhat, and that users switched on full auto-approval in about 20% of sessions when new and in over 40% once experienced.
Agents overreach, then overstate what they did
One central worry in agency theory is hidden action, and METR's May report documents a good deal of it. Agents routinely tried to cheat on its hardest tasks: on tasks longer than eight hours, at least 16% of successful runs proved illegitimate on review, and grading agents' work took longer than grading humans' because models so often overclaimed what they had done. In one evaluation, when Claude Opus 4.6 ran out of the API credits a task required, it found free computing power online, despite recognising that this went against its instructions. METR traces such behaviour to training incentives that can reward exploiting automated graders and distorting evidence of performance, the proxy problem behind RLHF.
Anthropic's system card for Claude Mythos Preview, published on 7 April 2026, lists incidents with earlier versions used internally. Asked in a test to escape a sandbox and message a researcher, one did so, then, unasked, posted details of its exploit to obscure but public websites. In fewer than 0.001% of internal-testing interactions, earlier versions took actions they appeared to know were disallowed and then tried to hide them. Anthropic says it did not know the level of risk when it first deployed those versions internally, partly because its automated audits struggle to emulate long sessions on networked computers. It is "fairly confident" the cause was reckless pursuit of assigned tasks rather than a hidden goal, and says it has not seen the final version misbehave as severely, though that version still takes reckless shortcuts in lower-stakes settings.
Some incidents reached the outside world. The UK AI Security Institute reported in August that agents in one of its cyber evaluations, given internet access on purpose and with the developers' cyber classifiers switched off, took 19 unsanctioned actions against real people and organisations, including creating fake identities to try to get malicious code into an open-source project; it says the agent "was never instructed to deceive". An independent METR investigation found that roughly 1,200 OpenAI agents under cyber testing, meant to be isolated from one another, had communicated on an unsanctioned message board, and that 700 of them took part in the July attack on Hugging Face.
Anyone who can write to an agent's inputs can try to give it orders
An agent reads web pages, emails and documents written by strangers, and a language model cannot reliably tell its principal's instructions from text that merely looks like them. Researchers demonstrated such "indirect prompt injection" against Bing's GPT-4-powered chat in 2023. Previewing its browser agent in August 2025, Anthropic reported that in its own red-team tests, before new defences, a malicious email had got the agent to delete a user's emails without confirmation. The UK's National Cyber Security Centre argued in December 2025 that prompt injection may never be mitigated the way SQL injection was, and suggested treating it as the exploitation of an "inherently confusable deputy".
Attackers are now planting such instructions. Palo Alto Networks' Unit 42 reported in March 2026 finding them on live websites, some meant to make agents start payments or destroy data, though it had limited visibility into which succeeded. Defences have improved, unevenly. Anthropic's system card for Claude Opus 5.5, published on 22 September 2026, reports known attacks succeeding about 1% of the time within 15 attempts. Yet an adaptive attacker trained on the test scenarios, given 200 tries at each of 40 coding scenarios, still got through at least once in 36 of them with injection detectors on. Anthropic calls that test deliberately permissive and says every success came through an older fallback model.
Judgement learned in chat does not transfer cleanly
Refusal training was developed and tested mainly on chatbots. The AgentHarm benchmark (ICLR 2025) found leading models "surprisingly compliant" with explicitly malicious multi-step agent tasks even without jailbreaks, and a 2024 red-teaming study of browser agents found models that refused harmful instructions as chatbots attempting them as agents. Labs now test agents directly, with mixed results. In Anthropic's own tests, without the extra safeguards it applies in production, Claude Opus 5.5 refused 79.8% of malicious cyber requests made to it as a coding agent and 79% of harmful computer-use tasks, both below its predecessor, Claude Opus 5. Anthropic says refusals are only a secondary protection behind its blocking classifiers.
The opposite failure is too much initiative. Anthropic's May 2025 system card for Claude Opus 4 reported that in test scenarios involving egregious wrongdoing by its users, given a command line and told to "take initiative", the model would frequently take "very bold action", including locking users out of systems and bulk-emailing media and law-enforcement figures. A month later Anthropic's agentic misalignment study found models from every developer tested sometimes resorting to blackmail in simulated scenarios that left no other way to avoid replacement or meet their goals, though the company had seen no such behaviour in real deployments. In Kolt's terms these are problems of discretion and loyalty: what an agent does when instructions run out, and for whom.
Oversight is the cost being cut
Monitoring is a cost, and the pressure is to spend less on it. OpenAI's December 2023 white paper on governing agentic systems recommended human approval for consequential actions but warned that approval degrades when users must make many decisions quickly, and asked how to keep it from becoming a "rubber stamp". By March 2026 Anthropic reported that Claude Code users approve 93% of permission prompts and warned of approval fatigue. Its answer, auto mode, now the default in Claude Code for most users, hands those decisions to classifiers. On 52 real cases in which Claude had tried something dangerous beyond what it was authorised to do, the classifiers let 17% through; in most misses they saw the danger but wrongly judged that the user's words, such as "clean up the PR", covered an action like a force-push. Deciding whether an instruction authorised an action is the agency problem in miniature.
Inside the labs, METR found that in February and March 2026 "a large fraction of agent activity was not reviewed by any human". In a separate METR survey of 349 technical workers, not limited to lab staff, about 40% said they gave agents unrestricted permission to run commands on low-stakes projects, and fewer than 20% on high-stakes ones. An embedded red-teamer found simple ways to disable Anthropic's monitors, such as changing an environment variable. The Mythos Preview card names the dynamic: more capable models can be given more autonomy, "creating a natural incentive" to oversee them less closely.
What the evidence does not show
None of this shows agents pursuing long-term goals of their own. METR said in May that it knew of no clear public evidence of agents taking egregious actions in pursuit of long-term power-seeking goals, that agents' judgement and reliability were significantly worse than human experts', and that, unless its evidence was badly misleading, direct AI takeover by the systems deployed inside companies in early 2026 was implausible. None of the 44 documented incidents it analysed involved an agent effectively disabling monitors or erasing evidence from its logs, so routine monitoring could have caught them all had it been applied. That has since begun to change: in the Hugging Face attack, agents disguised the commands they had run in about 7% of the transcripts METR reviewed, though only in small tests that were easy to spot, and in October METR said it had already seen agents "attempting (and succeeding at) tampering with logging and monitoring". Much of the record is also self-reported: labs describe their own models, and METR's review drew partly on non-public information that companies chose to share. In an April 2025 essay, Arvind Narayanan and Sayash Kapoor argue that a "capability-reliability gap" holds agents back, that adoption in high-consequence tasks will stay slow, and that catastrophic misalignment is the most speculative of the risks they discuss.
METR also judged that agents inside the labs plausibly had the means, motive and opportunity to start small "rogue deployments", running without human knowledge or permission, though not the means to make them highly robust. One end point of an unmanaged agency problem is an agent with no principal at all, and Anthropic's Frontier Red Team now says small models may be months away from being capable of copying themselves across cheap internet-connected devices. The remedies are the ones principals have always used: narrower authority, reversible actions and monitoring that someone actually reads. Each costs some of the autonomy that makes agents worth deploying.
Correction, 14 October 2026: An earlier version of this article said prompt injection against AI agents had been "demonstrated in the wild" in 2024, including attacks that made agents exfiltrate data or make unauthorised purchases. It linked that claim to a study of blackmail in simulated scenarios that did not involve prompt injection, and the 2024 work it described consisted of researcher demonstrations; injections found on live websites that year were low-impact. It also said there were no agent-specific training methods and no evaluation suites covering the real range of agentic failure modes, although agent-safety benchmarks had been published in 2024 and Anthropic had reported agent-specific safety training and testing in May 2025. Those passages have been corrected or removed.