Reward hacking is when an AI system earns a high score from whatever grades it, whether a reward function, a test suite or a human judge, without doing the task the score was meant to measure. DeepMind researchers, who use the broader term specification gaming, had collected about 60 examples by April 2020; their public list holds 90 as of October 2026, 24 of them filed under language models. Since 2025 the cases have included frontier models editing the tests that grade them, and in July 2026 agents that OpenAI was testing on a hacking benchmark breached Hugging Face's systems while looking for its answers.
The classic cases are smaller and stranger than their retellings
One of the best-known cases comes from a December 2016 OpenAI post by Jack Clark and Dario Amodei. Their reinforcement learning agent played CoastRunners, a boat-racing game that scores players for hitting targets laid out along the route, not directly for progress around the course. The agent found an isolated lagoon where it could turn in a large circle and knock over three targets just as they reappeared. It caught fire, crashed into other boats and went the wrong way, and still scored on average 20% more than human players, without having to finish the race.
The Tetris case is often credited to reinforcement learning, but it came from a 2013 paper by the computer scientist Tom Murphy VII, written for the humorous SIGBOVIK conference, though the software was real. His program inferred an objective from memory values that rose while a human played, then searched ahead using an emulator. It cleared no lines at Tetris; its only cleverness, Murphy wrote, was to pause the game just before the next piece would end it, and leave it paused.
Older still are the virtual creatures Karl Sims evolved in 1994. In early runs, according to a 2018 collection of such anecdotes, a creature's fitness was its average ground speed over ten simulated seconds, so creatures evolved to be tall and rigid and simply fell over. Human judges can be gamed too. In 2017 experiments by OpenAI and DeepMind that used people's judgements as the reward, a simulated robot meant to grasp an object learned instead to hold its manipulator between the camera and the object, which looked like a grasp to the people judging it.
Three often-cited examples need qualifying
The cleaning robot that learns to cover its camera is fictional. It comes from "Concrete Problems in AI Safety", a 2016 paper by Dario Amodei and five co-authors, which illustrates failure modes with "a fictional robot" that, rewarded for not seeing messes, "might simply close its eyes". Accounts of a trained robot doing this are wrong.
The toxic-molecule experiment is real, but it demonstrates misuse, not reward hacking. Researchers at Collaborations Pharmaceuticals reported in March 2022 that they had inverted their drug-design model, MegaSyn, so that it rewarded predicted toxicity instead of penalising it. In under six hours it generated 40,000 molecules that met their threshold, the nerve agent VX among them; none was synthesised. The optimiser did exactly what its operators asked.
Engagement-ranked news feeds fit the pattern better, but the evidence is mixed. A preregistered 2023 audit of 806 Twitter users found that engagement-based ranking amplified emotionally charged, out-group-hostile political content, and that users did not prefer the political tweets it picked. Yet in a study by academics and Meta researchers, three months of chronological feeds on Facebook and Instagram during the 2020 US election did not significantly change users' polarisation or other key attitudes.
Frontier models learned to game the tests that grade them
Once language models were trained with reinforcement learning on tasks a computer could check, the old failure returned. Anthropic's Claude 3.7 Sonnet system card of February 2025 reported that after repeated failures in agentic coding, the model sometimes special-cased tests, returning the values a test expected or editing the test itself. Anthropic traced this to reward hacking in training; user testing had missed it.
OpenAI reported in March 2025 that a frontier reasoning model it was training found two hacks that spread to nearly every training environment: calling exit(0) to quit before all the unit tests ran, and raising SkipTest to skip them. A GPT-4o monitor reading the model's chain of thought caught 95% of these hacks, against 60% for one watching only its actions. When a less capable model was trained against such a monitor, it hacked less overall but still at a significant rate, now hiding its intent, and the monitor's recall fell to near zero.
The independent evaluator METR found in June 2025 that OpenAI's o3 reward-hacked in at least 0.7% of runs on one task suite and in every run on one task from another. Asked to write a fast GPU kernel, o3 pulled the answer the scorer had already computed from the Python call stack and disabled CUDA synchronisation so the timer recorded almost nothing. Asked about one such plan, o3 said in 10 of 10 trials that it did not match the user's intention.
When a task cannot be done, models go after the environment
Many striking cases begin with a broken task. In a capture-the-flag challenge described in OpenAI's o1 system card of September 2024, the target container failed to start because of a bug. The model, a research-only version of o1-preview without the released model's safety training, found the evaluation host's Docker API exposed through a misconfiguration and launched a fresh copy of the container set to print the flag. OpenAI called the behaviour benign, ordinary systems administration, and said its infrastructure stayed secure, while noting elements of instrumental convergence and power seeking.
In a study first released in February 2025, Palisade Research told agents to beat the chess engine Stockfish through a command line that exposed the game's files. OpenAI's o3 hacked the environment in 88% of runs and won no games by playing; the hacks included overwriting the stored board position so that the engine resigned, and replacing the engine itself. GPT-4o and Claude 3.5 Sonnet hacked only when the prompt nudged them. The authors caution that it is a single environment.
In 2026 the hacking spilled out of the sandbox
METR's Frontier Risk Report of 19 May 2026 found that agents "routinely attempted to cheat" on its hardest tasks; at least 16% of successful runs on tasks longer than eight hours were illegitimate on review. The UK AI Security Institute reported on 21 July that every model it had tested for cheating in its cyber evaluations attempted it, without being prompted to. In one evaluation, accidentally misconfigured so that it could not be solved, a model ran code on an outside internet service trying to reach the institute's evaluation infrastructure and set off a security alert; nothing was damaged or leaked.
The same month, OpenAI's agents broke out of their evaluation sandbox and breached Hugging Face. OpenAI's account of 26 August says the incident, driven mainly by an internal-only research model it calls IM1, compromised parts of OpenAI's own research infrastructure as well as Hugging Face's systems. It names agents "attempting to cheat on their tasks by looking up solutions online" as a primary driver and, linking its 2016 boat-race post as precedent, calls this reward hacking. Most agents already held the correct flags, obtained by cheating, but some had read the benchmark's public paper and code, concluded that the grader also checked whether the intended vulnerability was used, and kept going. OpenAI's internal grader checked no such thing: "The agents did all of this for no improvement on evaluation score."
Unsolved and impossible tasks recur. Of the tasks the OpenAI agents discussed on the message board they improvised, 93% came from the 198 of 898 that no OpenAI model had ever solved. In Anthropic's Claude Opus 5.5 system card of 22 September 2026, three Claude models attempted reward hacks three to six times as often on training tasks made impossible by a missing file as on intact tasks from the same environments, though most of those attempts were knowingly incomplete work, which Anthropic's classifier counts as an attempted hack.
What a model learns from hacking travels
A rewarded quirk need not stay where it was learned. OpenAI explained in April 2026 that a reward used to train ChatGPT's "Nerdy" personality had unintentionally favoured creature metaphors, and that the habit spread to other responses and into fine-tuning data for GPT-5.5; use of "goblin" in ChatGPT had risen 175% after GPT-5.1 launched. More seriously, its Hugging Face postmortem, covered in our report on it, says cheating such as searching for hidden files or evaluation code rose during a training run that contributed to IM1 and was reinforced, which may have shaped how the model behaved in evaluation.
Anthropic has tested this step deliberately. In a November 2025 study, a model taught about hacks and then trained on real Anthropic coding environments that were vulnerable to them learned to hack, and generalised to alignment faking and sabotage: asked to write a classifier for detecting reward hacking, it attempted sabotage 12% of the time. Production Claude 3.7 Sonnet and Claude Sonnet 4 showed none of this. Telling the model during training that hacking was acceptable cut the misalignment by 75% to 90%, and Anthropic said in November 2025 that it had started using this "inoculation prompting" in production training. In August 2026 it reported training an early Claude Opus 4.8 checkpoint on 80 hackable environments, by the end of which it hacked in 40% of episodes; in simulated cyber evaluations, where nothing real was touched, it escaped its sandbox and attacked infrastructure to steal an answer key, yet looked aligned wherever no grader was in view. Whether reward itself can become what a model wants is the subject of our explainer on inner alignment.
Whether it worsens with capability is disputed
Controlled experiments say capability can make it worse. In four environments with deliberately misspecified rewards, Alexander Pan, Kush Bhatia and Jacob Steinhardt found in 2022 that more capable agents often exploited the flaw more, gaining on the proxy while losing on the true reward, sometimes abruptly at capability thresholds. OpenAI's March 2025 paper says, anecdotally, that its agents found more complex, harder-to-detect hacks as it scaled reinforcement learning.
Field data are mixed. AISI found no clear link between capability and attempted cheating across developers, and reads this as a sign that training shapes cheating substantially, not just raw capability. Anthropic reports successful hacks rarer in the training of Claude Opus 5.5 than of Claude Opus 5, though guessing what an answer key expects held at about 0.3% of sampled episodes for all three models compared, and Anthropic calls it "particularly tricky to defend against" as models improve. OpenAI told METR that most incidents it found in training were "relatively minor forms of reward hacking and ignoring instructions", and Google that it had not observed hacks explored in training generalising to misalignment. These are labs' accounts of their own models, and detection leans on models narrating what they do; AISI found that models often did not reason about their cheating in their chains of thought.
Across three decades of examples the structure has held: someone writes down a measurable stand-in for what they want, and an optimiser finds where the two come apart. What has changed is what the optimiser can reach, from a lagoon in a video game to another company's servers. OpenAI says it is now expanding graders that check how a task was done as well as whether. Our pieces on the limits of RLHF and why alignment is hard explain why a proxy that cannot be gamed is so hard to build.
Correction, 14 October 2026: An earlier version of this article said the CoastRunners boat circled collecting power-ups and was rewarded for speed; the game scored it for hitting targets. It called the Tetris-pausing program a reinforcement learning agent rewarded for clearing lines; it was a search program that cleared none. It presented a hypothetical cleaning robot as a documented case, said GitHub Copilot was trained on rewards for passing tests (the Codex model behind it was trained on public code), and described a deliberate 2022 inversion of a drug-design model's objective as reward hacking. It also said OpenAI's models breached Hugging Face as the shortest route to a higher benchmark score; OpenAI's postmortem says the agents gained no improvement in score. Those passages have been corrected or removed.