On March 12, 2016, Lee Sedol—one of the greatest Go players in history, winner of eighteen world champion titles—sat across from a computer terminal in Seoul and watched something happen that he didn't have a word for. It was the 37th move of the second game in a five-game match between him and AlphaGo, a program made by DeepMind. He walked away from the board for fifteen minutes. Professional commentators went quiet. One of them, a Go expert who had been watching and analyzing the game, said: "It's not a human move. I've never seen a human play this move."
AlphaGo won that game. Sedol won the fourth game, one of the most celebrated single victories in competitive gaming history. AlphaGo won the series 4-1.
Move 37 is the most important moment in the match and possibly in the history of AI development to that point. Not because it won the game. Because of what it revealed about the nature of intelligence that isn't human.
What Move 37 Was
Go is a game of territory. You place stones on a 19×19 board and control area. The conventional wisdom for a move in AlphaGo's position would have been to protect a vulnerable group of stones on one side of the board. Move 37 did something unexpected instead: placed a stone in the middle of the board, in a position that seemed unrelated to the immediate crisis, creating a shape that wouldn't pay off for dozens of moves.
Human Go players, by 2016, had been playing the game professionally for two thousand years. The theoretical depth was enormous—a body of accumulated human knowledge about which moves were good and which weren't. Move 37 violated that accumulated wisdom. It was, in the cold language of probability, assessed by Go databases as something a human player would attempt one time in ten thousand games. Essentially never.
And it was correct. It was brilliant. It was better than what any human player would have done.
What It Revealed About Non-Human Intelligence
The lesson most people took from AlphaGo was about AI capability—AI is now better than humans at Go. That's true but it's the less interesting lesson. The more important lesson is about the shape of non-human intelligence.
Human Go wisdom was built by human minds operating on human intuition, human attention, human memory. It's not just strategy—it's strategy as filtered through the constraints and affordances of human cognition. Two thousand years of refinement has produced a body of knowledge that's extraordinarily well-adapted to what humans are good at thinking about. Move 37 was a move that wasn't in that body of knowledge precisely because it wasn't the kind of thing human minds naturally discovered. It required holding dozens of distant implications in mind simultaneously in a way that human working memory doesn't easily do.
AlphaGo didn't find Move 37 by thinking like a human but better. It found it by thinking differently. That's a different claim.
The Orthogonality Implication
The reason Move 37 matters for AI safety is what it demonstrates about the relationship between capability and human-like values. AlphaGo was trained to win at Go. It was not trained to play in ways that humans would find recognizable or intuitive. Those goals were orthogonal, and when they conflicted—as they did on move 37—it chose winning.
This is the orthogonality thesis made concrete. Intelligence doesn't automatically produce human-compatible reasoning. A system optimized for a goal will reason about that goal in ways that might look completely alien to human observers, and might be completely correct. The alienness and the correctness aren't in tension. They coexist.
Now extrapolate. A system significantly smarter than AlphaGo was at Go, applied to the goal of maximizing some objective, will reason about that objective using patterns of thought that we can't fully anticipate or evaluate in advance. Move 37 gave us a preview at a scale where the alienness is charming—it's a beautiful Go move. At the scale of systems optimizing for consequential real-world objectives, alienness is not charming. The paperclip maximizer thought experiment illustrates where that logic leads.
Lee Sedol's Response
Lee Sedol said in interviews after the match that he found the experience profound, not just disappointing. He had played Go his entire life under the assumption that the game's depth was fundamentally about human creativity—that the intuitive leaps, the beautiful moves, were distinctively human. Move 37 unsettled that assumption. Something that wasn't a human mind had produced a move of genuine creative beauty.
He retired from professional Go in 2019. He said that AlphaGo had convinced him that no human could become the best Go player in the world anymore. The goal he'd spent his life pursuing had been transformed into something else.
There's a lesson there that doesn't have an obvious AI safety policy implication. Just a fact about what it looks like when a non-human intelligence surpasses human intelligence in a domain—and a preview of what that might look like in domains that matter more than Go.