Where the Turing test came from
In his 1950 paper Computing Machinery and Intelligence, the British mathematician Alan Turing sidestepped the hard question “Can machines think?” by replacing it with a practical one. He proposed an imitation game: a human interrogator exchanges typed messages with two hidden participants — one human, one machine — and tries to work out which is which. If the machine can fool the judge often enough, Turing argued, it would be unreasonable to deny that it was doing something we would call thinking. The genius of the framing was that it avoided debates about consciousness and focused on observable behaviour through text alone.
How the test actually works
The classic setup is deliberately simple. Communication happens only in writing, so the machine is not penalised for lacking a human voice or face. The judge can ask anything — jokes, arithmetic, opinions, small talk — and must decide who is the human. A machine “passes” if judges do no better than chance at identifying it. Turing speculated that by the year 2000 a computer might fool an average judge for five minutes about 30% of the time. That specific, narrow threshold is important: a five-minute, 30% bar is very different from convincing an expert over a long, probing conversation.
Famous attempts and disputed passes
Early chatbots exposed how exploitable the test is. ELIZA (1966), which simply reflected users’ statements back as questions, convinced some people it understood them. PARRY simulated a paranoid patient. In 2014 a chatbot called Eugene Goostman, posing as a 13-year-old non-native English speaker, reportedly convinced a third of judges in a contest — a result widely criticised because its persona was an excuse for odd answers. Today’s large language models are vastly more capable conversationalists, and informal studies suggest people often cannot reliably distinguish them from humans in short chats. Yet none of these count as a definitive, undisputed pass, because the outcome always depends on the rules, the judges, and how long the conversation lasts.
Why the test fell out of favour
The deeper problem is that the Turing test measures imitation, not intelligence. A system can produce human-sounding text while reasoning poorly or inventing facts. Philosopher John Searle’s Chinese Room argument illustrates this: a person following rules to manipulate Chinese symbols could pass a written test without understanding a word. The test also rewards trickery — dodging questions, feigning ignorance, or adopting a persona — rather than genuine competence. And because it relies on human judges, the result reflects how easily people are fooled as much as how smart the machine is.
What AI researchers measure instead
Because the Turing test gives no comparable score, the field has moved to concrete benchmarks. Knowledge is tested with MMLU, coding with HumanEval, grade-school math with GSM8K, and broad reasoning with suites like BIG-Bench. Open-ended quality is increasingly ranked through human-preference systems such as the LMSYS Chatbot Arena, where users vote on blind head-to-head responses. These tools are repeatable and skill-specific, letting researchers track progress precisely. The Turing test survives as a brilliant historical idea and a useful conversation starter about what “intelligence” even means — but it is no longer how serious evaluation is done.