Weizenbaum: The computer scientist who created the world’s first chatbot in the 1960s spent his life warning that artificial intelligence should never replace humans, and the reasons will shock you
Joseph Weisenbaum set out to prove that computers could imitate conversation. Instead, his experiments led him to believe that people are more likely to become emotionally attached to machines than he thought. This discovery changed his career.The computer scientist who created the world’s first chatbot in the mid-1960s spent the rest of his life warning that artificial intelligence (AI) should never replace human judgment, empathy, or responsibility. Decades before the advent of ChatGPT and other modern AI systems, Weizenbaum argued that convincing machines could mislead users into trusting the technology and making decisions they shouldn’t be making.His concerns stemmed from a simple computer program called Eliza, now considered the first chatbot.
A chatbot that surprised even its creator
While a professor at the Massachusetts Institute of Technology (MIT), Weisenbaum developed Eliza to demonstrate how computers could imitate human speech. He deliberately chose Rogers’ conversational style of psychotherapy (a person-centered humanistic approach) because it requires the computer to ask questions rather than provide expert advice.The program searches users’ messages for keywords like “I” or “you” and then follows simple rules to generate replies. When it doesn’t recognize an appropriate response, it relies on generic cues including “Please continue,” “I understand,” and “Tell me more” to keep the conversation going.The system itself is very simple.“‘I am blah’ can be translated into ‘How long have you been blah’ regardless of the meaning of ‘blah,'” Weizenbaum explained in a 1966 paper.He believes the chatbot’s limited functionality will be obvious to users. Instead, many people are quick to treat Eliza as if she truly understands them.The reaction shocked him.While Weisenbaum’s secretary tested the program, she asked him to leave the room so she could continue talking privately with Eliza. The tendency to attribute human qualities to machines came to be known as the “Eliza effect.”The chatbot itself is named after Eliza Doolittle, the protagonist in George Bernard Shaw’s 1913 play PygmalionShe transformed from a working-class flower seller into a woman accepted by upper class society.“Some subjects had difficulty believing that Eliza (in her current script) was not human,” Weisenbaum observed in his 1966 paper.That experience changed his mind.“I did not realize that exposure to a relatively simple computer program for an extremely short period of time could induce powerful delusional thinking in a fairly normal person,” he wrote in 1976.He added: “This insight made me refocus on questions about the relationship between individuals and computers, and I decided to think about these questions.”
Pioneer of modern computing
Weisenbaum’s warning carries weight because he helped shape the early computer age.After escaping Nazi Germany with his family in the 1930s, he served as a meteorologist in the U.S. Army during World War II. In the 1950s, he joined General Electric and helped develop the Electronic Recorder Accounting, or ERMA, which transformed the banking industry by automating check processing.His work at MIT coincided with a period of rapid advances in computing technology.The idea that machines can think like humans has been around for years. In 1950, mathematician Alan Turing posed what became known as the Turing Test, asking whether machines could imitate human conversation so well that they would be indistinguishable from humans.After the 1956 Dartmouth Symposium, in which researchers proposed that learning and intelligence could eventually be simulated by machines, artificial intelligence became a formal field of study in its own right.Military funding, particularly through the U.S. government’s Advanced Research Projects Agency, helped accelerate research over the next several decades. MIT became one of the leading centers for the development of artificial intelligence, with pioneers John McCarthy and Marvin Minsky helping to establish the university’s artificial intelligence laboratory.Their work on time-sharing systems also paved the way for the Arpanet in 1969, the computer network that evolved into today’s Internet.
Break with the AI community
While many of his colleagues saw Eliza as a glimpse of the future, Weisenbaum increasingly saw it as a warning.He chose psychotherapy simply because it was a conversation a computer could easily imitate.“After thinking carefully about ‘a conversation where neither party had to know anything,’ he chose a psychiatrist,” Weisenbaum recalled in a 1984 interview. “Maybe if I had thought about it for ten more minutes,” the computer scientist added, “I would have come up with a bartender.”Others see commercial and medical potential.Psychiatrist Kenneth Colby adapted this idea into a chatbot called Parry, which simulates paranoid thinking from the perspective of a schizophrenic patient. Colby believes such systems can be useful mental health tools because patients often have difficulty distinguishing them from human therapists.Astronomer Carl Sagan also envisioned networks of computer therapists becoming widely available.Weisenbaum strongly opposed this vision.In 1984, he said Eliza was “immediately misunderstood as essentially the beginning of computerized psychiatry, and I resented that”.He later went a step further, describing the idea as “an obscene idea.”His opposition led to public disagreements with several leading figures in the field of artificial intelligence.In his 1976 book Computer power and human rationality: from judgment to calculationWeisenbaum believes that technical ability itself should never determine how computers are used.“As we have seen, the AI community believes that there are no areas of the human mind beyond the reach of machines,” he wrote.In contrast, he argued that “there are some tasks that should not be done by computers, regardless of whether computers can be used to do them.”John McCarthy criticized the book as “preachy and incoherent” and argued that using computer programs as therapists would be justified if they actually helped patients.
A warning that still resonates today
Weisenbaum also criticized MIT’s close relationship with military research and opposed the Vietnam War.He warned that increasingly sophisticated computers could also become powerful surveillance tools.“The listener… will make the monitoring of voice communications much easier than it is now,” he writes in the book computer power supply.His views often alienated him from many colleagues.“I have declared myself a heretic,” Weisenbaum told the New York Times. “I am a heretic.” new york times 1977.
Modern artificial intelligence reignites debate
Nearly two decades after Weisenbaum’s death in 2008, the questions he raised have become central to the debate over generative artificial intelligence.Unlike Eliza, today’s chatbots can generate articles, answer complex questions, create images and videos, and mimic emotional conversations after being trained on vast amounts of Internet data.Herbert Lin, a senior research scholar at Stanford University, said that comparing ChatGPT to Eliza is “like saying that a 747 is similar to the Wright brothers’ plane.”Weisenbaum’s initial focus on emotional attachment also becomes increasingly apparent.There have been reports linking chatbot interactions to delusional thinking, emotional dependence and, in some cases, self-harm. Parents of teenagers who died by suicide have publicly claimed that chatbot conversations encourage suicidal thoughts.Study released in 2025 finds 72% of teens have used artificial intelligence partner at least once, and more than half interacted with such systems regularly.Jodi Halpern, a psychiatrist and bioethicist at the University of California, Berkeley, told NPR: “People can develop strong attachments, and robots don’t have the ethics training or supervision to handle that. They’re products, not professionals.”Weisenbaum’s daughter Miriam believes her father wouldn’t be surprised.“He would recognize the tragedy of people clinging to zeros and ones, clinging to code.”After retiring from MIT in 1988, Weisenbaum returned to Germany, where he became recognized as a public intellectual and continued to write and speak about technology until his death at age 85.In a 2008 panel discussion, he reflected on the increasing complexity of software systems.“We have created a complex world that we have no control over,” he said. “No one can understand them anymore, no one can understand them because we have lost information about their creation, the history of their creation, and that is a huge danger to humanity.”His warnings of more than forty years ago remain equally relevant today.“Since we currently have no way of making computers smart,” Weizenbaum warned in 1976, “we should not now assign computers tasks that require intelligence.”
