The ELIZA Effect: Why We Can

A program built to show a trick

Joseph Weizenbaum was a computer scientist at MIT. Between 1964 and 1966 he wrote a program that he called ELIZA. It ran on an IBM 7094 mainframe under the Compatible Time-Sharing System, which let several people use the computer at once from typewriter terminals. Weizenbaum wrote it in MAD-SLIP, a combination of the MAD programming language and a list-processing package he had created, called SLIP.

He named the program after Eliza Doolittle, the flower seller in George Bernard Shaw's play Pygmalion, who is taught to speak in a way that makes her pass as a lady. The name fit. The program was meant to show how a machine could seem to converse without understanding anything. Weizenbaum published a description of it in 1966 in Communications of the ACM.

The DOCTOR script

ELIZA read its behavior from a script, so the same program could play different roles. The best known script was called DOCTOR. It imitated a psychotherapist in the style of Carl Rogers, who taught a method in which the therapist reflects the patient's statements back and avoids offering opinions. The choice was clever. A Rogerian therapist can say "Tell me more" without seeming strange, so the program had no need to know anything about the world.

The program worked by looking for keywords in the user's sentence. When it found one, it applied a rule that took the sentence apart and put it back together as a reply. A user who typed "I am feeling sad today" might be told, "Why do you say you are feeling sad today?" If no keyword matched, the program used a stock phrase, such as "Please go on." It also kept a small memory of earlier remarks so that it could return to them. The whole system was short. It had no model of meaning and no knowledge of what sadness is.

What happened next

Weizenbaum expected that people would see how shallow the program was. They did not. Many users took it seriously and talked to it as if it understood them. They told it personal troubles. Some asked for time alone with it. Weizenbaum recounted that his own secretary, who had watched him write the program and knew it was only following rules, asked him to leave the room so that she could have a private conversation with it.

He also learned that some professionals saw a future in the idea. A few psychiatrists suggested that a program like ELIZA could treat patients at scale. Weizenbaum was alarmed. He had built a demonstration of a trick, and people were proposing to use it for care.

The ELIZA effect

The name "ELIZA effect" now describes our tendency to attribute understanding, intention or feeling to a computer program because of how it behaves. People are social creatures. We respond to language as if a mind produced it, and we fill in the gaps. A program that gives even a few good cues may be credited with much more than it can do.

Researchers treat this as a feature of human psychology, not a failing of foolish people. The people who talked to ELIZA included experts. Knowing how a trick works does not always stop it from working on us.

Weizenbaum's warning

Weizenbaum spent the rest of his career arguing against confusing simulation with understanding. In 1976 he published Computer Power and Human Reason. In it he drew a line between calculation, which computers do well, and judgment, which he believed belongs to people because it grows from experience, responsibility and care. He argued that some tasks, including counseling and judging others, should not be given to machines even if machines could be made to perform them, because they require human beings to take responsibility for each other.

Other scientists disagreed. Some thought he overstated the danger, and some believed that machines would eventually match human judgment. The debate has continued ever since, and it has grown louder as programs have become more fluent.

From a few rules to modern systems

ELIZA's offspring include chatbots built for customer service, games and companionship. Modern language models are far more capable. They draw on enormous amounts of text and can produce coherent answers on almost any topic. They work in a completely different way from ELIZA's keyword rules. But the human response has not changed. Users thank them, argue with them and sometimes grieve when a favorite companion application changes or shuts down.

The lesson of ELIZA is not that conversation with machines is bad. It is that fluent language is not proof of understanding, and that designers carry a responsibility. A system that appears caring may lead people to trust it more than it deserves. Good designs tell users clearly what the system is and where its limits lie.

The code returns

For many years the original program was thought lost, and people knew ELIZA only through later imitations. In 2021 a group of researchers announced that they had found a printout of the original code in Weizenbaum's papers at MIT. They published it, and enthusiasts have since run it on modern computers. The rediscovery let people compare the real program with the many versions that had circulated.

Questions to consider

• Why might people share personal thoughts with a program more easily than with a person?
• What should an assistant tell a user about itself, so that the user does not trust it too much?
• Weizenbaum thought some jobs should never be given to a machine. Do you agree, and which jobs would be on your list?