A short history of AI

Artificial intelligence is younger than most engineering fields but older than most people assume. Its history is a story of bold predictions, disappointments called "AI winters", quiet progress, and sudden breakthroughs when ideas, data and computing power finally came together. Knowing that history helps you judge today's claims: many current debates, about whether machines can reason, whether bigger is better, and whether progress will stall, have been argued before.

This chapter follows AI from its founding in the 1950s through symbolic AI and expert systems, the long rise of neural networks, the deep learning revolution of the 2010s and the arrival of large language models. It closes with the people and institutions that shaped the field, including the remarkable role Canada has played.

The founding: 1950s

Turing's 1950 paper (Lesson 1) set out the question. The field got its name in 1956, at a summer workshop at Dartmouth College in New Hampshire organized by John McCarthy, Marvin Minsky, Nathaniel Rochester and Claude Shannon. Their proposal stated that "every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it", and suggested that a carefully selected group of scientists could make significant progress in one summer. They were wrong about the timescale by many decades, a pattern that would repeat.

The early years produced real achievements. Allen Newell and Herbert Simon's Logic Theorist proved theorems from Principia Mathematica. Arthur Samuel's checkers program at IBM learned to play better than its creator, and Samuel coined the term "machine learning" in 1959. In 1958 the psychologist Frank Rosenblatt built the perceptron, a single artificial neuron that learned to classify simple patterns by adjusting its weights (you will implement one in Lesson 3). The press predicted machines that would soon walk, talk and see.

Symbolic AI and the first winter

Through the 1960s, the dominant approach was symbolic: represent knowledge with logic and symbols, and solve problems by search. Programs solved algebra word problems, played games and manipulated blocks in simulated worlds. Joseph Weizenbaum's ELIZA (1966) imitated a psychotherapist with simple pattern-matching rules, and some users became emotionally attached to it, which disturbed Weizenbaum.

The trouble came with scale. Methods that worked on toy problems collapsed on real ones because the number of possibilities grew exponentially (the "combinatorial explosion"), and because real-world knowledge was far larger and messier than anyone had estimated. In 1969 Minsky and Seymour Papert's book Perceptrons showed that a single-layer perceptron cannot learn some simple functions, such as exclusive-or (XOR), and interest in neural networks faded. In 1973 the Lighthill Report to the UK government judged that AI had failed to deliver on its promises, and British funding was cut sharply; American funding agencies also pulled back. The first AI winter lasted through much of the 1970s.

Expert systems and the second winter

AI revived in the 1980s with expert systems, programs that captured a human expert's knowledge as hundreds or thousands of if-then rules. MYCIN, developed at Stanford in the 1970s, recommended antibiotics for blood infections about as well as specialists in tests. Digital Equipment Corporation's XCON configured computer orders and was reported to save the company millions of dollars a year. Companies built AI departments, and specialized "Lisp machines" were sold to run AI software.

Expert systems proved expensive to build and brittle to maintain: every new situation needed new rules, the rules interacted in unexpected ways, and the systems could not learn or handle cases their builders had not anticipated. When cheaper general-purpose workstations made Lisp machines obsolete and the promised returns did not arrive, the market collapsed in the late 1980s, starting the second AI winter. For years afterwards, many researchers avoided the term "AI" and described their work as machine learning, pattern recognition or informatics instead.

Quiet progress: statistics, learning and backpropagation

While "AI" was out of fashion, the foundations of today's systems were being laid.

• In 1986 David Rumelhart, Geoffrey Hinton and Ronald Williams popularized backpropagation, an efficient way to train neural networks with hidden layers. Multi-layer networks can learn XOR and much more, answering Minsky and Papert's objection.
• In 1989 Yann LeCun and colleagues at Bell Labs trained convolutional neural networks to read handwritten digits; versions of this system read a large share of checks in the United States in the 1990s.
• Probabilistic methods, such as Bayesian networks and hidden Markov models, gave AI a rigorous way to reason under uncertainty and powered speech recognition.
• Richard Sutton and Andrew Barto developed reinforcement learning, in which an agent learns from rewards and penalties; their textbook appeared in 1998.
• Support vector machines and decision tree ensembles became the workhorses of practical machine learning in the 1990s and 2000s.

Milestones reached the public too. In 1997 IBM's Deep Blue defeated world chess champion Garry Kasparov in a six-game match, mainly through massive search on custom hardware. In 2011 IBM's Watson won the quiz show Jeopardy! against two human champions.

The deep learning revolution: 2012 onward

Neural networks had been around for decades, but three things changed in the late 2000s: huge labeled datasets (ImageNet, built by Fei-Fei Li's team, contained more than 14 million labeled images), graphics processing units (GPUs) that could train networks many times faster than ordinary processors, and better training techniques.

In 2012 a network called AlexNet, built at the University of Toronto by Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton, won the ImageNet image-classification challenge with a top-5 error rate of about 16% (15.3% for its best version), against about 26% for the next-best entry. (Top-5 error counts an image as wrong if the correct label is not among the model's five best guesses.) Within a few years almost every entry was a deep network, and by 2015 the best systems made fewer errors on this benchmark than a human annotator who had trained for the task.

Deep learning then swept through speech recognition, machine translation and game playing. In 2016 DeepMind's AlphaGo defeated Lee Sedol, one of the world's strongest Go players, four games to one. Go had been considered a decade or more away because its search space is vastly larger than chess's; AlphaGo combined deep networks, reinforcement learning and search. In 2020 DeepMind's AlphaFold 2 predicted protein structures with accuracy comparable to laboratory methods, solving a 50-year-old problem in biology.

Transformers and large language models

In 2017 a team at Google published "Attention Is All You Need", introducing the transformer, a neural network architecture built around a mechanism called attention, which lets every word in a sequence draw information directly from every other word. Transformers train efficiently on GPUs and scale well, and they became the basis of nearly all modern language models (AI-340 covers them in detail).

Researchers found that language models trained simply to predict the next word in vast amounts of text kept improving as they were made larger and trained on more data and compute, following surprisingly regular scaling laws. OpenAI's GPT-3 (2020), with 175 billion parameters, could write essays and answer questions from a few examples in its prompt. When OpenAI released ChatGPT on November 30, 2022, a version trained to follow instructions and hold conversations, it became one of the fastest-adopted consumer products in history. Since then, many companies and research groups have released models that work with text, images, audio and video, write and run code, and carry out multi-step tasks as agents.

The scale of training has grown extraordinarily fast. Epoch AI, a research group that tracks these trends, estimates that the compute used to train frontier language models has grown about 5 times per year since 2020, and training costs about 3.5 times per year.

Recognition: the field comes of age

The pioneers of deep learning, once on the fringe, received the field's highest honors. Yoshua Bengio, Geoffrey Hinton and Yann LeCun shared the 2018 A.M. Turing Award, often called the Nobel Prize of computing. In 2024 the Nobel Prize in Physics went to John Hopfield and Geoffrey Hinton for foundational discoveries that enable machine learning with neural networks, and the Nobel Prize in Chemistry was shared by David Baker, and by Demis Hassabis and John Jumper of DeepMind for AlphaFold. Andrew Barto and Richard Sutton received the 2024 Turing Award, announced in 2025, for reinforcement learning.

Several of the same pioneers have also become prominent voices warning about the risks of advanced AI, a debate taken up in Lesson 5 and AI-430.

The American AI ecosystem

The United States has been the center of gravity of AI from the start. The Dartmouth workshop, the first AI laboratories at MIT, Stanford and Carnegie Mellon, and decades of funding from the Defense Advanced Research Projects Agency (DARPA) and the National Science Foundation built the field. DARPA's Grand Challenges for autonomous vehicles in 2004 and 2005 launched the modern self-driving car industry. Bell Labs, IBM, Microsoft Research and later Google, Meta and others turned research into products.

Today most of the world's frontier AI models are developed by American companies, which also design most of the specialized chips used to train them and operate most of the largest data centers. Much of the foundational academic research continues at American universities, often with researchers trained around the world. Federal policy has shifted with each administration, and Lesson 5 describes the current approach.

The field has always been international. Several of the most important contributions came from researchers abroad, including the Canadian groups of Geoffrey Hinton, Yoshua Bengio and Richard Sutton and British groups such as DeepMind, and many of the founders of American AI companies were born or trained in other countries.

Lessons from history

A few patterns recur and are worth carrying forward:

• Predictions about timing have often been wrong in both directions. Early researchers expected human-level AI within a generation; later skeptics said Go was decades away. Treat confident timelines with caution.
• Hype cycles are real. Overpromising led to two funding collapses. Claims should be backed by careful measurement.
• Ideas often wait for data and compute. Neural networks and backpropagation existed for decades before GPUs and large datasets made them practical.
• General methods that scale with computation have tended to beat hand-crafted knowledge over time. Richard Sutton called this "the bitter lesson" in a 2019 essay. It remains debated, but it describes the last decade well.
• Each wave raised new social questions, from Weizenbaum's unease about ELIZA to today's debates on safety, jobs and power.