A short history of AI

Artificial intelligence is younger than most engineering fields but older than most people assume. Its history is a story of bold predictions, disappointments called "AI winters", quiet progress, and sudden breakthroughs when ideas, data and computing power finally came together. Knowing that history helps you judge today's claims: many current debates, about whether machines can reason, whether bigger is better, and whether progress will stall, have been argued before.

This chapter follows AI from its founding in the 1950s through symbolic AI and expert systems, the long rise of neural networks, the deep learning revolution of the 2010s and the arrival of large language models. It closes with the people and institutions that shaped the field, including the remarkable role Canada has played.

The founding: 1950s

Turing's 1950 paper (Lesson 1) set out the question. The field got its name in 1956, at a summer workshop at Dartmouth College in New Hampshire organized by John McCarthy, Marvin Minsky, Nathaniel Rochester and Claude Shannon. Their proposal stated that "every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it", and suggested that a carefully selected group of scientists could make significant progress in one summer. They were wrong about the timescale by many decades, a pattern that would repeat.

The early years produced real achievements. Allen Newell and Herbert Simon's Logic Theorist proved theorems from Principia Mathematica. Arthur Samuel's checkers program at IBM learned to play better than its creator, and Samuel coined the term "machine learning" in 1959. In 1958 the psychologist Frank Rosenblatt built the perceptron, a single artificial neuron that learned to classify simple patterns by adjusting its weights (you will implement one in Lesson 3). The press predicted machines that would soon walk, talk and see.

Symbolic AI and the first winter

Through the 1960s, the dominant approach was symbolic: represent knowledge with logic and symbols, and solve problems by search. Programs solved algebra word problems, played games and manipulated blocks in simulated worlds. Joseph Weizenbaum's ELIZA (1966) imitated a psychotherapist with simple pattern-matching rules, and some users became emotionally attached to it, which disturbed Weizenbaum.

The trouble came with scale. Methods that worked on toy problems collapsed on real ones because the number of possibilities grew exponentially (the "combinatorial explosion"), and because real-world knowledge was far larger and messier than anyone had estimated. In 1969 Minsky and Seymour Papert's book Perceptrons showed that a single-layer perceptron cannot learn some simple functions, such as exclusive-or (XOR), and interest in neural networks faded. In 1973 the Lighthill Report to the UK government judged that AI had failed to deliver on its promises, and British funding was cut sharply; American funding agencies also pulled back. The first AI winter lasted through much of the 1970s.

Expert systems and the second winter

AI revived in the 1980s with expert systems, programs that captured a human expert's knowledge as hundreds or thousands of if-then rules. MYCIN, developed at Stanford in the 1970s, recommended antibiotics for blood infections about as well as specialists in tests. Digital Equipment Corporation's XCON configured computer orders and was reported to save the company millions of dollars a year. Companies built AI departments, and specialized "Lisp machines" were sold to run AI software.

Expert systems proved expensive to build and brittle to maintain: every new situation needed new rules, the rules interacted in unexpected ways, and the systems could not learn or handle cases their builders had not anticipated. When cheaper general-purpose workstations made Lisp machines obsolete and the promised returns did not arrive, the market collapsed in the late 1980s, starting the second AI winter. For years afterwards, many researchers avoided the term "AI" and described their work as machine learning, pattern recognition or informatics instead.

Quiet progress: statistics, learning and backpropagation

While "AI" was out of fashion, the foundations of today's systems were being laid.

• In 1986 David Rumelhart, Geoffrey Hinton and Ronald Williams popularized backpropagation, an efficient way to train neural networks with hidden layers. Multi-layer networks can learn XOR and much more, answering Minsky and Papert's objection.
• In 1989 Yann LeCun and colleagues at Bell Labs trained convolutional neural networks to read handwritten digits; versions of this system read a large share of cheques in the United States in the 1990s.
• Probabilistic methods, such as Bayesian networks and hidden Markov models, gave AI a rigorous way to reason under uncertainty and powered speech recognition.
• Richard Sutton and Andrew Barto developed reinforcement learning, in which an agent learns from rewards and penalties; their textbook appeared in 1998.
• Support vector machines and decision tree ensembles became the workhorses of practical machine learning in the 1990s and 2000s.

Milestones reached the public too. In 1997 IBM's Deep Blue defeated world chess champion Garry Kasparov in a six-game match, mainly through massive search on custom hardware. In 2011 IBM's Watson won the quiz show Jeopardy! against two human champions.

The deep learning revolution: 2012 onward

Neural networks had been around for decades, but three things changed in the late 2000s: huge labelled datasets (ImageNet, built by Fei-Fei Li's team, contained more than 14 million labelled images), graphics processing units (GPUs) that could train networks many times faster than ordinary processors, and better training techniques.

In 2012 a network called AlexNet, built at the University of Toronto by Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton, won the ImageNet image-classification challenge with a top-5 error rate of about 16% (15.3% for its best version), against about 26% for the next-best entry. (Top-5 error counts an image as wrong if the correct label is not among the model's five best guesses.) Within a few years almost every entry was a deep network, and by 2015 the best systems made fewer errors on this benchmark than a human annotator who had trained for the task.

Deep learning then swept through speech recognition, machine translation and game playing. In 2016 DeepMind's AlphaGo defeated Lee Sedol, one of the world's strongest Go players, four games to one. Go had been considered a decade or more away because its search space is vastly larger than chess's; AlphaGo combined deep networks, reinforcement learning and search. In 2020 DeepMind's AlphaFold 2 predicted protein structures with accuracy comparable to laboratory methods, solving a 50-year-old problem in biology.

Transformers and large language models

In 2017 a team at Google published "Attention Is All You Need", introducing the transformer, a neural network architecture built around a mechanism called attention, which lets every word in a sequence draw information directly from every other word. Transformers train efficiently on GPUs and scale well, and they became the basis of nearly all modern language models (AI-340 covers them in detail).

Researchers found that language models trained simply to predict the next word in vast amounts of text kept improving as they were made larger and trained on more data and compute, following surprisingly regular scaling laws. OpenAI's GPT-3 (2020), with 175 billion parameters, could write essays and answer questions from a few examples in its prompt. When OpenAI released ChatGPT on November 30, 2022, a version trained to follow instructions and hold conversations, it became one of the fastest-adopted consumer products in history. Since then, many companies and research groups have released models that work with text, images, audio and video, write and run code, and carry out multi-step tasks as agents.

The scale of training has grown extraordinarily fast. Epoch AI, a research group that tracks these trends, estimates that the compute used to train frontier language models has grown about 5 times per year since 2020, and training costs about 3.5 times per year.

Recognition: the field comes of age

The pioneers of deep learning, once on the fringe, received the field's highest honours. Yoshua Bengio, Geoffrey Hinton and Yann LeCun shared the 2018 A.M. Turing Award, often called the Nobel Prize of computing. In 2024 the Nobel Prize in Physics went to John Hopfield and Geoffrey Hinton for foundational discoveries that enable machine learning with neural networks, and the Nobel Prize in Chemistry was shared by David Baker, and by Demis Hassabis and John Jumper of DeepMind for AlphaFold. Andrew Barto and Richard Sutton received the 2024 Turing Award, announced in 2025, for reinforcement learning.

Several of the same pioneers have also become prominent voices warning about the risks of advanced AI, a debate taken up in Lesson 5 and AI-430.

Canada's role

Canada had an outsized role in the deep learning story, largely because of patient public funding of research that was unfashionable elsewhere. The Canadian Institute for Advanced Research (CIFAR) funded a program on neural computation from 2004, led by Geoffrey Hinton at the University of Toronto, which kept a small international community of neural network researchers working together through the years when the approach was out of favour. Yoshua Bengio built a leading group at the Université de Montréal, and Richard Sutton, one of the founders of reinforcement learning, moved to the University of Alberta in 2003.

In 2017 Canada became the first country to launch a national AI strategy, the Pan-Canadian Artificial Intelligence Strategy, administered by CIFAR. It funds three national AI institutes: Amii (the Alberta Machine Intelligence Institute) in Edmonton, Mila in Montréal and the Vector Institute in Toronto. Canada is also home to the Canadian AI Safety Institute, launched in 2024, and in June 2026 the federal government launched a new national strategy, "AI for All", which Lesson 5 describes.

Talent flowed in both directions. Many Canadian-trained researchers went on to found or lead major AI companies and laboratories in the United States and Europe, including OpenAI co-founder Ilya Sutskever, a former student of Hinton's. Keeping talent and building companies at home is a recurring theme of Canadian AI policy.

Lessons from history

A few patterns recur and are worth carrying forward:

• Predictions about timing have often been wrong in both directions. Early researchers expected human-level AI within a generation; later skeptics said Go was decades away. Treat confident timelines with caution.
• Hype cycles are real. Overpromising led to two funding collapses. Claims should be backed by careful measurement.
• Ideas often wait for data and compute. Neural networks and backpropagation existed for decades before GPUs and large datasets made them practical.
• General methods that scale with computation have tended to beat hand-crafted knowledge over time. Richard Sutton called this "the bitter lesson" in a 2019 essay. It remains debated, but it describes the last decade well.
• Each wave raised new social questions, from Weizenbaum's unease about ELIZA to today's debates on safety, jobs and power.