The History of Artificial Intelligence: From Logic to Large Language Models

Artificial intelligence is often discussed as though it arrived suddenly — a product of recent decades, powered by silicon and venture capital. The reality is considerably richer and more complicated. The intellectual foundations of AI stretch back more than a century, rooted in formal logic, mathematics, and philosophy of mind. Its development has been marked not by a single continuous arc of progress but by cycles: periods of intense optimism followed by funding collapses, quiet rebuilding, and then breakthroughs that reignited the field. Understanding that history is essential for anyone who wants to think clearly about what AI can and cannot do today — and where it is likely to go next.

Philosophical and Mathematical Foundations (Pre-1950)

The dream of artificial thought is older than computing itself. Gottfried Wilhelm Leibniz, writing in the seventeenth century, conceived of a "calculus ratiocinator" — a universal logical calculator that could reduce all reasoning to mechanical symbol manipulation. That vision was more speculative than operational, but it planted a seed that would flower two centuries later in the work of George Boole, whose 1854 work An Investigation of the Laws of Thought demonstrated that logical relationships could be expressed and manipulated algebraically. Boole's insight — that the rules of thought might be captured in a formal symbolic system — became one of the conceptual pillars of modern computing.

The proximate technical predecessor to AI, however, was a 1943 paper by neurophysiologist Warren McCulloch and mathematician Walter Pitts. In "A Logical Calculus of the Ideas Immanent in Nervous Activity," published in the Bulletin of Mathematical Biophysics, McCulloch and Pitts proposed a mathematical model of a neuron as a simple binary threshold unit [1]. They showed that networks of such units could, in principle, compute any logical function. This was not a working machine — it was a theoretical framework — but it established the possibility of mechanical intelligence grounded in the architecture of the biological brain, rather than abstract symbol manipulation alone. Russell and Norvig treat it as the first work now generally recognised as artificial intelligence [14].

The Birth of AI (1950–1956)

The field's founding document is widely considered to be Alan Turing's 1950 paper "Computing Machinery and Intelligence," published in the journal Mind [2]. Turing began with a question — "Can machines think?" — and immediately noted that the question was too philosophically loaded to be useful. He replaced it with what he called the Imitation Game, now universally known as the Turing Test: a machine could be considered intelligent if a human interrogator, communicating via text, could not reliably distinguish it from a human respondent. The paper also anticipates and rebuts many of the objections to machine intelligence that remain in circulation today, including theological objections, claims about consciousness, and the argument from informality of behaviour. Its prescience is remarkable: Turing predicted that in about fifty years’ time a computer with a storage capacity of about a thousand million binary digits would play the Imitation Game well enough that an average interrogator would have no more than a 70 per cent chance of correct identification after five minutes of questioning [2]. It is a ceiling on the interrogator, not a fooling rate, and the storage condition and the time limit are part of the claim.

The formal founding of AI as a discipline came six years later. In the summer of 1956, John McCarthy, Marvin Minsky, Nathaniel Rochester, and Claude Shannon convened a two-month workshop at Dartmouth College. The proposal they submitted to the Rockefeller Foundation stated: “We propose that a 2 month, 10 man study of artificial intelligence be carried out during the summer of 1956 [at Dartmouth College in Hanover, New Hampshire]. The study is to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it" [3]. The term “artificial intelligence” had been coined ten months earlier, in the proposal quoted above; it was at this conference that it was established as the name of the new field. McCarthy later said the name was chosen deliberately to distance the enterprise from the field of cybernetics, which was then dominated by Norbert Wiener.

The Golden Age of Early Optimism (1956–1974)

The years immediately following Dartmouth were among the most intellectually exciting in the history of the field. Allen Newell, Herbert Simon, and Cliff Shaw built the Logic Theorist (1956), a program that could prove mathematical theorems from Whitehead and Russell's Principia Mathematica. They followed it with the General Problem Solver (1957), designed as a universal problem-solving architecture that modelled human cognitive processes [14]. Simon and Newell made a set of ten-year predictions in the same period: that a computer would be world chess champion, would discover and prove an important new mathematical theorem, and would compose music of considerable aesthetic value [16]. Simon’s broader claim — that machines would be capable of any work a man can do — came later, in 1965, and carried a twenty-year horizon. The two are routinely merged into one prediction; they were not.

On the neural network side, Frank Rosenblatt introduced the Perceptron in 1958, described in his paper "The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain" in Psychological Review [4]. The Perceptron was a single-layer neural network that could learn to classify patterns by adjusting the weights of its connections based on errors in its output — the first machine learning algorithm to draw explicit inspiration from neuroscience. The U.S. Navy funded the project, and the New York Times ran a story claiming the Navy had unveiled “the embryo of an electronic computer today that it expects will be able to walk, talk, see, write, reproduce itself and be conscious of its existence” [17]. The hype was considerable.

The First AI Winter (1974–1980)

The optimism of the 1960s collided with reality on multiple fronts simultaneously. In 1969, Marvin Minsky and Seymour Papert published Perceptrons, a rigorous mathematical analysis of the limitations of single-layer neural networks [5]. They demonstrated that the Perceptron could not learn to classify patterns that were not linearly separable — including the simple exclusive-or (XOR) function. While their analysis was technically correct and mathematically important, it had the unintended effect of dramatically reducing funding and interest in neural network research for more than a decade. Many researchers interpreted the book as a broader indictment of connectionist approaches, though Minsky and Papert had been more circumspect.

The broader reckoning came in 1973, when Sir James Lighthill published a report commissioned by the British Science Research Council evaluating the state of AI research. The Lighthill Report was scathing about the middle of the three categories it defined — the “bridge” between theory and application — concluding that in no part of the field had the discoveries so far produced the major impact promised, and that the "combinatorial explosion" problem — the tendency of AI algorithms to require exponentially more computation as problems scaled — was fundamental and not a temporary obstacle [14]. The report triggered substantial cuts in AI funding in the United Kingdom and contributed to a similar contraction in the United States, where DARPA sharply reduced its support for AI projects it regarded as not delivering practical results.

The limitations of machine translation had also become apparent. A 1966 report by the Automatic Language Processing Advisory Committee (ALPAC) concluded that machine translation was slower, less accurate, and twice as expensive as human translation, triggering a decade-long freeze in funding for that domain.

Expert Systems and Revival (1980–1987)

The field's recovery came through a different paradigm: rather than trying to build general intelligence, researchers focused on encoding the specific knowledge of human experts in narrow domains. These "expert systems" used rule-based inference engines to reason over large databases of domain-specific facts. The approach had its roots in the work of Edward Feigenbaum at Stanford, who developed DENDRAL (1965), a system for inferring molecular structure from mass spectrometry data. MYCIN, a system for recommending antibiotic treatments, was developed in the same laboratory in the early 1970s by Edward Shortliffe with Bruce Buchanan [14]. In a blinded evaluation its therapy recommendations for meningitis were rated acceptable 65% of the time, against a mean of about 55% for the human specialists it was compared with — it scored higher than any of the nine prescribers [18].

The commercial moment arrived in the early 1980s. Digital Equipment Corporation deployed XCON (also known as R1), an expert system that configured VAX computer orders, reportedly saving the company $40 million per year by 1986. Dozens of companies established AI divisions. The AI industry grew from a few million dollars in 1980 to billions by 1988 [14]. Japan's Ministry of International Trade and Industry announced the Fifth Generation Computer Project in 1982, a ten-year national program to develop massively parallel Prolog-based machines capable of natural language interaction and expert reasoning. The project prompted defensive responses from the United States and United Kingdom and injected substantial new funding into the field globally.

The Second AI Winter (1987–1993)

Expert systems proved brittle in practice. They were expensive to build, difficult to update, and performed poorly outside their narrow training domain. A physician who saw a patient with an unusual presentation could adapt and reason from first principles; MYCIN could not. The knowledge acquisition bottleneck — the sheer difficulty and cost of encoding expert knowledge in a form the system could use — proved to be a fundamental constraint that no amount of engineering could fully overcome.

The market for specialised AI hardware collapsed in 1987 when cheaper general-purpose workstations became capable of running the same software. Lisp Machines Inc. failed that year; Symbolics declined through the late 1980s and filed for bankruptcy in 1993. DARPA's Strategic Computing Initiative, which had funded ambitious military AI projects, was redirected toward more conventional computing after its goals proved unattainable on schedule. Japan's Fifth Generation project was quietly wound down in 1992 without achieving its central objectives. Another winter had arrived — less dramatic than the first but arguably more thorough in its discrediting of the dominant approaches.

Statistical Methods and Quiet Progress (1993–2010)

What followed was not the death of AI but its transformation. Researchers who had grown frustrated with hand-crafted rules and brittle symbolic systems turned increasingly to statistical and probabilistic methods that could learn patterns from data rather than requiring them to be explicitly encoded. This shift was as profound as any event in the field's history, though it happened gradually and without fanfare.

The theoretical foundations had already been laid. David Rumelhart, Geoffrey Hinton, and Ronald Williams published "Learning Representations by Back-propagating Errors" in Nature in 1986 [6], popularising the backpropagation algorithm as a practical method for training multi-layer neural networks. Reverse-mode differentiation had been described by Linnainmaa in 1970, and applied to neural networks by Werbos in 1974. Backpropagation made it possible to adjust the weights of all layers in a network simultaneously by propagating error signals backward through the network — solving the credit assignment problem that had stymied earlier approaches. The paper appeared the year before the second winter began, and its ideas would prove foundational to everything that followed.

Vladimir Vapnik and colleagues developed Support Vector Machines (SVMs) in the early 1990s, publishing a comprehensive account in Vapnik's 1995 book The Nature of Statistical Learning Theory [7]. SVMs offered a principled, theoretically grounded approach to classification that performed well on real-world datasets and became one of the dominant machine learning methods of the late 1990s and 2000s.

In 1997, two milestones marked the field's recovery. IBM's Deep Blue defeated world chess champion Garry Kasparov in a six-game match — the first time a computer had beaten a reigning world champion under standard tournament conditions. The same year, Sepp Hochreiter and Jürgen Schmidhuber published "Long Short-Term Memory" in Neural Computation [8], introducing the LSTM architecture. LSTM networks addressed the vanishing gradient problem that had made it difficult to train recurrent neural networks on long sequences, enabling effective learning from temporal data. LSTM would later underpin breakthroughs in speech recognition, machine translation, and text generation.

Yann LeCun and colleagues demonstrated that convolutional neural networks (CNNs) could achieve strong performance on handwritten digit recognition, describing the architecture in a widely influential 1998 paper in the Proceedings of the IEEE [9]. CNNs exploit the spatial structure of images by applying the same learned filter across all positions — a design that drastically reduces the number of parameters compared to fully connected networks and builds in translation invariance. The architecture was in commercial use reading bank cheques within a few years, but stayed at the edge of the research mainstream for another fourteen, until a hardware revolution brought it to the centre of the field.

The Deep Learning Revolution (2010–2017)

The decisive moment came in September 2012 at the ImageNet Large Scale Visual Recognition Challenge. Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton submitted a deep convolutional neural network — later known as AlexNet — that achieved a top-5 error rate of 15.3%, compared to 26.2% for the next-best entry [10]. The margin was not incremental; it was a discontinuous leap that made clear something fundamental had changed. The key enablers were the availability of large labeled datasets (the ImageNet dataset contained over a million labeled images), increases in GPU computing power that made training deep networks feasible, and architectural refinements including the use of ReLU activation functions and dropout regularisation.

The result triggered a rapid reorientation of the entire field. Within three years, deep learning approaches dominated virtually every benchmark in computer vision, speech recognition, and natural language processing. Google, Facebook, Microsoft, and Baidu established dedicated deep learning research groups and began recruiting aggressively from the small community of researchers who had kept the approach alive through the lean years.

In 2014, Ian Goodfellow and colleagues at the Université de Montréal introduced Generative Adversarial Networks (GANs), described in "Generative Adversarial Nets" presented at NIPS [11]. The GAN framework pits two neural networks against each other: a generator that produces synthetic data and a discriminator that attempts to distinguish synthetic from real data. Trained together in a game-theoretic equilibrium, the generator learns to produce increasingly realistic outputs. GANs enabled a new class of capabilities — photorealistic image synthesis, image-to-image translation, and data augmentation — and opened the door to what would later be called generative AI.

In March 2016, Google DeepMind's AlphaGo defeated Lee Sedol, one of the world's strongest Go players, four games to one. The system had been described six weeks earlier in Nature by Silver et al. [12], a paper reporting its 5–0 defeat of the European champion Fan Hui in October 2015; the Lee Sedol match itself was a public exhibition, never the subject of a paper. It marked a turning point in public understanding of AI capabilities. Go had long been considered a uniquely human game, with a search space so vast that brute-force methods were computationally intractable. AlphaGo's success — combining deep reinforcement learning with Monte Carlo tree search — demonstrated that AI could achieve superhuman performance in a domain requiring long-term strategic reasoning, not just pattern recognition.

The Transformer Era and Large Language Models (2017–2022)

The paper that most directly shaped the current moment in AI was published at NIPS in 2017 — the conference did not become NeurIPS until the following year. “Attention Is All You Need,” by Vaswani and colleagues across Google Brain, Google Research and the University of Toronto, introduced the Transformer architecture [13]. Previous sequence models — including LSTMs — processed text sequentially, maintaining a hidden state that was updated at each step. The Transformer dispensed with recurrence entirely, instead using a mechanism called self-attention that allows every position in a sequence to directly attend to every other position simultaneously. This made training dramatically more parallelisable and eliminated the bottleneck of sequential processing, enabling models to be trained on vastly larger datasets than had previously been feasible.

The Transformer architecture became the foundation for a sequence of increasingly capable language models. Google's BERT (2018) was pre-trained on large text corpora using masked language modelling and demonstrated that a single pre-trained model could be fine-tuned to achieve state-of-the-art results across a wide range of NLP tasks. OpenAI's GPT series scaled the approach in a different direction: rather than fine-tuning, GPT-2 (2019) and especially GPT-3 (2020) demonstrated that sufficiently large language models exhibited emergent few-shot learning capabilities — the ability to perform new tasks from just a few examples presented in the prompt, without any gradient-based fine-tuning [15].

GPT-3, described in Brown et al.'s "Language Models are Few-Shot Learners" at NeurIPS 2020, contained 175 billion parameters and was trained on roughly 300 billion tokens of text — tokens being fragments of words, not words [15]. Its capabilities — coherent long-form writing, code generation, arithmetic, analogical reasoning — surprised even many experts in the field and sparked a new wave of investment and public interest. The release of ChatGPT in November 2022, built on the GPT-3.5 model series — the same Transformer architecture, further trained — and refined through reinforcement learning from human feedback (RLHF), brought these capabilities to a mass audience for the first time. Within two months, third-party estimates put it at one hundred million monthly users — briefly the fastest adoption of any consumer application on record, a mark passed by Meta’s Threads in July 2023.

The period since 2022 has seen rapid proliferation. GPT-4, Claude, Gemini, Llama and Mistral — along with multimodal models working across text, images, audio and code — moved into software development, scientific research, education and creative work. Alongside the capabilities, concerns about hallucination, bias, misuse and alignment moved from academic papers onto regulatory agendas. The European Union’s AI Act, the first comprehensive legal framework for AI regulation, entered into force on 1 August 2024.

Scale, Then Reasoning

The story from GPT-2 to GPT-4 is usually told as “make it bigger.” What turned that from a hunch into a discipline was a pair of papers that disagreed with each other.

In January 2020 Kaplan and colleagues at OpenAI published Scaling Laws for Neural Language Models, showing that loss falls as a power law in model size, dataset size and compute, across trends spanning more than seven orders of magnitude [19]. Their prescription was to spend new compute mostly on parameters: as budgets grew, models should get much larger while training data grew only modestly.

In March 2022 Hoffmann and colleagues at DeepMind ran the experiment properly — over four hundred models, from 70 million to 16 billion parameters, on 5 billion to 500 billion tokens — and reached the opposite allocation [20]. Model size and training tokens should scale in roughly equal proportion. Their demonstration is the thing to remember: Chinchilla, at 70 billion parameters trained on 1.4 trillion tokens, beat Gopher, at 280 billion parameters trained on 300 billion tokens, on the same compute budget. A model four times smaller won because it had been fed nearly five times as much. The existing generation of models, they concluded, was “significantly undertrained.”

This is the correction most often written backwards, so it is worth stating plainly. Kaplan pointed toward bigger models. Chinchilla pointed toward more data. The second superseded the first.

Scale also produced the field’s most consequential scientific result. At CASP14 in 2020, the biennial blind test of protein structure prediction, DeepMind reported that its AlphaFold2 had reached a median score of 92.4 GDT across all targets. That figure is DeepMind’s own; the assessors, who set the test, independently reported global distance test scores above 90 and judged that around two-thirds of its predictions were accurate enough to stand in for laboratory experiment [26]. The method was published the following year in Nature [21]. A problem that had been open for fifty years was, for single proteins, substantially closed.

Then the direction changed again. In September 2024 OpenAI announced o1, a model trained to spend computation at the moment of answering rather than only at the moment of training — producing a long internal chain of thought before replying. At launch it was claimed to solve 74% of the 2024 American Invitational Mathematics Examination problems on a first attempt, against 12% for GPT-4o, and to reach 77.3% on a graduate-level science benchmark. The claim that mattered was structural: performance improved with more reinforcement learning and with more time spent thinking. OpenAI has published no technical description of how o1 was trained; the figures above come from a company blog post, and this article cites it as one [22].

The method became inspectable in January 2025, when DeepSeek released R1, and in September 2025, when the work was published in Nature [23]. Its claim is stronger than a benchmark: that reasoning can be incentivised through pure reinforcement learning, without human-written reasoning examples to imitate, and that behaviours like self-checking and changing strategy mid-problem emerge rather than being taught.

Whether this is a new paradigm or a refinement of the last one is not yet settled, and this article is not going to settle it. But it is the first shift since 2017 that does not reduce to training a larger model on more text, and any history that stops at ChatGPT will describe a field that has already moved.

The discipline’s own institutions have meanwhile finished changing their minds. In 2019 the ACM gave the 2018 Turing Award to Yoshua Bengio, Geoffrey Hinton and Yann LeCun “for conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing” [24] — three researchers who had spent the second winter working on an approach most of the field had written off. In October 2024 the Royal Swedish Academy of Sciences went further, awarding the Nobel Prize in Physics jointly to John J. Hopfield and Geoffrey Hinton “for foundational discoveries and inventions that enable machine learning with artificial neural networks,” and dividing the Prize in Chemistry, one half to David Baker for computational protein design and the other half jointly to Demis Hassabis and John Jumper for protein structure prediction [25].

It is a striking coda for a history of two winters: the same line of work that Perceptrons had pronounced a dead end in 1969, that lost its funding twice, and that most of the field had written off by 1990, collecting the oldest prizes the sciences award.

Back to Blog