Back to series
· 12 min read

The History of Artificial Intelligence: From Symbolic AI to Generative AI


I. The Development of Artificial Intelligence

Artificial Intelligence (AI) is not a technology that emerged only in recent years.

Starting in the 1950s, artificial intelligence went through decades of exploration, and major technical routes such as symbolism, connectionism, statistical machine learning, deep learning, and generative artificial intelligence emerged one after another.

Every technological advance has been closely related to computing power, data scale, algorithmic innovation, and application demands.

From the perspective of technological evolution, the development of artificial intelligence can be roughly divided into the following stages:

  1. Symbolism stage: Simulate human reasoning through logical rules and knowledge representation.
  2. Early connectionism stage: Explore machine learning capabilities through artificial neural networks.
  3. Expert systems stage: Transform domain expert knowledge into rule-based systems.
  4. Statistical machine learning stage: Learn patterns from data and reduce reliance on manually crafted rules.
  5. Deep learning stage: Use multilayer neural networks to automatically learn complex features.
  6. Generative AI stage: Models begin to generate text, images, audio, video, and code at scale.

It should be noted that these stages are not strictly replacement relationships. Different technical routes coexisted for a long time and played roles in different application scenarios.

II. The Birth of Artificial Intelligence: The Era of Symbolism

2.1 The Proposal of the Concept of Artificial Intelligence

As an independent research field, artificial intelligence usually takes the 1956 Dartmouth Conference as an important starting point.

In 1955, researchers such as John McCarthy, Marvin Minsky, Nathaniel Rochester, and Claude Shannon proposed a project plan for artificial intelligence research.

In the summer of 1956, researchers held a workshop at Dartmouth College in the United States, formally advancing the development of the research field of Artificial Intelligence.

The core idea at the time was:

Can machines be made to simulate behaviors such as learning, reasoning, and problem solving by precisely describing the processes of human intelligent activity?

Early researchers had high expectations for artificial intelligence, hoping to use mathematical logic, symbolic representation, and search algorithms to give machines human-like reasoning abilities.

2.2 What Is Symbolism?

Symbolism is also called logicism or knowledge-based artificial intelligence.

Its core idea is:

Represent objects, concepts, relationships, and rules in the real world as symbols, then process problems through logical reasoning.

For example, a simple logical reasoning system:

  • All humans are mortal.
  • Socrates is a human.
  • Therefore, Socrates is mortal.

A machine can convert this knowledge into logical expressions and derive conclusions according to rules.

Its basic working process can be abstracted as:

flowchart TB
    A["Real-world problem"] --> B["Knowledge representation"]
    B --> C["Symbols and logical rules"]
    C --> D["Inference engine"]
    D --> E["Derived result"]

    classDef input fill:#E8F3EC,stroke:#79A88A,color:#315B42
    classDef process fill:#EAF2F8,stroke:#7FA9C7,color:#31566F
    classDef rule fill:#F8F0DF,stroke:#D8B76C,color:#765B24
    classDef output fill:#EEEAF7,stroke:#A696C8,color:#51436F

    class A input
    class B,D process
    class C rule
    class E output

Symbolist systems mainly rely on the following technologies:

Technology Function
Knowledge representation Convert real-world knowledge into symbols that machines can process
Predicate logic Describe objects, properties, and relationships between objects
Rule systems Use IF-THEN rules to express knowledge
Search algorithms Find solutions in a possible state space
Automated reasoning Derive new conclusions from existing knowledge

2.3 Early Representative Achievements

Symbolism advanced the development of early artificial intelligence research.

Logic Theorist (1956)

Developed with the participation of Allen Newell, Herbert A. Simon, Cliff Shaw, and others, it was a representative early artificial intelligence program.

It could prove some mathematical logic theorems through heuristic search, demonstrating the possibility of machines using symbols and rules to reason.

General Problem Solver (development began in 1957)

Also promoted by Newell, Simon, and others, it attempted to establish a general solving mechanism capable of handling multiple types of problems.

It represented problems as states and used operation rules to find a path from the initial state to the goal state.

ELIZA (1966)

An early natural language dialogue program developed by Joseph Weizenbaum.

It simulated a psychotherapist's conversational style through pattern matching and rule substitution. Although it did not possess true language understanding ability, it demonstrated the early possibility of natural language interaction.

2.4 Limitations of Symbolism

Symbolism performed relatively well on problems involving logical reasoning and clear rules, but it had obvious limitations when facing an open, complex, and uncertain real world.

Main problems include:

  1. Difficulty in knowledge acquisition: A large amount of knowledge must be manually organized, defined, and entered.
  2. Complex rule maintenance: As the number of rules increases, conflicts may arise between rules.
  3. Lack of automatic learning capability: Systems usually cannot automatically extract knowledge directly from massive amounts of data.
  4. Difficulty handling fuzzy information: Language, images, and sound in the real world are highly uncertain.
  5. Limited generalization ability: Systems often struggle to handle new problems that are not explicitly covered by rules.

The core problem of symbolicism is that it is very difficult for humans to fully translate all knowledge and reasoning processes in the real world into explicit rules.

This prompted researchers to explore another route: letting machines learn automatically from data.

III. The Exploration of Connectionism: Artificial Neural Networks

3.1 What Is Connectionism?

Connectionism holds that intelligence can arise from the connections and coordinated computation among a large number of simple computational units.

It draws on the structure of biological nervous systems and abstracts the computational process as connections between artificial neurons.

Unlike symbolicism, which relies on manually defined rules, connectionism hopes to allow models to learn patterns from data by adjusting the connection weights in the network.

Its core idea is:

Instead of directly telling the machine how to solve each problem, let the machine gradually learn methods for solving problems through data and training.

3.2 Perceptron: An Early Neural Network Model

In 1957, Frank Rosenblatt proposed the perceptron.

The perceptron is an important representative of early artificial neural networks and can perform simple classification tasks by adjusting weights through training.

The computation process of a single-layer perceptron can be expressed as:

y=f\left(\sum_{i=1}^{n}w_ix_i+b\right)

where:

  • x_i: input feature.
  • w_i: weight corresponding to the input.
  • b: bias.
  • f: activation function.
  • y: model output.

The perceptron determines the classification result through weights and bias.

For example, a perceptron can be used to perform binary classification on data in a two-dimensional plane.

However, a single-layer perceptron can only solve linearly separable problems and cannot correctly handle the XOR (exclusive OR) problem.

3.3 The XOR Problem and the Development of Neural Networks

XOR is an important problem in early neural network research.

Its input-output relationship is as follows:

Input A Input B Output
0 0 0
0 1 1
1 0 1
1 1 0

This problem cannot be classified with a single straight line, so a single-layer perceptron cannot directly solve it.

This problem exposed the capability boundary of early single-layer neural networks and also prompted researchers to explore multilayer neural networks.

Multilayer networks can learn more complex decision boundaries through hidden layers and nonlinear activation functions.

flowchart LR
    subgraph I["Input layer"]
        A["Input A"]
        B["Input B"]
    end

    subgraph H["Hidden layer"]
        C["Neuron 1"]
        D["Neuron 2"]
    end

    subgraph O["Output layer"]
        E["Classification result"]
    end

    A --> C
    A --> D
    B --> C
    B --> D
    C --> E
    D --> E

    classDef input fill:#E8F3EC,stroke:#79A88A,color:#315B42
    classDef hidden fill:#EAF2F8,stroke:#7FA9C7,color:#31566F
    classDef output fill:#EEEAF7,stroke:#A696C8,color:#51436F

    class A,B input
    class C,D hidden
    class E output

3.4 Backpropagation: A Key Breakthrough in Neural Network Training

In 1986, David Rumelhart, Geoffrey Hinton, Ronald Williams, and other researchers published an important paper on the backpropagation learning process, which promoted the application of the backpropagation algorithm in the training of multilayer neural networks.

Backpropagation calculates the gradients of the loss function with respect to the network parameters through the chain rule.

The training process mainly includes:

  1. Forward propagation: compute the model's prediction results.
  2. Loss computation: measure the difference between the prediction results and the true results.
  3. Backpropagation: compute the gradients of the parameters at each layer.
  4. Parameter update: use an optimization algorithm to adjust the model parameters.

The training objective of a neural network can be abstracted as:

\theta^*=\arg\min_{\theta}L(\theta)

where:

  • \theta: model parameters.
  • L(\theta): the model loss function.
  • \theta^*: a set of parameters that makes the loss function as small as possible.

Backpropagation solves the problem of gradient computation, while parameter updates are usually handled by gradient descent and its variants. Together, the two form the foundation of modern neural network training.

IV. The First AI Winter: The Gap Between Technical Ideals and Reality

4.1 What Is an AI Winter?

An AI winter refers to a period in which AI research and industry development fall into a slump.

Early researchers held very high expectations for artificial intelligence, but the computing power, data scale, and algorithmic capabilities of the time were all insufficient to support complex intelligent systems.

As actual results fell short of expectations, research funding and industry investment gradually declined.

In the history of artificial intelligence, the following two periods are usually regarded as important AI winters:

  • The mid-to-late 1970s: early AI research hit a bottleneck.
  • The late 1980s to the early 1990s: expert systems ran into difficulties in commercialization, and related investment declined.

4.2 The Causes of the First AI Winter

Cause 1: Insufficient computing power

Computers at the time had limited processing capability, and complex search and neural network training were costly.

Cause 2: Insufficient data resources

Large-scale digitized data was lacking, making it difficult for machines to learn complex patterns from rich samples.

Cause 3: Limited algorithmic capability

Early systems struggled to handle problems in complex, uncertain, and open environments.

Cause 4: Overly optimistic research goals

Some researchers underestimated the complexity of natural language understanding, visual recognition, and general reasoning.

The AI winter did not mean that AI research stopped; rather, it pushed the research direction away from idealized general intelligence toward more concrete, verifiable problems.

V. The Rise of Expert Systems: The Era of Knowledge Engineering

5.1 What Is an Expert System?

From the 1970s to the 1980s, expert systems became an important research direction in artificial intelligence.

By converting the experience and knowledge of domain experts into rules, expert systems simulate the judgment process of experts in a specific domain.

They do not pursue general intelligence, but focus on one specific domain.

For example:

  • Medical diagnosis.
  • Chemical analysis.
  • Equipment fault diagnosis.
  • Industrial production decisions.
  • Financial risk assessment.

5.2 The Core Architecture of Expert Systems

A typical expert system usually contains the following components:

Component Function
Knowledge base Stores domain knowledge and rules
Inference engine Performs logical reasoning according to the rules
Working memory Stores the facts and intermediate results of the current problem
Knowledge acquisition module Converts expert knowledge into system rules
Explanation module Explains the basis on which the system reaches its conclusions

Its working process is as follows:

flowchart TB
    A["User inputs a problem"] --> B["Fact extraction"]
    B --> C["Working memory"]
    C --> D["Inference engine"]
    E["Knowledge base"] --> D
    D --> F["Rule matching and inference"]
    F --> G["Output diagnosis or decision"]
    G --> H["Explain the reasoning basis"]

    classDef input fill:#E8F3EC,stroke:#79A88A,color:#315B42
    classDef storage fill:#F8F0DF,stroke:#D8B76C,color:#765B24
    classDef process fill:#EAF2F8,stroke:#7FA9C7,color:#31566F
    classDef output fill:#EEEAF7,stroke:#A696C8,color:#51436F

    class A input
    class E,C storage
    class B,D,F process
    class G,H output

5.3 Representative Expert Systems

DENDRAL

An expert system whose development began in the 1960s, used to help infer the molecular structures of organic compounds from mass spectrometry data; it is a representative of early knowledge-based expert systems.

MYCIN

A medical expert system developed at Stanford University in the 1970s, used to help identify bacterial infections and provide recommendations for antibiotic treatment.

MYCIN used rules and uncertainty-handling mechanisms, demonstrating the value of expert systems in specialized domains.

5.4 The Limitations of Expert Systems

Expert systems achieved practical results in specific domains, but they also had obvious problems:

  • Knowledge depended on manual input by experts, making construction costly.
  • The number of rules was enormous, making maintenance difficult.
  • When domain knowledge changed, manual updates were required.
  • They lacked the ability to handle situations not covered by the rules.
  • It was not easy to transfer knowledge from one domain to another.

The rise of expert systems proved the commercial value of knowledge engineering, but it also exposed the cost problem of manually building knowledge systems.

VI. Statistical Machine Learning: From Manual Rules to Data-Driven Approaches

6.1 A Shift in the Technical Paradigm

Starting in the 1990s, with improvements in computing power, the development of the Internet, and the growth of data resources, statistical machine learning gradually became an important technical route in artificial intelligence.

Unlike symbolism and expert systems, statistical machine learning no longer requires researchers to write rules for every situation; instead, it learns statistical patterns from training data.

For example, a traditional rule-based system might require manually defining:

  • Which words indicate spam.
  • Which email features need to be checked closely.
  • What conditions must be met for an email to be classified as spam.

Machine learning can use a large number of labeled email samples to train a classification model, allowing the model to automatically learn the classification boundary.

6.2 Basic Workflow of Statistical Machine Learning

flowchart TB
    A["Raw data"] --> B["Data cleaning and preprocessing"]
    B --> C["Feature extraction"]
    C --> D["Training dataset"]
    D --> E["Model training"]
    E --> F["Model evaluation"]
    F --> G["Deployment and prediction"]

    classDef data fill:#E8F3EC,stroke:#79A88A,color:#315B42
    classDef process fill:#EAF2F8,stroke:#7FA9C7,color:#31566F
    classDef model fill:#F8F0DF,stroke:#D8B76C,color:#765B24
    classDef output fill:#EEEAF7,stroke:#A696C8,color:#51436F

    class A,D data
    class B,C process
    class E,F model
    class G output

6.3 Representative Algorithms of Statistical Machine Learning

Algorithm Core idea Typical applications
Linear regression Establish a linear relationship between input features and a continuous variable House price prediction
Logistic regression Use a probabilistic model for classification Risk classification
Decision tree Build a decision structure based on feature conditions User classification
Random forest Combine multiple decision trees for prediction Risk assessment
Support vector machine (SVM) Find a classification hyperplane with a large margin Text classification
K-Means Cluster based on sample similarity User segmentation
Naive Bayes Perform probabilistic classification based on Bayes' theorem Spam detection
Hidden Markov model (HMM) Perform probabilistic modeling on sequences with hidden states Speech recognition

6.4 The Importance of Feature Engineering

Traditional machine learning usually requires manual feature design (Feature Engineering).

For example, in a spam classification task, researchers may need to extract:

  • The number of keywords in the email subject.
  • Suspicious words in the body.
  • The number of links.
  • Sender reputation.
  • Email length.

The model uses these features for learning.

This means that model performance depends not only on the algorithm but also heavily on feature design and data quality.

An important characteristic of traditional machine learning is:

Features are usually designed manually, and the model is responsible for learning the relationship between features and the target.

This is clearly different from later deep learning.

Seven. The Deep Learning Revolution: Letting Machines Learn Features Automatically

7.1 The Rise of Deep Learning

Deep learning is a machine learning method based on multi-layer neural networks.

It extracts data features layer by layer through multiple network layers, learning complex representations from raw input.

Compared with traditional machine learning, deep learning reduces reliance on manual feature engineering.

For example, in image recognition tasks:

  • Shallow layers may learn edges and textures.
  • Middle layers may learn local shapes and structures.
  • Deep layers may learn object and semantic features.

Deep learning can integrate feature extraction and task prediction into the same model through end-to-end training.

7.2 Key Milestones in the Development of Deep Learning

2006: Research on Deep Belief Networks

Geoffrey Hinton and other researchers advanced training methods for deep neural networks, rekindling academic attention to deep learning.

2012: AlexNet

AlexNet, proposed by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton, achieved a significant advantage in the ImageNet image classification competition.

It used deep convolutional neural networks (CNN), GPU acceleration, ReLU, and other technologies, driving the rapid development of deep learning in computer vision.

2014: GAN

Ian Goodfellow et al. proposed the Generative Adversarial Network (GAN).

GAN learns data distributions through adversarial training between a generator and a discriminator, advancing the development of image generation technology.

2016: AlphaGo

AlphaGo, developed by DeepMind, defeated Lee Sedol, one of the world's top Go players, demonstrating the powerful capabilities of combining deep neural networks with Monte Carlo tree search, reinforcement learning, and other technologies.

2017: Transformer

The Google research team published the paper "Attention Is All You Need", proposing the Transformer architecture.

Transformer uses self-attention mechanisms to process sequence data, significantly changing the direction of development in natural language processing and also becoming an important technical foundation for subsequent large language models.

7.3 Key Technologies of Deep Learning

Technology Core Function Typical Application
CNN Extract local spatial features Image recognition
RNN Process sequential information Early language models
LSTM Improve recurrent networks' ability to model long-term dependencies Speech recognition
GAN Learn data distributions through adversarial training Image generation
Autoencoder Learn compressed representations and reconstruction of data Dimensionality reduction, anomaly detection
Transformer Use attention mechanisms to model sequence relationships Large language models

7.4 Why Has Deep Learning Been Able to Achieve Breakthroughs?

The development of deep learning did not rely solely on algorithmic innovation; rather, it was the result of multiple conditions maturing together.

First, growth in data scale

The Internet, mobile devices, and digital businesses have generated massive amounts of data usable for training.

Second, improved GPU computing power

Deep neural networks involve many matrix operations, and GPUs' parallel computing power can significantly improve training efficiency.

Third, continuous algorithm improvements

Technologies such as backpropagation, activation functions, optimizers, normalization, and network architectures have continued to advance.

Fourth, expansion of model scale

Larger models, richer data, and more computing resources enable neural networks to learn more complex features and patterns.

The development of deep learning demonstrates that machines can automatically learn complex representations through training on large-scale data, without having to rely entirely on manually defined features.

VIII. The Era of Transformers and Large Language Models

8.1 The Emergence of the Transformer

In 2017, a Google research team published a paper:

Attention Is All You Need

This paper proposed the Transformer architecture, which became an important foundation for the development of modern large language model technology.

The core innovation of Transformer is the self-attention mechanism (Self-Attention).

Traditional RNNs rely on sequence order to process data step by step, whereas Transformer can directly compute the associations between different positions in a sequence through the attention mechanism.

This makes it easier for the model to perform parallel computation and also enhances its ability to model long-range dependencies.

8.2 The Core Structure of the Transformer

The original Transformer consists of two parts: Encoder and Decoder.

flowchart TB
    A["Input Sequence"] --> B["Embedding"]
    B --> C["Positional Encoding"]
    C --> D["Encoder"]
    D --> E["Decoder"]
    F["Target Sequence Input"] --> E
    E --> G["Linear Mapping"]
    G --> H["Softmax"]
    H --> I["Output Probability Distribution"]

    classDef input fill:#E8F3EC,stroke:#79A88A,color:#315B42
    classDef embed fill:#F8F0DF,stroke:#D8B76C,color:#765B24
    classDef network fill:#EAF2F8,stroke:#7FA9C7,color:#31566F
    classDef output fill:#EEEAF7,stroke:#A696C8,color:#51436F

    class A,F input
    class B,C embed
    class D,E network
    class G,H,I output

Important components of Transformer include:

  • Embedding: Converts tokens into vector representations.
  • Positional Encoding: Provides sequence position information.
  • Self-Attention: Computes associations between different positions within a sequence.
  • Multi-Head Attention: Learns different relationships from multiple representation subspaces.
  • Feed Forward Network: Applies non-linear transformations to the representation at each position.
  • Residual Connection: Improves information flow and training in deep networks.
  • Layer Normalization: Stabilizes the network training process.