The field of artificial intelligence involves a large number of technical terms, covering multiple areas such as mathematical foundations, machine learning, deep learning, neural networks, large language models, generative AI, model training, inference deployment, and agents.
These terms are not independent of one another.
For example:
- Machine learning is an important branch of artificial intelligence.
- Deep learning is an important technical approach within machine learning.
- Transformer is a deep neural network architecture.
- Large language models are usually built on architectures such as Transformer.
- RAG enhances large language models with external knowledge.
- An Agent combines large models, tools, memory, and task planning to handle more complex tasks.
If you only memorize the definitions of individual terms, it is easy to end up with fragmented knowledge points, making it difficult to build a complete technical understanding.
Following the technical system, this article divides common terms in the field of artificial intelligence into modules such as basic concepts, machine learning, deep learning, generative AI, large language models, training and inference, knowledge augmentation, Agent, multimodality, safety evaluation, and engineering deployment, to help build a complete AI technical knowledge system.
I. Basic Concepts of Artificial Intelligence
This part mainly addresses three questions:
- What is artificial intelligence?
- What is the relationship between machine learning and deep learning?
- What are the differences between different AI technical approaches?
1.1 Artificial Intelligence (AI)
Artificial intelligence is an important research field of computer science, with the goal of enabling machines to perform intelligent behaviors such as perception, learning, reasoning, decision-making, understanding, and generation.
AI is a broad concept and does not specifically refer to any single model or algorithm.
Main research directions include:
- Knowledge Representation
- Automated Reasoning
- Machine Learning
- Computer Vision
- Natural Language Processing
- Speech Recognition and Synthesis
- Planning and Decision Making
- Robotics
1.2 Machine Learning (ML)
Machine learning is an important branch of artificial intelligence.
Its core idea is to let computers learn patterns from data rather than relying entirely on manually written fixed rules.
For example, traditional rule-based systems may require humans to define:
- Which features indicate spam.
- Which conditions indicate anomalous transactions.
- Which rules can determine the target in an image.
Machine learning, by contrast, uses training data to let models automatically learn the relationship between inputs and outputs.
Typical tasks:
- Classification
- Regression
- Clustering
- Dimensionality Reduction
- Anomaly Detection
1.3 Deep Learning (DL)
Deep learning is an important technical approach within machine learning, mainly using multi-layer neural networks to learn complex representations in data.
Its characteristics include:
- Uses multi-layer neural networks.
- Trains model parameters through backpropagation.
- Can learn complex nonlinear relationships.
- Usually requires more data and computing resources.
- Can automatically learn multi-level feature representations.
Common network architectures:
- CNN: Convolutional Neural Network.
- RNN: Recurrent Neural Network.
- LSTM: Long Short-Term Memory Network.
- Transformer: A network architecture based on the attention mechanism.
- GNN: Graph Neural Network.
1.4 Relationship Among the Three
flowchart TB
A["Artificial Intelligence (AI)"] --> B["Machine Learning (ML)"]
A --> C["Symbolic AI"]
A --> D["Search, Planning and Reasoning"]
B --> E["Traditional Machine Learning"]
B --> F["Deep Learning (DL)"]
F --> G["CNN, RNN, Transformer"]
F --> H["Generative Deep Learning Models"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style D fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style E fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style F fill:#EAF3EC,stroke:#83A995,color:#315D49
style G fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style H fill:#E5F0E9,stroke:#83A995,color:#315D49
Note that AI, ML, and DL are not three completely parallel technologies.
They can usually be expressed as:
However, artificial intelligence also includes technologies that do not rely on machine learning, such as symbolic reasoning, search, and rule-based systems.
1.5 Weak AI, Strong AI, and Artificial General Intelligence
| Term | English | Core Meaning |
|---|---|---|
| Narrow AI | Narrow AI | AI designed for specific tasks |
| Strong AI | Strong AI | Theoretical concept of possessing true general intelligence |
| Artificial General Intelligence | AGI | AI capable of general competence across a wide range of tasks |
| Artificial Superintelligence | ASI | Hypothetical intelligence whose capabilities significantly exceed those of humans across a broad range of domains |
The AI systems widely deployed today mainly fall within the category of AI designed for specific tasks or a specific range of capabilities.
Note that Strong AI and AGI may be defined differently in different research contexts, and the two should not simply be treated as completely identical terms.
II. Core Machine Learning Terms
Machine learning mainly studies how to learn models from data, and how to use models to make predictions and decisions.
2.1 Dataset
A dataset is the collection of data used for model training, validation, and testing.
Common dataset splits:
| Dataset | English | Main Purpose |
|---|---|---|
| Training Set | Training Set | Learning model parameters |
| Validation Set | Validation Set | Tuning hyperparameters, selecting models |
| Test Set | Test Set | Evaluating model generalization performance |
Typical data workflow:
flowchart TB
A["Raw dataset"] --> B["Data cleaning and preprocessing"]
B --> C["Dataset splitting"]
C --> D["Training set"]
C --> E["Validation set"]
C --> F["Test set"]
D --> G["Model training"]
E --> H["Model selection and hyperparameter tuning"]
G --> H
H --> I["Final model evaluation"]
F --> I
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#EAF3EC,stroke:#83A995,color:#315D49
style D fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style E fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style F fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style G fill:#EAF3EC,stroke:#83A995,color:#315D49
style H fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style I fill:#E5F0E9,stroke:#83A995,color:#315D49
Note: The validation set and the test set must not be mixed up at will.
If the test set is repeatedly used to tune the model, it will gradually become involved in model selection, and the final evaluation results may be overly optimistic.
2.2 Feature
A feature is the input information used to describe a sample.
For example, when predicting house prices, the following may be used:
- House area.
- Number of rooms.
- Year built.
- Location.
- Surrounding transportation conditions.
Features can be numeric, categorical, textual, image-based, or represented in other structured forms.
2.3 Label
A label is the information used in supervised learning to represent the target result.
For example:
- Spam detection: spam or legitimate email.
- House price prediction: actual transaction price.
- Image classification: the category corresponding to the image.
Features usually represent the input, while labels usually represent the target that the model needs to learn to predict.
2.4 Model
A model is a computational structure that models the relationship between input and output through algorithms and parameters.
It can be abstractly expressed as:
Where:
- x: input.
- \hat{y}: model prediction result.
- f: model computation function.
- \theta: model parameters.
A model is not the same as a training algorithm.
The model is responsible for computing the output from the input, while the training algorithm is responsible for finding suitable parameters.
2.5 Parameter
A parameter is an internal numerical value that the model learns during training.
For example, in a neural network:
- Weight.
- Bias.
Parameters determine how the model maps input to output.
2.6 Hyperparameter
A hyperparameter is a configuration set by the developer or the training strategy during the training process; it is not an ordinary model parameter learned directly through the current training process.
Common hyperparameters:
- Learning Rate: learning rate.
- Batch Size: batch size.
- Epoch: number of training epochs.
- Hidden Size: hidden layer dimension.
- Number of Layers: number of network layers.
- Dropout: dropout ratio.
- Weight Decay: weight decay coefficient.
Difference between parameters and hyperparameters:
| Item | Parameter | Hyperparameter |
|---|---|---|
| English | Parameter | Hyperparameter |
| Source | Learned during model training | Manually set or automatically searched |
| Role | Determines the specific computational behavior of the model | Controls the model structure or the training process |
| Example | Weights, biases | Learning rate, batch size |
2.7 Loss Function
A loss function is used to measure the difference between the model's predictions and the target results.
For example, mean squared error in regression tasks:
Cross-entropy loss is common in classification tasks:
Where:
- N: number of samples.
- C: number of classes.
- y_i: true label.
- \hat{y}_i: model-predicted probability.
The loss function is an important basis for optimizing model parameters during training.
2.8 Optimizer
An optimizer updates model parameters based on the gradients of the loss function.
Common optimizers:
- SGD: stochastic gradient descent.
- Momentum: momentum optimization.
- AdaGrad: adaptive gradient algorithm.
- RMSprop: root mean square propagation.
- Adam: adaptive moment estimation.
- AdamW: an Adam variant with decoupled weight decay.
2.9 Overfitting and Underfitting
| Term | English | Meaning |
|---|---|---|
| Overfitting | Overfitting | The model fits the training data too closely, and its generalization ability declines |
| Underfitting | Underfitting | The model fails to adequately learn the patterns in the data |
| Generalization ability | Generalization | The model's ability to handle unseen data |
Overfitting and underfitting are very important model performance issues in machine learning.
Common mitigation methods:
- Regularization.
- Dropout.
- Data augmentation.
- Early Stopping.
- Adjusting model complexity.
- Adding high-quality training data.
III. Main Learning Paradigms of Machine Learning
3.1 Supervised Learning
Train a model using labeled data.
The goal is to learn the mapping relationship between inputs and target outputs.
Main tasks:
- Classification.
- Regression.
Typical algorithms:
- Linear Regression.
- Logistic Regression.
- Decision Tree.
- Random Forest.
- SVM.
- XGBoost.
- Neural networks.
3.2 Unsupervised Learning
Use data without explicit target labels to learn the internal structure, distribution, or underlying patterns of the data.
Typical tasks:
- Clustering.
- Dimensionality reduction.
- Density estimation.
- Anomaly detection.
Typical algorithms:
- K-Means.
- DBSCAN.
- PCA.
- Gaussian Mixture Model.
3.3 Semi-supervised Learning
Train a model using a small amount of labeled data together with a large amount of unlabeled data.
It is suitable for scenarios where annotation is costly and unlabeled data is easy to obtain.
3.4 Self-supervised Learning
Training targets are constructed from the data itself, without the need to manually provide traditional labels for each sample.
For example, large language models learn language patterns by predicting the next Token.
Self-supervised learning is an important technical foundation for the pretraining of current large models.
3.5 Reinforcement Learning (RL)
Learn a policy through the interaction between an agent and its environment.
Core concepts:
| Term | English | Meaning |
|---|---|---|
| Agent | Agent | The learning entity that performs actions |
| Environment | Environment | The external system with which the agent interacts |
| State | State | The current state of the environment |
| Action | Action | The behavior performed by the agent |
| Reward | Reward | Feedback from the environment on the result of an action |
| Policy | Policy | The method for selecting actions based on the state |
| Value function | Value Function | Evaluates the long-term value of a state or action |
| Return | Return | Cumulative reward |
Basic interaction in reinforcement learning:
flowchart TB
A["Agent"] -->|"Select Action"| B["Environment"]
B -->|"Return State"| A
B -->|"Return Reward"| A
A --> C["Update Policy"]
C --> A
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
3.6 Comparison of Different Learning Paradigms
| Learning paradigm | Depends on manual labels | Core objective | Typical applications |
|---|---|---|---|
| Supervised learning | Yes | Learn the mapping from inputs to targets | Classification, prediction |
| Unsupervised learning | No | Discover data structure | Clustering, dimensionality reduction |
| Semi-supervised learning | Partially | Learn using a small amount of labels | Image classification |
| Self-supervised learning | Does not require traditional manual labels | Construct training objectives from data | Large model pretraining |
| Reinforcement learning | Uses reward signals | Learn long-term decision-making policies | Games, robotics, policy optimization |
Note that self-supervised learning is related to unsupervised learning, but the two are not exactly the same. Self-supervised learning usually has explicitly constructed surrogate supervision objectives.
IV. Deep Learning and Neural Network Terminology
Deep learning learns complex representations of data through neural networks and is an important foundation for current computer vision, speech processing, and large language models.
4.1 Neural Network
A neural network is a model structure composed of multiple computational units.
Typical components:
- Input layer.
- Hidden layer.
- Output layer.
- Weight.
- Bias.
- Activation function.
4.2 Neuron
A neuron is the basic computational unit in a neural network.
Its computation process can be expressed as:
Where:
- x_i: input.
- w_i: weight.
- b: bias.
- f: activation function.
- y: output.
4.3 Activation Function
Activation functions introduce nonlinear expressive capability into neural networks.
Common activation functions:
| Activation function | Formula or characteristic | Common uses |
|---|---|---|
| Sigmoid | Output range is (0,1) | Binary classification output |
| Tanh | Output range is (-1,1) | Some recurrent networks |
| ReLU | \max(0,x) | Traditional deep networks |
| Leaky ReLU | Retains a small slope in the negative region | Mitigates neuron deactivation |
| GELU | Smooth nonlinear activation | Transformer |
| SiLU | x\cdot sigmoid(x) | Various modern networks |
4.4 Forward Propagation
The process in which input data passes through the layers of a network and produces a prediction.
4.5 Backpropagation
Using the loss function and the chain rule to compute the gradients of the model parameters.
4.6 Gradient Descent
Adjusting the model parameters along the gradient direction so that the loss function gradually decreases.
Basic update formula:
Where:
- \theta_t: the current parameters.
- \eta: the learning rate.
- \nabla_{\theta}L: the gradient of the loss function with respect to the parameters.
4.7 Epoch, Batch, Iteration
These three terms are very common in model training.
| Term | Meaning |
|---|---|
| Epoch | One complete pass through the training data |
| Batch | A batch of data fed into the model at one time |
| Iteration / Step | One training update step, usually corresponding to processing one Batch |
For example:
The training set has 10,000 samples, and the Batch Size is 100.
Then one Epoch theoretically contains:
If training for 20 Epochs, then theoretically 2,000 parameter updates are needed (assuming an update is performed for every Batch and no data is dropped).
4.8 Vanishing Gradients and Exploding Gradients
| Term | English | Symptom |
|---|---|---|
| Vanishing gradient | Vanishing Gradient | Gradients gradually become smaller during backpropagation |
| Exploding gradient | Exploding Gradient | Gradients become too large during backpropagation |
Common solutions:
- Initialize weights properly.
- Use appropriate activation functions.
- Use residual connections.
- Use normalization techniques.
- Use gradient clipping.
- Adjust the network architecture and learning rate.
4.9 Common Neural Network Architectures
| Architecture | English | Core Feature | Typical Application |
|---|---|---|---|
| CNN | Convolutional Neural Network | Convolution operations extract local features | Image recognition |
| RNN | Recurrent Neural Network | Processes sequence information recurrently | Sequence modeling |
| LSTM | Long Short-Term Memory | Uses gating structures to handle long-term dependencies | Time series |
| GRU | Gated Recurrent Unit | A simplified gated recurrent network | Sequence modeling |
| Transformer | Transformer | Uses attention mechanisms to model relationships | Large language models |
| GNN | Graph Neural Network | Designed for graph-structured data | Social networks, knowledge graphs |
| Autoencoder | AE | Learns a compressed representation of the input | Dimensionality reduction, reconstruction |
| U-Net | U-Net | Encoder and decoder with skip connections | Image segmentation, diffusion models |
V. Generative Artificial Intelligence Terminology
Generative AI is an important direction in the current field of artificial intelligence.
Unlike traditional models that mainly perform classification and prediction, generative models can learn data distributions and generate new content.
5.1 Generative AI
Generative AI is an artificial intelligence technology capable of generating new content based on learned data patterns.
The generated content includes:
- Text.
- Images.
- Audio.
- Video.
- Code.
- 3D content.
5.2 Discriminative Model
Mainly learns the relationship between inputs and labels, or directly learns the conditional probability:
Typical tasks:
- Classification.
- Regression.
- Recognition.
- Prediction.
5.3 Generative Model
Learns the distribution of the data itself or a conditional generation distribution, for example:
Or:
Where c represents the conditioning information.
5.4 Main Generative Model Architectures
| Technology | English | Core Principle | Typical Applications |
|---|---|---|---|
| GAN | Generative Adversarial Network | Adversarial training between generator and discriminator | Image generation |
| VAE | Variational Autoencoder | Learn latent variable distributions | Data generation, representation learning |
| Diffusion Model | Diffusion Model | Learn to generate data by stepwise denoising | Image, audio, and video generation |
| Autoregressive Model | Autoregressive Model | Predict the next content unit step by step based on existing content | Text generation, code generation |
| Flow-based Model | Flow-based Model | Model probability distributions using invertible transformations | Density estimation, generation |
5.5 Differences Between Discriminative Models and Generative Models
| Comparison Dimension | Discriminative Model | Generative Model |
|---|---|---|
| Main Objective | Distinguish or predict | Learn data distribution and generate |
| Typical Modeling Target | P(y\mid x) | P(x), P(x,y), P(x\mid c) |
| Common Tasks | Classification, regression | Text, image, and audio generation |
| Typical Models | Logistic Regression, SVM | GAN, VAE, Diffusion, LLM |
Note that generative and discriminative describe modeling objectives and methods; they do not mean that every model can only be used for one type of task.
5.6 Multimodal Generation
Multimodal generation refers to a model's ability to process or generate multiple different types of data.
For example:
- Text-to-image generation.
- Text-to-video generation.
- Text-to-speech generation.
- Image-to-description generation.
- Video summarization.
- Speech-driven text conversation.
6. Core Terms of Large Language Models (LLM)
Large language models are an important technical foundation of current generative AI.
They learn language structure, knowledge associations, and contextual representations from large-scale text and other training data, and can perform text understanding, generation, and various complex tasks.
6.1 Large Language Model (LLM)
A large language model generally refers to a language model with a relatively large number of parameters that is trained on large-scale data.
Typical characteristics:
- Based on deep neural networks.
- Usually uses Transformer or its variants.
- Pretrained on large-scale data.
- Models text through Tokens.
- Has contextual modeling and text generation capabilities.
6.2 Token
A Token is the basic discrete unit used by a model when processing text.
A Token may correspond to:
- A complete word.
- Part of a word.
- A single Chinese character.
- A fragment consisting of multiple characters.
- Punctuation.
- Special control markers.
A Token is not equivalent to a word in natural language, nor does it necessarily correspond one-to-one with characters.
6.3 Tokenization
Tokenization is the process of converting raw text into a Token sequence.
For example:
Raw text:
Artificial Intelligence
After passing through a Tokenizer, it may be split into multiple Tokens.
Different models use different Tokenizers, so the same piece of text may correspond to different numbers of Tokens in different models.
6.4 Vocabulary
Vocabulary is the token vocabulary used by a model's Tokenizer.
It defines the mapping between Tokens and Token IDs.
6.5 Token ID
A Token ID is the numerical index of a Token in the vocabulary.
Models typically perform computations using Token IDs rather than raw text directly.
6.6 Embedding
Embedding is the process of mapping discrete Token IDs to continuous vectors.
For example:
The resulting vectors can serve as input representations for a neural network.
Embedding is not merely simple numerical encoding; it is an important foundation for a model to learn semantics and contextual relationships.
6.7 Positional Encoding
Positional encoding is used to provide the model with positional information in a sequence.
Because Attention itself does not inherently contain complete sequence order information, Transformer needs to express the positional relationships of Tokens through positional encoding or position-related mechanisms.
Common methods:
- Sinusoidal Positional Encoding.
- Learned Positional Embedding.
- RoPE (Rotary Positional Embedding).
- ALiBi (Attention with Linear Biases).
6.8 Attention
Attention is a mechanism that dynamically aggregates information based on the correlations between inputs.
The formula for Scaled Dot-Product Attention:
Where:
- Query: query vector.
- Key: key vector.
- Value: value vector.
- d_k: key dimension.
6.9 Self-Attention
Self-Attention refers to an attention mechanism in which Query, Key, and Value come from the same sequence representation.
It enables different Tokens in a sequence to establish information associations.
6.10 Multi-Head Attention
Multi-head attention computes in parallel through multiple different attention heads, learning different information relationships.
It can model interactions between Tokens in different representation subspaces.
6.11 Transformer
Transformer is a neural network architecture centered on Attention.
Main components include:
- Embedding.
- Positional Encoding or positional mechanism.
- Multi-Head Attention.
- Feed Forward Network.
- Residual Connection.
- Layer Normalization.
Common structural types:
| Structure | Main characteristics | Representative use |
|---|---|---|
| Encoder-only | Bidirectional context representation | Text understanding, classification |
| Decoder-only | Causal autoregressive generation | Large language models |
| Encoder-Decoder | Encode input, then generate output | Translation, sequence transformation |
6.12 Context Window
The context window is the range of Tokens that a model processes or references at one time.
It usually includes:
- System instructions.
- User input.
- Conversation history.
- Content returned by tools.
- Currently generated content.
A larger context window does not mean the model can necessarily use all the information in it equally effectively.
Long contexts also involve issues such as attention computation cost, information interference, and long-distance information retrieval.
6.13 Inference
Inference is the process of using a model to make predictions or generate after training is complete.
Large language model inference usually includes:
- Input Tokenization.
- Embedding.
- Transformer forward computation.
- Output Logits.
- Probability distribution computation.
- Token sampling or selection.
- Autoregressive generation of subsequent Tokens.
6.14 Logits
Logits are unnormalized scores produced by the model's output layer.
After Logits pass through Softmax, a probability distribution can be obtained.
6.15 Softmax
Softmax maps a set of real numbers to a probability distribution whose values sum to 1.
6.16 Decoding
Decoding is the process of selecting subsequent Tokens based on the probability distribution output by the model.
Common strategies:
- Greedy Decoding: greedy decoding.
- Beam Search: beam search.
- Top-K Sampling: limits the number of candidate Tokens.
- Top-P Sampling: selects the candidate range according to cumulative probability.
- Temperature: adjusts the sharpness of the probability distribution.
6.17 KV Cache
KV Cache is the caching of the Key and Value representations of historical Tokens during autoregressive generation.
Its purpose is to avoid recomputing the Key and Value of all historical Tokens each time a new Token is generated.
It can reduce redundant computation and improve inference efficiency.
6.18 Prompt
A Prompt is the input prompt given to the model, used to provide task requirements, context, and constraints.
6.19 Prompt Engineering
Prompt Engineering is a method of improving model task performance by designing and optimizing prompt content.
Common techniques:
- Clarify the role and task.
- Provide context.
- Specify the output format.
- Provide examples.
- Clarify constraints.
- Break down complex tasks.
6.20 Large Language Model Generation Process
flowchart TB
A["User input text"] --> B["Tokenizer"]
B --> C["Token ID"]
C --> D["Embedding"]
D --> E["Transformer"]
E --> F["Output Logits"]
F --> G["Softmax probability distribution"]
G --> H["Decoding"]
H --> I["Select the next Token"]
I --> J{"Is generation finished?"}
J -->|"No"| E
J -->|"Yes"| K["Output final text"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style D fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style E fill:#EAF3EC,stroke:#83A995,color:#315D49
style F fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style G fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style H fill:#EAF3EC,stroke:#83A995,color:#315D49
style I fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style J fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style K fill:#E5F0E9,stroke:#83A995,color:#315D49
7. Large Language Model Training and Alignment Terminology
Large language models do not directly acquire mature conversational abilities through a single training run.
They usually need to go through multiple training stages so that the model gradually acquires language modeling, instruction following, and preference alignment capabilities.
7.1 Pre-training
Pre-training is the process of learning general language and knowledge representations from large-scale data.
For a typical autoregressive language model, the common training objective is to predict the next Token:
Pre-training mainly enables the model to learn:
- Language structure.
- Lexical and semantic relationships.
- Knowledge associations.
- Contextual patterns.
- Some reasoning and task patterns.
7.2 Fine-tuning
Fine-tuning is the process of continuing to train an existing pre-trained model on specific data so that the model adapts to a particular task or application scenario.
Common types:
- Full fine-tuning.
- Supervised fine-tuning (SFT).
- Parameter-efficient fine-tuning (PEFT).
7.3 SFT (Supervised Fine-Tuning)
Supervised fine-tuning trains the model on data consisting of instructions and expected responses.
Its main goal is to teach the model how to complete tasks according to instructions.
7.4 RLHF (Reinforcement Learning from Human Feedback)
RLHF is a method for reinforcement learning optimization based on human feedback.
Typical process:
- Collect human preference data.
- Train a reward model.
- Use reinforcement learning to optimize the language model.
- Make the model's outputs better align with human preferences.
7.5 DPO (Direct Preference Optimization)
DPO is a method that directly uses preference data to optimize a model.
It does not require separately training a reward model and carrying out the full reinforcement learning optimization process as in the traditional RLHF pipeline.
DPO directly uses preference samples to optimize the policy model, making it more inclined to generate preferred responses.
7.6 Reward Model
A reward model is used to evaluate the quality or preference level of model outputs.
It is usually trained on preference data and outputs a score or reward signal.
7.7 Alignment
Alignment refers to making the behavior of AI models consistent with human intentions, values, norms, and safety requirements.
Common research directions:
- Instruction following.
- Human preference alignment.
- Safety alignment.
- Value alignment.
- Refusing unsafe requests.
7.8 PEFT
PEFT (Parameter-Efficient Fine-Tuning) is a parameter-efficient fine-tuning technique.
Its goal is to adapt a model to a specific task while minimizing trainable parameters and computational resources as much as possible.
Common methods:
- LoRA.
- QLoRA.
- Adapter.
- Prefix Tuning.
- Prompt Tuning.
7.9 LoRA
LoRA (Low-Rank Adaptation) uses low-rank matrix decomposition to make a parameter-efficient approximation of updates to some weights in a pre-trained model.
Its core form can be expressed as:
Where:
Here A and B are low-rank matrices.
By limiting the rank of the update matrices, the number of parameters that need to be trained can be significantly reduced.
7.10 Training and Alignment Process
flowchart TB
A["Large-scale training data"] --> B["Pre-training"]
B --> C["Base model"]
C --> D["Supervised fine-tuning SFT"]
D --> E["Instruction model"]
E --> F["Preference data and alignment optimization"]
F --> G["RLHF or DPO"]
G --> H["Aligned model"]
H --> I["Evaluation and safety testing"]
I --> J["Deployment and application"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style D fill:#EAF3EC,stroke:#83A995,color:#315D49
style E fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style F fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style G fill:#EAF3EC,stroke:#83A995,color:#315D49
style H fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style I fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style J fill:#E5F0E9,stroke:#83A995,color:#315D49
Note: This is a typical training and alignment path; not all models strictly follow the same process. Some models use other data recipes, training objectives, and post-training methods.
VIII. Terminology for Knowledge Enhancement and External Capabilities
The knowledge of large language models is mainly learned from training data and encoded in model parameters, but parametric knowledge has limitations in updating, retrieval, and factual accuracy.
Therefore, practical applications usually introduce external knowledge and tools.
8.1 RAG (Retrieval-Augmented Generation)
Retrieval-augmented generation.
Before generating an answer, relevant content is first retrieved from an external knowledge base, and the retrieval results are then provided to the large language model as context.
Main components:
- Document parsing.
- Text chunking (Chunking).
- Embedding.
- Vector database.
- Retrieval.
- Reranking.
- Context construction.
- Answer generation.
8.2 Embedding Model
An Embedding Model is a model that maps text, images, or other data into vector representations.
In RAG, it is often used to convert documents and user questions into vectors for semantic retrieval.
8.3 Vector Database
A vector database is used to store and retrieve vector data.
Common capabilities:
- Vector storage.
- Similarity search.
- Metadata filtering.
- Index management.
- Hybrid search.
8.4 Similarity Search
Similarity search is a method for finding related content based on the degree of similarity between vectors.
Common metrics:
- Cosine Similarity: cosine similarity.
- Dot Product: dot product.
- Euclidean Distance: Euclidean distance.
8.5 Chunking
Chunking is the process of splitting long documents into smaller text fragments.
Reasonable text segmentation can help retrieval systems locate relevant content.
Segmentation methods include:
- Fixed-length chunking.
- Paragraph-based chunking.
- Heading-structure-based chunking.
- Recursive text chunking.
- Semantic chunking.
8.6 Reranker
A Reranker is a reranking model.
It recalculates relevance scores for candidate documents obtained from initial retrieval, thereby optimizing the context ultimately provided to the large model.
8.7 Hybrid Search
Hybrid search usually combines keyword retrieval with vector semantic retrieval.
For example:
- BM25: a keyword retrieval method based on term statistics.
- Dense Retrieval: semantic retrieval based on dense vectors.
The two approaches can complement each other.
Keyword retrieval is good at matching explicit terms, numbers, and names, while vector retrieval is better at handling content with similar semantics but different wording.
8.8 Knowledge Graph
A knowledge graph organizes knowledge through entities, relations, and attributes.
Typical structure:
- Entity: entity.
- Relation: relation.
- Attribute: attribute.
- Triple: triple.
For example:
Python —— belongs to —— programming language
Python —— has —— dynamic typing features
Transformer —— uses —— Attention mechanism
8.9 Difference Between RAG and Fine-tuning
| Aspect | RAG | Fine-tuning |
|---|---|---|
| Core approach | Retrieve external knowledge | Update model parameters |
| Knowledge updates | Update the knowledge base only | Usually requires retraining |
| Primary role | Supplement facts and context | Adjust model behavior and task capabilities |
| Traceability | Can link to retrieval sources | Knowledge in parameters is usually difficult to trace directly |
| Suitable scenarios | Enterprise knowledge Q&A, real-time information | Task-specific adaptation, output style and behavior optimization |
The two do not conflict.
In real systems, RAG and fine-tuning can be used together.
IX. AI Agent and Agent Technology Terminology
AI Agents have been an important direction for AI engineering applications in recent years.
Unlike traditional one-question-one-answer model calls, an Agent can perform task planning, tool calling, state management, and multi-step execution around a goal.
9.1 Agent
An Agent generally refers to an intelligent entity that can perceive its environment, make decisions based on goals, and execute actions.
In LLM applications, an Agent usually uses a large language model as the core decision-making component and completes tasks by combining tools, context, memory, and an execution loop.
9.2 Tool Calling
Tool calling is a mechanism in which the model generates structured tool requests according to task needs, an external program executes them, and the execution results are returned to the model.
Common tools:
- Search engines.
- Databases.
- File systems.
- Code execution environments.
- Enterprise APIs.
- Browser automation tools.
Note that the model is usually responsible for proposing tool call requests, while the external system actually executes the tools.
9.3 Function Calling
Function Calling usually refers to the model's ability to generate structured call parameters according to a predefined function interface.
For example:
{
"name": "get_weather",
"arguments": {
"city": "Singapore"
}
}
This is only an example of a structured call request; it does not mean that the model directly executed the function.
9.4 Planning
Planning is task planning.
An Agent breaks down a complex goal into multiple subtasks and determines their execution order and dependencies.
9.5 Reasoning
Reasoning is the process of drawing inferences.
The model analyzes input conditions, existing knowledge, and intermediate results to draw conclusions or choose the next action.
9.6 Memory
Memory refers to the memory mechanism.
In Agent systems, it is usually used to store information generated during task execution.
Common categories:
| Type | Function |
|---|---|
| Short-term Memory | Stores the current task or conversation context |
| Long-term Memory | Stores reusable information across tasks |
| Episodic Memory | Stores past events and execution experiences |
| Semantic Memory | Stores structured facts and knowledge |
| Procedural Memory | Stores task execution methods or processes |
Different systems may classify Memory differently.
9.7 Reflection
Reflection refers to the reflection mechanism.
An Agent can inspect existing execution results, analyze errors, and adjust subsequent strategies.
However, model self-reflection is not the same as objective verification. Reflection results still need to be combined with external evidence and execution feedback.