Back to series
· 13 min read

What Are Large Language Models? From Language Modeling Principles to Core Architectures and Application Mechanisms


1. What Is a Large Language Model?

A large language model (LLM) is a type of language model built on deep learning. Through training on large-scale text and other types of data, it learns language structure, semantic relationships, knowledge associations, and contextual patterns, thereby performing text understanding and generation tasks.

Compared with traditional language models, large language models typically have the following characteristics:

  1. Large parameter scale: The model contains billions, tens of billions, or even hundreds of billions of trainable parameters.
  2. Large training data scale: It is trained on large-scale text, code, and multimodal data.
  3. Strong contextual modeling capability: It can understand the current input in light of a relatively long context.
  4. Strong task adaptability: It can adapt to different tasks through prompting, fine-tuning, and other methods.
  5. Strong generation capability: It can generate text, code, summaries, explanations, and other structured content.
  6. Some generalization capability: It can transfer patterns learned during training to tasks or scenarios it has not directly seen.

It should be noted that a large language model is not a simple knowledge database, nor is it a question-answering system that runs according to fixed rules.

In essence, it is a neural network that learns a parameterized function through training. The model converts the input into an internal representation, then computes the probability distribution of the next Token based on the context, and generates the output step by step.

2. Basic Principles of Language Models

To understand large language models, one must first understand what a language model is.

The core task of a language model (LM) is:

To predict the probability of the next word or Token based on the existing context.

For example, given the input:

The weather today is very

the model needs to determine what the next Token might be:

Candidate Token Example probability
Good 0.55
Not bad 0.20
Terrible 0.10
Cold 0.08
Other 0.07

The numbers above are only for illustrating the probability distribution and are not actual model outputs.

The model does not look up answers directly from a database; rather, it computes the probabilities of different candidate Tokens based on the parameters it has learned and the current context.

2.1 Autoregressive Language Modeling

Many mainstream generative large language models currently use autoregressive language modeling.

Autoregressive models predict subsequent Tokens step by step in sequence order.

Assume the input sequence is:

X=(x_1,x_2,\ldots,x_T)

According to the chain rule of probability, the joint probability of the entire sequence can be decomposed as:

P(X)=\prod_{t=1}^{T}P(x_t|x_1,\ldots,x_{t-1})

It can also be written in shorthand as:

P(X)=\prod_{t=1}^{T}P(x_t|x_{<t})

where:

  • x_t: The Token at the current position.
  • x_{<t}: All Tokens before the current position.
  • P(x_t|x_{<t}): The conditional probability of the Token at the current position given the historical context.

This approach enables the model to generate a complete sequence step by step based on the existing content.

2.2 From Predicting the Next Token to Generating Complete Text

For example, the model needs to generate:

Artificial intelligence is changing the way software is developed.

The generation process can be abstracted as:

flowchart TB
    A["Input context: Artificial intelligence is"] --> B["Predict the next Token"]
    B --> C["Select Token: changing"]
    C --> D["Add the new Token to the context"]
    D --> E["Predict the next Token again"]
    E --> F["Select Token: software"]
    F --> G["Continue generating subsequent Tokens"]
    G --> H["Obtain the complete text"]

    style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style B fill:#EAF3EC,stroke:#83A995,color:#315D49
    style C fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style D fill:#EAF3EC,stroke:#83A995,color:#315D49
    style E fill:#EAF3EC,stroke:#83A995,color:#315D49
    style F fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style G fill:#EAF3EC,stroke:#83A995,color:#315D49
    style H fill:#DCEFE5,stroke:#4F8B70,color:#24563F

Note the following:

  • Autoregressive generation usually proceeds token by token.
  • Each generation incorporates the new token into the subsequent context.
  • The model generates a probability distribution rather than directly determining a single unique answer.
  • Which token is actually selected depends on the decoding strategy and sampling parameters.

Predicting the next token is the training objective, but to accomplish this objective, the model needs to learn language structure, semantic associations, contextual dependencies, and a large number of knowledge patterns.

3. Token: The Basic Unit of Text Processing in Large Language Models

Large language models do not directly process the text that humans see; they first convert the text into tokens.

A token is the basic unit that the model uses to process text sequences.

A token may correspond to:

  • A complete Chinese character.
  • An English word.
  • Part of an English word.
  • A punctuation mark.
  • A number or special symbol.
  • Certain text fragments composed of multiple characters.

For example:

Hello, world!

After tokenization, it may be split into:

["Hello", ",", " world", "!"]

The exact split result depends on the tokenizer used by the model.

3.1 The Role of the Tokenizer

The Tokenizer is responsible for converting between text and token sequences.

It mainly involves two processes:

  1. Encoding: converting text into token IDs.
  2. Decoding: converting token IDs back into text.

Processing flow:

flowchart TB
    A["Raw text"] --> B["Tokenizer encoding"]
    B --> C["Token sequence"]
    C --> D["Token ID sequence"]
    D --> E["Embedding vector mapping"]
    E --> F["Input Transformer"]

    style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style B fill:#EAF3EC,stroke:#83A995,color:#315D49
    style C fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style D fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style E fill:#EAF3EC,stroke:#83A995,color:#315D49
    style F fill:#DCEFE5,stroke:#4F8B70,color:#24563F

3.2 Common Tokenization Algorithms

Algorithm Core idea Characteristics
BPE Repeatedly merges frequent adjacent symbols Common in GPT-series tokenization schemes
WordPiece Builds the vocabulary based on word-piece probabilities or scores Common in the BERT series
Unigram Selects subwords from a larger candidate vocabulary Can be used in SentencePiece
SentencePiece Directly performs subword modeling on raw text Supports multiple languages and different text forms

The design of the Tokenizer affects:

  • The model's vocabulary size.
  • Input sequence length.
  • Training and inference cost.
  • Encoding efficiency for different languages.
  • The model's ability to handle special symbols and code.

3.3 Token ID and Embedding

A Token ID is the integer index of a token in the vocabulary.

For example:

Token Token ID
Artificial intelligence 1024
Software 2056
Development 3108

The numbers above are only examples.

The model cannot directly understand the semantics of a token from an integer index, so it needs to map the Token ID to a continuous vector.

This process is called Embedding (word embedding).

E=Embedding(TokenID)

Assuming the model's hidden dimension is d, each token is mapped to a d-dimensional vector.

For example:

TokenID=1024

After Embedding:

e=[0.12,-0.31,0.45,\ldots]

Embedding converts discrete symbols into continuous numerical representations that neural networks can process.

4. The Core Architecture of Large Language Models: Transformer

Current mainstream large language models are mostly built on the Transformer architecture.

The Transformer was first proposed by a Google research team in the 2017 paper "Attention Is All You Need".

Its key innovation is the use of the Attention mechanism to establish relationships between different positions within a sequence, instead of relying on traditional recurrent neural networks to pass hidden states along step by step.

4.1 Overall Structure of the Transformer

A typical Transformer mainly consists of the following modules:

  1. Embedding: maps Token IDs to vectors.
  2. Positional Encoding: provides positional information.
  3. Multi-Head Attention: establishes relationships between different Tokens.
  4. Feed Forward Network: performs nonlinear feature transformation.
  5. Residual Connection: residual connections that improve training of deep networks.
  6. Layer Normalization: normalizes the internal representations of the network.

In autoregressive generative large language models, a Decoder-only Transformer is usually used.

Its overall structure can be abstracted as:

flowchart TB
    A["Input Token ID"] --> B["Token Embedding"]
    B --> C["Positional Encoding or Position Representation"]
    C --> D["Transformer Block"]
    D --> E["Masked Self-Attention"]
    E --> F["Feed Forward Network"]
    F --> G["Residual Connection and Normalization"]
    G --> H{"Any Transformer Layers Left?"}
    H -->|"Yes"| D
    H -->|"No"| I["Output Hidden States"]
    I --> J["LM Head Outputs Logits"]
    J --> K["Softmax Probability Distribution"]
    K --> L["Generate the Next Token"]

    style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style B fill:#EAF3EC,stroke:#83A995,color:#315D49
    style C fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style D fill:#EAF3EC,stroke:#83A995,color:#315D49
    style E fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style F fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style G fill:#EAF3EC,stroke:#83A995,color:#315D49
    style H fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
    style I fill:#EAF3EC,stroke:#83A995,color:#315D49
    style J fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style K fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style L fill:#DCEFE5,stroke:#4F8B70,color:#24563F

Particular attention should be paid here: different Transformer implementations may differ in the placement of normalization, the style of positional encoding, the attention structure, and the design of the feed-forward network. The figure above expresses the conceptual structure of a typical Decoder-only model, and is not the strict implementation detail of every model.

4.2 Attention: The Attention Mechanism

Attention is one of the most important core mechanisms of the Transformer.

Its role is:

To weight and integrate contextual information according to the degree of association between the current Token and other Tokens.

For example:

Xiaoming put the book on the table because it is heavy.

The model needs to use the context to understand what "it" may refer to.

Attention can help the model establish associations between different Tokens.

The standard Scaled Dot-Product Attention formula is:

Attention(Q,K,V)= softmax\left(\frac{QK^T}{\sqrt{d_k}}\right)V

Where:

  • Q: Query, the query matrix.
  • K: Key, the key matrix.
  • V: Value, the value matrix.
  • d_k: the dimension of the Key vectors.
  • QK^T: computes the similarity between different positions.
  • \sqrt{d_k}: scaling factor, used to control the scale of the dot-product values.
  • Softmax: converts similarities into normalized weights.

The computation process of Attention:

flowchart TB
    A["Input vector"] --> B["Linear projection"]
    B --> C["Generate Q, K, V"]
    C --> D["Compute similarity between Q and K"]
    D --> E["Scale by dividing by √dk"]
    E --> F["Compute attention weights with Softmax"]
    F --> G["Weighted sum with V"]
    G --> H["Obtain Attention output"]

    style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style B fill:#EAF3EC,stroke:#83A995,color:#315D49
    style C fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style D fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style E fill:#EAF3EC,stroke:#83A995,color:#315D49
    style F fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
    style G fill:#EAF3EC,stroke:#83A995,color:#315D49
    style H fill:#DCEFE5,stroke:#4F8B70,color:#24563F

4.3 Self-Attention: Self-Attention

Self-Attention means that Query, Key, and Value are all derived from representations of the same input sequence.

It allows each position in the sequence to aggregate information from other positions according to attention weights.

For example:

I enjoy learning about artificial intelligence because it is very interesting.

When processing “it,” the model can use the representations of other Tokens in the context that are relevant to it.

For autoregressive language models, a causal mask is also required, restricting the current position to attending only to itself and previous positions, without access to future Tokens.

For example:

Current position Allowed positions to attend to
Token 1 Token 1
Token 2 Token 1, Token 2
Token 3 Token 1, Token 2, Token 3
Token 4 Token 1, Token 2, Token 3, Token 4

This mechanism ensures that prediction tasks during training do not see the answer in advance.

4.4 Multi-Head Attention: Multi-Head Attention

A single Attention can establish one type of weighted association, while multi-head attention computes different representation subspaces in parallel through multiple attention heads.

MultiHead(Q,K,V)= Concat(head_1,\ldots,head_h)W^O

Where:

head_i=Attention(QW_i^Q,KW_i^K,VW_i^V)

Different attention heads can learn different types of association patterns, for example:

  • Syntactic relationships.
  • Semantic associations.
  • Long-distance dependencies.
  • Positional relationships.
  • Integration of contextual information.

However, one cannot simply interpret each attention head as one specific, clearly defined human-language function. The patterns actually learned may be complex and overlapping.

4.5 Feed Forward Network: Feedforward Network

Attention is responsible for contextual information interaction, while the Feed Forward Network (FFN) is responsible for further nonlinear transformation of the representation at each position.

A typical FFN can be expressed as:

FFN(x)=\sigma(xW_1+b_1)W_2+b_2

Where:

  • W_1, W_2: learnable weight matrices.
  • b_1, b_2: biases.
  • \sigma: nonlinear activation function.

An FFN usually first expands the hidden dimension and then maps it back to the original dimension.

In modern large language models, gated feedforward structures such as SwiGLU are also frequently used.

4.6 Residual Connections and Normalization

As the number of Transformer layers increases, network training faces issues with gradient propagation and optimization stability.

Residual connections allow input information to bypass some computational modules:

y=x+F(x)

Where:

  • x: module input.
  • F(x): module computation result.
  • y: output after the residual connection.

Layer Normalization is used to normalize hidden representations and improve training stability.

Common structures include:

  • Post-LN: perform sublayer computation first, then normalization.
  • Pre-LN: perform normalization first, then sublayer computation.

Modern large language models widely adopt Pre-LN or its variants.

5. Different Architecture Types of Transformer

There is more than one Transformer architecture.

Based on task objectives and information processing methods, they can be divided into three categories.

Architecture Core structure Main characteristics Typical uses
Encoder-only Encoder only Bidirectional context modeling Text classification, semantic representation
Decoder-only Decoder only Autoregressive generation Dialogue, text generation, code generation
Encoder-Decoder Encoder + Decoder Encode the input, then generate the output Translation, summarization, sequence transformation

5.1 Encoder-only

Encoder-only models can use both the preceding and following context of the input sequence.

The typical representative is BERT.

Common training objectives include Masked Language Modeling (MLM), which randomly masks some Tokens and asks the model to recover the masked content based on context.

For example:

Original text: Artificial intelligence is changing software development.
Training input: Artificial intelligence is [MASK] software development.
Prediction target: changing

It is suitable for tasks such as text understanding, classification, and semantic representation, but it is not a typical autoregressive text generation architecture.

5.2 Decoder-only

Decoder-only models use a causal mask, allowing the current position to attend only to previous Tokens.

Typical representatives include the GPT series of models.

Their main training objective is usually to predict the next Token:

\mathcal{L}(\theta)= -\sum_{t=1}^{T} \log P_\theta(x_t|x_{<t})

Where:

  • \theta: model parameters.
  • x_t: the target Token.
  • x_{<t}: the historical context.
  • P_\theta: the conditional probability predicted by the model.

The Decoder-only architecture is very well suited to continuous text generation, and it is also an important mainstream architecture for current generative large language models.

5.3 Encoder-Decoder

Encoder-Decoder consists of two main parts:

  • Encoder: understands and encodes the input sequence.
  • Decoder: generates the target sequence based on the encoded information.

The Decoder usually obtains the Encoder's output representations through Cross-Attention.

Typical representatives include T5, the original Transformer, and others.

It is suitable for tasks with a clear input-output relationship, such as translation, summarization, and text transformation.

6. How Are Large Language Models Trained?

Training a large language model typically involves stages such as data processing, model initialization, pre-training, post-training, and evaluation.

6.1 Pre-training Data

Large language models are usually trained on large-scale data, including:

  • Web page text.
  • Books and public documents.
  • Academic papers.
  • News and encyclopedia content.
  • Code repositories.
  • Mathematical and technical materials.
  • Licensed or purpose-built datasets.

The quality of training data directly affects the model's knowledge coverage, language ability, and generation quality.

Data processing usually includes:

  1. Data collection.
  2. Format parsing.
  3. Deduplication.
  4. Quality filtering.
  5. Handling of sensitive information.
  6. Tokenization.
  7. Data mixing and sampling.
  8. Organization of training data.

6.2 Pre-training

Pre-training is a key stage in which a large language model builds general language ability.

For autoregressive models, the core task is to predict the next Token based on the preceding text.

For example:

Input: A software engineer needs to master
Target: programming, design, testing...

Through a large number of training samples, the model continuously adjusts its parameters so that the predicted probability of the target Token increases.

The core objective of pre-training can be expressed as:

\theta^*= \arg\min_\theta \mathbb{E}_{X\sim D} \left[ -\log P_\theta(X) \right]

Where:

  • D: the training data distribution.
  • X: a training text sequence.
  • \theta: model parameters.
  • P_\theta(X): the model's probability estimate for the sequence.

6.3 Loss Function

Autoregressive language models commonly use cross-entropy loss.

For a single target Token:

L=-\log P_\theta(x_t|x_{<t})

The higher the model's predicted probability for the correct Token, the lower the loss.

For a sequence, the losses at each position can be summed or averaged.

During training, the model computes gradients from the loss and then updates its parameters through an optimizer.

6.4 Backpropagation and Parameter Updates

The training process can be summarized as follows:

flowchart TB
    A["Prepare training text"] --> B["Tokenizer converts to Token IDs"]
    B --> C["Model forward pass"]
    C --> D["Compute Logits"]
    D --> E["Compute cross-entropy loss"]
    E --> F["Backpropagation to compute gradients"]
    F --> G["Optimizer updates model parameters"]
    G --> H{"Training condition met?"}
    H -->|"No"| C
    H -->|"Yes"| I["Obtain pretrained model"]

    style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style B fill:#EAF3EC,stroke:#83A995,color:#315D49
    style C fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style D fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style E fill:#EAF3EC,stroke:#83A995,color:#315D49
    style F fill:#EAF3EC,stroke:#83A995,color:#315D49
    style G fill:#EAF3EC,stroke:#83A995,color:#315D49
    style H fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
    style I fill:#DCEFE5,stroke:#4F8B70,color:#24563F

Three concepts need to be distinguished:

Concept Role
Forward propagation Calculate model output based on input
Backpropagation Calculate parameter gradients based on loss
Optimizer Update model parameters based on gradients

Together, these three form the basic training mechanism of neural networks.

7. From Pretrained Model to Chat Model: Post-training

A pretrained model has language modeling capability, but this does not mean it already provides a good conversational experience.

For example, the user inputs:

Please explain what a database index is.

A base pretrained model may continue generating a related passage of text, but it will not necessarily follow a clear question-and-answer format, nor can it reliably follow user instructions.

Therefore, post-training is usually also required.

7.1 Supervised Fine-Tuning: SFT

SFT (Supervised Fine-Tuning) means supervised fine-tuning.

At its core, it uses high-quality instruction and answer data to further train a pretrained model.

Training data typically includes:

  • User instructions.
  • Conversation context.
  • Expected answers.
  • Specific formatting requirements.

For example:

Instruction: Explain what a database index is.

Answer: A database index is a data structure used to improve data query efficiency...

By learning from these samples, the model gradually adapts to instruction following and dialogue tasks.

7.2 Human Preference Alignment

Training solely on standard answers may not fully capture human preferences regarding response quality.

For example, the same question may have multiple correct answers, but they differ in accuracy, completeness, expression, and safety.

Therefore, post-training may also use preference data.

Common methods include:

  • RLHF: Reinforcement learning from human feedback.
  • DPO: Direct preference optimization.
  • RLAIF: Reinforcement learning from AI feedback.

7.3 Basic Mechanism of RLHF

RLHF (Reinforcement Learning from Human Feedback) typically includes the following key steps:

  1. Collect human preference data.
  2. Train a reward model.
  3. Use the reward model to evaluate generated results.
  4. Use reinforcement learning algorithms to optimize the language model.

Typical process:

flowchart TB
    A["Pretrained language model"] --> B["Supervised fine-tuning (SFT)"]
    B --> C["Generate multiple candidate responses"]
    C --> D["Humans compare response quality"]
    D --> E["Build preference data"]
    E --> F["Train reward model"]
    F --> G["Reinforcement learning optimizes language model"]
    G --> H["Obtain aligned model"]

    style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style B fill:#EAF3EC,stroke:#83A995,color:#315D49
    style C fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style D fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style E fill:#EAF3EC,stroke:#83A995,color:#315D49
    style F fill:#EAF3EC,stroke:#83A995,color:#315D49
    style G fill:#EAF3EC,stroke:#83A995,color:#315D49
    style H fill:#DCEFE5,stroke:#4F8B70,color:#24563F

Note that RLHF is not a training method that every large language model must adopt. Different models may use SFT, DPO, reinforcement learning, or a combination of other post-training methods.

VIII. How Do Large Language Models Generate Answers?

After understanding the training process, you also need to understand how the model works when it is actually used.

Inference refers to the process of using an already trained model to compute an output from an input.

Unlike training, the ordinary inference stage usually does not update the model parameters.

8.1 Input Processing

User input:

What is machine learning?

The model first converts the text into Token IDs using a Tokenizer.

It then maps the Token IDs to vectors through Embedding and feeds them into the Transformer network.

8.2 Computing Logits

After multiple layers of computation in the Transformer, the model obtains the contextual hidden states.

For autoregressive generation, the model typically uses the hidden state at the last position and maps it to the vocabulary dimension through the output projection layer (LM Head):

logits=hW_{vocab}+b

Where:

  • h: the hidden state at the current position.
  • W_{vocab}: the output projection matrix.
  • b: the bias term.
  • logits: the unnormalized score corresponding to each Token in the vocabulary.

Logits are not probabilities.

8.3 Converting to Probabilities with Softmax

Softmax can convert logits into a probability distribution:

P(x_i)= \frac{e^{z_i}}{\sum_j e^{z_j}}

Where:

  • z_i: the logit of the i-th Token.
  • P(x_i): the probability corresponding to that Token.

The model thus obtains a probability distribution over the entire vocabulary.

8.4 Decoding Strategies

Once the probability distribution is obtained, you still need to decide which Token to select.

Common decoding strategies include:

Strategy Core Principle Characteristics
Greedy Search Selects the highest-probability Token each time Highly deterministic, prone to repetition
Beam Search Keeps multiple candidate sequences Suitable for some sequence generation tasks
Top-K Sampling Samples only from the K Tokens with the highest probabilities Restricts the candidate range
Top-P Sampling Samples from the candidate set whose cumulative probability reaches a threshold The candidate set changes dynamically
Temperature Adjusts the sharpness of the probability distribution Controls the randomness of generation

Temperature is usually applied before Softmax:

P(x_i)= \frac{e^{z_i/T}} {\sum_j e^{z_j/T}}

where T is the temperature parameter.

  • Lower T: the probability distribution is more concentrated, and the output is usually more stable.
  • Higher T: the probability distribution is flatter, and the output is usually more diverse.
  • T approaching zero: the selection behavior tends toward greedy decoding.

Temperature does not add knowledge to the model, nor can it guarantee improved answer accuracy.