Back to series
· 9 min read

What Exactly Are Q, K, and V? From Mathematical Definitions to the Attention Information Interaction Mechanism


1. Why Does Attention Need Q, K, and V?

In Transformer, the core task of Self-Attention is to let the tokens in a sequence exchange information with one another.

Suppose the input sequence is:

I like artificial intelligence

When the model processes a given token, it needs to solve two problems:

  1. Which other tokens should the current token obtain information from?
  2. How should the obtained information participate in computing the representation of the current token?

If the input vectors were used directly to compute similarity and aggregate information, the model's expressive power would be limited.

Transformer introduces three distinct vector representations:

  • Query (Q): the query vector, which represents what features the current token uses to look for relevant information.
  • Key (K): the key vector, which represents what features the current token uses to participate in matching.
  • Value (V): the value vector, which represents what content the current token can provide.

Together, the three form the information-interaction mechanism of Attention.

It can be understood as a retrieval system:

Attention concept Retrieval system analogy Core responsibility
Query Query condition Express what information is currently needed
Key Index information Determine whether it matches the query
Value Retrieved content Provide the information that actually needs to be aggregated

This analogy helps build intuition, but Q, K, and V are essentially all continuous vectors learned by the model; they are not three fixed fields in a database.

2. How Are Q, K, and V Generated?

Q, K, and V are not three attributes looked up directly from a token. Rather, they are generated from the input hidden representation through three different learnable linear projections.

2.1 Input Representation X

Suppose that after Embedding and positional encoding, the input sequence yields the hidden representation:

X\in\mathbb{R}^{n\times d_{model}}

where:

  • n: sequence length.
  • d_{model}: model hidden dimension.

For example:

n=4,\quad d_{model}=512

then:

X\in\mathbb{R}^{4\times512}

This can be understood as:

  • The sequence contains 4 tokens.
  • Each token corresponds to a 512-dimensional vector.

If the input has already passed through preceding Transformer layers, then X is the hidden state output by the previous layer, rather than the original Token Embedding.

2.2 Three Different Projection Matrices

The model defines three independent learnable weight matrices:

W_Q\in\mathbb{R}^{d_{model}\times d_k}
W_K\in\mathbb{R}^{d_{model}\times d_k}
W_V\in\mathbb{R}^{d_{model}\times d_v}

and computes, respectively:

Q=XW_Q
K=XW_K
V=XW_V

Therefore:

Q\in\mathbb{R}^{n\times d_k}
K\in\mathbb{R}^{n\times d_k}
V\in\mathbb{R}^{n\times d_v}

In actual implementations, the linear layers may also include bias terms. The formulas above omit bias for brevity.

2.3 Generation Process

flowchart TB
    A["Input hidden representation X"] --> B["Linear projection"]
    B --> C["WQ"]
    B --> D["WK"]
    B --> E["WV"]
    C --> F["Query"]
    D --> G["Key"]
    E --> H["Value"]

    style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style B fill:#EAF3EC,stroke:#83A995,color:#315D49
    style C fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style D fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style E fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style F fill:#E5F0E9,stroke:#83A995,color:#315D49
    style G fill:#E5F0E9,stroke:#83A995,color:#315D49
    style H fill:#E5F0E9,stroke:#83A995,color:#315D49

Here is a very important conclusion:

Q, K, and V come from the same input X, but they use different projection matrices, and thus form different representation spaces.

These three are not three independent pieces of raw data.

III. Query: What does the current Token want to look up?

The core responsibility of Query is to initiate matching.

For the i-th Token in the sequence:

q_i=x_iW_Q

Where:

  • x_i: the input hidden representation of the i-th Token.
  • W_Q: the Query projection matrix.
  • q_i: the Query vector of the i-th Token.

Query represents the features that the current Token uses in the current attention layer to look for relevant information.

3.1 Query is not the same as a natural-language question

It is easy to fall into a misunderstanding: that Query is a question generated internally by the model.

In fact, it is not.

Query is a high-dimensional floating-point vector, for example:

q_i= [0.12,-0.38,0.71,0.05,\ldots]

It has no directly readable natural-language meaning.

The model learns W_Q through training, so that Query can form useful matching representations within the current task and network structure.

3.2 Every Token has its own Query

Suppose the input has three Tokens:

X= \begin{bmatrix} x_1\\ x_2\\ x_3 \end{bmatrix}

Then:

Q= \begin{bmatrix} x_1W_Q\\ x_2W_Q\\ x_3W_Q \end{bmatrix}

Every Token generates its own Query.

This means each position can independently initiate an attention query.

3.3 The core role of Query

Query determines how the current Token matches against the Keys of other Tokens.

The same Token may have different input hidden representations in different contexts, so the Query it generates may also differ.

In different Transformer layers, because the input hidden representation keeps changing, Query is also recomputed.

IV. Key: How do other Tokens participate in matching?

The core responsibility of Key is to provide the basis for matching.

For the j-th Token:

k_j=x_jW_K

Where:

  • x_j: the input hidden representation of the j-th Token.
  • W_K: the Key projection matrix.
  • k_j: the Key vector of the j-th Token.

If Query is understood as the query condition, then Key is the index representation that participates in matching.

4.1 Key is not a unique identifier of a Token

Key is not a Token ID, nor is it a fixed word index.

For example, Token IDs might be:

101
205
309

Whereas Key is a continuous vector obtained by linearly projecting the hidden representation:

k_j\in\mathbb{R}^{d_k}

Its values change with the model parameters, the input context, and the network layer.

4.2 Key determines the matching space

Query and Key must lie in compatible dimensional spaces so that a dot product can be computed.

Suppose:

q_i\in\mathbb{R}^{64}
k_j\in\mathbb{R}^{64}

Then we can compute:

s_{ij}=q_i k_j^T

The dot product result is a scalar, used to represent the matching score of these two vectors in the current attention space.

If the dimensions of Q and K do not match, standard dot-product attention cannot be computed directly.

4.3 Key and Query form a relative matching relationship

Key itself has no fixed attention score independent of Query.

The same Key:

  • may receive different scores when facing different Queries.
  • may have different representations in different attention heads.
  • may change across different network layers.

Therefore, the Attention Score is determined jointly by Query and Key, not by Key alone.

V. Value: What information is actually aggregated?

The core responsibility of Value is to provide the vector content that actually participates in information fusion.

For the j-th Token:

v_j=x_jW_V

Where:

  • x_j: the input hidden representation of the j-th Token.
  • W_V: the Value projection matrix.
  • v_j: the Value vector of the j-th Token.

In Attention, Query and Key are responsible for computing the weights, while Value provides the content to be weighted and aggregated.

5.1 Why can't we just aggregate Key directly?

The main responsibility of Key is matching, whereas the main responsibility of Value is content expression.

If Key were used directly as the aggregated content, the matching space and the information-expression space would be tied together.

By separating the two, the model can learn separately:

  • what kinds of features are suitable for establishing associations.
  • what kinds of information are suitable for passing to other Tokens.

This increases the model's expressive flexibility.

5.2 Value is not the same as the raw Token Embedding

Value is the result of projecting the input hidden representation through W_V:

V=XW_V

It does not necessarily preserve the meaning of every dimension in the original Embedding.

In a multi-layer Transformer, the input to Value comes from the hidden representation of the previous layer, so it may already have absorbed several rounds of contextual information.

5.3 How Does Value Participate in the Output Computation?

Suppose the attention weights of a given Query over three Tokens are:

\alpha=[0.2,0.3,0.5]

The corresponding Values are:

v_1=[1,0]
v_2=[0,2]
v_3=[2,1]

The final output is:

o=0.2v_1+0.3v_2+0.5v_3

Expanding:

o= 0.2[1,0]+ 0.3[0,2]+ 0.5[2,1]

We get:

o=[1.2,1.1]

This result is the context representation that the current Query aggregates from the three positions.

Six. How Do Q, K, and V Work Together?

The most important thing in understanding these three vectors is not to memorize their definitions one by one, but to understand how they cooperate.

The complete computation process:

Q=XW_Q
K=XW_K
V=XW_V

First compute the attention scores from Q and K:

S=QK^T

Then apply scaling and normalization:

A=Softmax\left( \frac{QK^T}{\sqrt{d_k}} \right)

Finally, aggregate the Values:

O=AV

The full formula:

\boxed{ Attention(Q,K,V)= Softmax\left( \frac{QK^T}{\sqrt{d_k}} \right)V }

The overall process:

flowchart TB
    A["Input X"] --> B["Generate Q"]
    A --> C["Generate K"]
    A --> D["Generate V"]

    B --> E["Dot product of Q and K"]
    C --> E

    E --> F["Scaled scores"]
    F --> G["Softmax"]
    G --> H["Attention weights A"]

    H --> I["Multiply A by V"]
    D --> I
    I --> J["Output O"]

    style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style B fill:#EAF3EC,stroke:#83A995,color:#315D49
    style C fill:#EAF3EC,stroke:#83A995,color:#315D49
    style D fill:#EAF3EC,stroke:#83A995,color:#315D49
    style E fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style F fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style G fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
    style H fill:#EAF3EC,stroke:#83A995,color:#315D49
    style I fill:#EAF3EC,stroke:#83A995,color:#315D49
    style J fill:#E5F0E9,stroke:#83A995,color:#315D49

Here we need to distinguish two stages:

Stage one: computing whom to attend to.

The attention weights are obtained from how well Q matches K.

Stage two: what information to obtain.

Use the attention weights to compute a weighted sum over V.

Therefore:

  • Q determines the direction of the query.
  • K determines the basis for matching.
  • V determines the content that gets aggregated.

Seven. Why Do Q, K, and V Use Different Weight Matrices?

This is the key question for understanding the design philosophy behind Attention.

Suppose there were only one projection matrix:

Q=K=V=XW

Then the matching space and the content space would be forced into the same projection result.

Such a structure is not entirely unworkable, but its expressive power would be constrained.

Transformer uses three independent projection matrices:

W_Q,\quad W_K,\quad W_V

so that the model can learn different ways of representing things separately.

7.1 Separating Matching from Content

Query and Key mainly take part in computing the attention scores:

QK^T

Value mainly takes part in information aggregation:

AV

This separation allows the model to learn:

  • which features are suitable for matching.
  • which features are suitable for conveying information.
  • how to dynamically combine content based on the matching results.

7.2 Learning Automatically Through Training

The projection matrices of Q, K, and V are all learnable parameters.

During training, the model backpropagates based on the task loss and updates these matrices.

The training process never directly tells the model:

  • Which dimension represents syntax.
  • Which dimension represents semantics.
  • Which dimension represents entity relations.

The model gradually forms internal representations that are useful for the task, driven by data and the optimization objective.

Therefore, one should not directly interpret a particular Q, K, or V dimension as a fixed human semantic concept.

8. Why are the dimensions of Q, K, and V usually the same?

Standard dot-product attention requires:

Q\in\mathbb{R}^{n\times d_k}
K\in\mathbb{R}^{n\times d_k}

Because when computing:

QK^T

the feature dimensions of Q and K must match.

The dimension of Value, however, can be different:

V\in\mathbb{R}^{n\times d_v}

Finally:

O=AV

where:

A\in\mathbb{R}^{n\times n}

So:

O\in\mathbb{R}^{n\times d_v}

This means:

  • The dimensions of Q and K must be consistent in order to compute the standard dot product.
  • The dimension of V need not be the same as that of Q and K.
  • The output dimension is determined by the dimension of V.

In the multi-head attention of the classic Transformer, one usually sets:

d_k=d_v=\frac{d_{model}}{h}

where h is the number of attention heads.

9. Q, K, V in Multi-Head Attention

In Multi-Head Attention, Q, K, and V are not generated as just one set; rather, multiple sets of representations are generated through different projection matrices.

Suppose:

d_{model}=768
h=12

Then the dimension of each Head is:

d_k=d_v=64

For each attention head:

Q_i=XW_i^Q
K_i=XW_i^K
V_i=XW_i^V

Compute respectively:

head_i=Attention(Q_i,K_i,V_i)

The outputs of all Heads are concatenated:

MultiHead= Concat(head_1,\ldots,head_h)W^O

9.1 Dimensions of multi-head Q, K, V

Suppose:

  • Batch Size: 32.
  • Sequence length: 128.
  • Hidden dimension: 768.
  • Number of Heads: 12.
  • Dimension per Head: 64.

Input:

X\in\mathbb{R}^{32\times128\times768}

Project and split into multiple heads:

Q,K,V\in \mathbb{R}^{32\times12\times128\times64}

Attention scores:

QK^T\in \mathbb{R}^{32\times12\times128\times128}

Multiply the attention weights by V:

AV\in \mathbb{R}^{32\times12\times128\times64}

Multi-head concatenation:

O\in\mathbb{R}^{32\times128\times768}

9.2 Why is multi-head valuable?

Different attention heads use different projection matrices, so they can learn different matching and information-aggregation patterns in different representation subspaces.

However, one cannot simply assume that each Head corresponds to a definite syntactic, semantic, or anaphoric function. In practice, attention heads may overlap in function and may also exhibit complex compositional behavior.

10. Q, K, V in Self-Attention and Cross-Attention

The sources of Q, K, and V depend on the type of Attention.

10.1 Self-Attention

In Self-Attention:

Q=XW_Q
K=XW_K
V=XW_V

All three come from the same set of input hidden representations.

Typical applications:

  • Transformer Encoder.
  • The Decoder's Masked Self-Attention.
  • Decoder-only large language models.

10.2 Cross-Attention

In Cross-Attention, Q comes from one sequence, while K and V come from another sequence.

For example, in an Encoder-Decoder architecture:

Q=H_{dec}W_Q
K=H_{enc}W_K
V=H_{enc}W_V

where:

  • H_{dec}: the Decoder's current hidden representation.
  • H_{enc}: the Encoder's output representation.

This enables the Decoder to extract relevant information from the Encoder output based on its current generation state.

10.3 Comparison

Comparison Dimension Self-Attention Cross-Attention
Query Source Current sequence One sequence
Key Source Current sequence Another sequence
Value Source Current sequence Another sequence
Information Interaction Within the sequence Between different sequences
Typical Use Context modeling Information interaction between Encoder and Decoder

Note that modern Decoder-only models usually use Causal Self-Attention, rather than Cross-Attention in the classic Encoder-Decoder architecture.

11. Relationship Between Q, K, V, and Positional Encoding

Q, K, and V themselves do not inherently contain explicit Token order information.

If only ordinary dot-product Self-Attention is used, without an additional positional mechanism, then the model cannot, based solely on this computation, distinguish certain inputs that differ only in Token order.

Therefore, Transformer needs to introduce positional information.

Common approaches include:

  • Sinusoidal positional encoding.
  • Learnable absolute positional encoding.
  • Relative positional encoding.
  • RoPE (Rotary Position Embedding).
  • ALiBi.

11.1 Absolute Positional Encoding

The classic Transformer adds positional encoding to the Token Embedding:

X=Embedding+PositionEncoding

It then generates:

Q=XW_Q
K=XW_K
V=XW_V

Therefore, positional information affects Q, K, and V through the input representation.

11.2 RoPE

RoPE works differently.

It usually applies a rotation transformation to Q and K according to the Token position after generating Q and K:

Q'=RoPE(Q,pos)
K'=RoPE(K,pos)

Attention is then computed using the rotated Q and K.

Standard RoPE usually does not directly apply the same rotation to V.

An important property is that the dot product of two position vectors can reflect relative positional information.

Therefore, in modern large language models, one cannot simply assume that all positional encodings are applied to the original Embedding via addition.

12. Q, K, V and KV Cache

During large language model inference, K and V also have a very important engineering use: caching.

12.1 Why Cache K and V?

Autoregressive generation is usually performed Token by Token.

For example:

Step 1: Generate Token 1
Step 2: Generate Token 2
Step 3: Generate Token 3