Back to series
· 13 min read

Optimizing RAG Recall and Accuracy: From Retrieval Evaluation to Full-Pipeline Quality Improvement


I. Core Issues in RAG Retrieval Quality

The core goal of RAG (Retrieval-Augmented Generation) is to find evidence relevant to the user's question in an external knowledge base before generating an answer.

But a retrieval system is not successful simply because it finds relevant documents.

In real production environments, the following problems occur frequently:

  • The user's question touches on multiple topics, but the retrieval results cover only some of them.
  • The retrieval results contain many documents that are topically related but cannot answer the question.
  • The correct document exists in the knowledge base but never enters the candidate set.
  • Relevant documents are ranked too low and ultimately never make it into the LLM context.
  • The retrieval results themselves are correct, but the chunk lacks the necessary context.
  • An outdated version of a document is retrieved, causing the answer to be inconsistent with the current business state.

These problems can be reduced to two core dimensions:

Recall: Did we find what should have been found?

Precision: Of what was found, how much is truly relevant?

In the field of information retrieval, Precision is usually translated as 精确率, sometimes called 查准率; Recall is usually translated as 召回率, also known as 查全率.

Together, the two determine whether RAG can provide the large model with high-quality context.

II. Basic Definitions of Recall and Precision

2.1 The Four Retrieval Result States

Suppose there is a query and the system returns a batch of documents.

From the perspective of relevance, the results can be divided into four types:

Type Meaning
TP (True Positive) Relevant documents that were retrieved
FP (False Positive) Irrelevant documents that were retrieved
FN (False Negative) Relevant documents that were not retrieved
TN (True Negative) Irrelevant documents that were not retrieved

Where:

  • TP: correct content was found.
  • FP: wrong or irrelevant content was found.
  • FN: content that should have been found was missed.
  • TN: content that was irrelevant to begin with was not found.

Note that retrieval systems usually return only the Top-K candidates, so TN is hard to enumerate completely in real retrieval evaluation. Precision and Recall are mainly computed from TP, FP, and FN.

2.2 Precision

Precision measures the proportion of relevant documents among the retrieved results.

Precision=\frac{TP}{TP+FP}

For example:

The system retrieves 10 documents, of which:

  • 8 are relevant.
  • 2 are irrelevant.

Then:

Precision=\frac{8}{10}=0.8

That is, the precision is 80%.

The higher the precision, the less irrelevant content there is in the retrieval results.

2.3 Recall

Recall measures the proportion of all relevant documents that the system successfully retrieves.

Recall=\frac{TP}{TP+FN}

For example:

The knowledge base contains a total of 20 documents relevant to the query, but the system retrieves only 8 of them.

Then:

Recall=\frac{8}{20}=0.4

That is, the recall is 40%.

The higher the recall, the less relevant content the system misses.

2.4 The Difference Between the Two

Dimension Precision Recall
Chinese name 精确率 召回率
Focus Whether the retrieval results are relevant Whether all relevant content was found
Denominator Number of documents retrieved Number of all relevant documents
Main problem How much irrelevant content is introduced How much relevant content is missed
Low-value behavior More noise Missing evidence
Optimization direction Improve ranking quality, filter noise Expand coverage, improve recall

A simple way to remember:

  • Precision: Of the things you found, how many are correct?
  • Recall: Of the things you should have found, how many were found?

III. The Trade-off Between Precision and Recall

Recall and precision usually have a certain trade-off relationship.

Suppose the system returns only the top 3 ranked documents:

  • All 3 are relevant.
  • Precision = 100%.
  • But it may miss many relevant documents, so Recall is low.

If the system returns the top 100 ranked documents:

  • It may find more relevant documents.
  • Recall may increase.
  • But it may also introduce many irrelevant documents, lowering Precision.

3.1 The Impact of Top-K on Retrieval Quality

Suppose a query corresponds to 10 labeled relevant documents:

Top-K Relevant Documents Retrieved Precision@K Recall@K
5 4 80% 40%
10 7 70% 70%
20 9 45% 90%
30 10 33.3% 100%

As K increases, recall coverage usually improves, but precision may drop.

This assumes that irrelevant documents are gradually introduced among the newly added candidates. The specific trend depends on the ranking results, and it does not necessarily hold for every dataset.

3.2 Precision-Recall Curve

You can observe how Precision and Recall change by varying the retrieval threshold or Top-K.

Illustrative data, used only to explain the relationship between the two.

Recall Precision
0.1 0.98
0.2 0.95
0.3 0.91
0.4 0.86
0.5 0.80
0.6 0.72
0.7 0.63
0.8 0.52
0.9 0.38
1.0 0.25

The closer the curve is to the upper right, the better the overall performance of the retrieval system usually is.

3.3 F1 Score

If you want to measure Precision and Recall together, you can use the F1 Score.

F1=2\times \frac{Precision\times Recall}

F1 is the harmonic mean of Precision and Recall.

For example:

Precision=0.8
Recall=0.6

Then:

F1=2\times\frac{0.8\times0.6}{0.8+0.6} \approx0.686

F1 is suitable for a combined evaluation when the two are equally important.

If the business cares more about recall, you can use F-beta:

F_\beta=(1+\beta^2) \frac{Precision\times Recall}

Where:

  • \beta>1: greater emphasis on Recall.
  • \beta<1: greater emphasis on Precision.
  • \beta=1: equivalent to F1.

Note: F1 and F-beta cannot replace a complete ranking evaluation; they mainly measure classification-style retrieval performance under a given threshold or candidate set.

4. At Which Stages Should RAG Retrieval Quality Be Evaluated?

RAG is not a single vector retrieval operation, but a complete pipeline composed of multiple stages.

Retrieval quality issues can occur at:

  1. Document parsing stage.
  2. Chunk splitting stage.
  3. Embedding stage.
  4. Initial recall stage.
  5. Multi-path result fusion stage.
  6. Rerank stage.
  7. Final context construction stage.

Therefore, you cannot judge whether the retrieval system is working properly solely by whether the final answer is correct.

4.1 Complete RAG Quality Pipeline

flowchart TD
    A["Raw knowledge documents"] --> B["Document parsing"]
    B --> C["Chunk splitting"]
    C --> D["Embedding and index building"]

    D --> E["User query"]
    E --> F["Query processing"]

    F --> G["Initial recall"]
    G --> H["Candidate document set"]
    H --> I["Result fusion"]
    I --> J["Rerank"]
    J --> K["Context filtering"]

    K --> L["LLM answer generation"]
    L --> M["Answer quality evaluation"]

    H --> N["Recall and precision evaluation"]
    J --> O["Ranking quality evaluation"]
    K --> P["Context quality evaluation"]
    M --> Q["End-to-end effectiveness evaluation"]

    style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style B fill:#EAF3EC,stroke:#83A995,color:#315D49
    style C fill:#EAF3EC,stroke:#83A995,color:#315D49
    style D fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style E fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style F fill:#EAF3EC,stroke:#83A995,color:#315D49
    style G fill:#EAE8FA,stroke:#A49BD5,color:#51477D
    style H fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style I fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style J fill:#EAE8FA,stroke:#A49BD5,color:#51477D
    style K fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
    style L fill:#EAF3EC,stroke:#83A995,color:#315D49
    style M fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style N fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style O fill:#EAE8FA,stroke:#A49BD5,color:#51477D
    style P fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
    style Q fill:#DCEFE5,stroke:#4F8B70,color:#24563F

4.2 Stage-by-Stage Evaluation

Stage Key Metrics Main Issue
Document Parsing Parsing Completeness Rate, Field Completeness Rate Whether content is lost
Chunking Chunk Integrity, Boundary Quality Whether semantics are cut off
Initial Retrieval Recall@K, Hit Rate@K Whether relevant documents are found
Result Fusion Recall@K, Candidate Coverage Rate Whether multi-route results are complementary
Rerank MRR, NDCG@K Whether relevant documents are ranked higher
Context Construction Context Relevance, Coverage Rate Whether necessary evidence is retained
Answer Generation Faithfulness, Answer Relevance Whether the answer is grounded

This staged evaluation approach helps locate problems rather than only observing the final generated output.

V. Core Metrics for RAG Retrieval Evaluation

In addition to Precision and Recall, real-world RAG systems also need to pay attention to ranking, coverage, and latency.

5.1 Recall@K

Recall@K indicates how many relevant documents are covered by the top K retrieval results.

Recall@K= \frac{|Relevant\cap Retrieved_K|}

For example:

  • Number of labeled relevant documents: 10.
  • The system's top 10 results hit: 7.

Then:

Recall@10=\frac{7}{10}=0.7

Recall@K is an important metric for measuring initial retrieval quality.

5.2 Precision@K

Precision@K indicates the proportion of relevant documents among the top K results.

Precision@K= \frac{|Relevant\cap Retrieved_K|}{K}

For example:

  • Among the top 10 results, 8 are relevant documents.

Then:

Precision@10=\frac{8}{10}=0.8

5.3 Hit Rate@K

Hit Rate@K measures whether the top K results contain at least one relevant document.

For a single query:

Hit@K= \begin{cases} 1,& Retrieved_K\cap Relevant\neq\emptyset\\ 0,& \text{otherwise} \end{cases}

Averaged over multiple queries:

HitRate@K= \frac{1}{|Q|} \sum_{q\in Q}Hit@K(q)

Hit Rate is particularly suitable for evaluating whether a question-answering system can find at least one piece of valid evidence.

However, it does not care how many relevant documents are found, nor does it fully reflect ranking quality.

5.4 MRR

MRR (Mean Reciprocal Rank) measures the position where the first relevant result appears.

MRR= \frac{1}{|Q|} \sum_{i=1}^{|Q|} \frac{1}{rank_i}

For example:

Query Rank of First Relevant Result Reciprocal Rank
Q1 1 1
Q2 2 0.5
Q3 4 0.25

Then:

MRR=\frac{1+0.5+0.25}{3} \approx0.583

MRR is suitable for evaluating whether the first piece of valid evidence can appear as early as possible.

5.5 NDCG@K

NDCG (Normalized Discounted Cumulative Gain) is used to evaluate ranking quality and supports relevance labels of different grades.

DCG@K= \sum_{i=1}^{K} \frac{2^{rel_i}-1}{\log_2(i+1)}
NDCG@K= \frac{DCG@K}{IDCG@K}

Where:

  • rel_i: the relevance grade at the i-th position.
  • IDCG@K: the DCG under the ideal ranking.

For example:

Level Meaning
3 Highly relevant; can directly answer
2 Relevant; contains important evidence
1 Partially relevant; only supporting information
0 Not relevant

NDCG is particularly suitable for evaluating a Reranker because it cares not only whether documents are relevant, but also whether highly relevant documents are ranked near the top.

5.6 How to Choose Metrics?

Optimization Goal Recommended Metric
Find more relevant evidence Recall@K
Reduce irrelevant content Precision@K
Find at least one piece of evidence Hit Rate@K
Place the first relevant result as high as possible MRR
Overall ranking quality NDCG@K
End-to-end response speed Latency
Retrieval service capacity Throughput

In production, you should not choose only one metric; instead, build a metric combination based on the business.

VI. Stage One: Improve Initial Recall

The core of improving Recall is to get more truly relevant documents into the candidate set.

But two questions need to be distinguished:

  • Does the document exist in the knowledge base?
  • Can the document be recalled by the current retrieval strategy?

If the correct content never enters the knowledge base, adjusting vector retrieval parameters will not solve the problem.

6.1 Optimize Document Parsing

Document parsing is the foundation of retrieval quality.

Common data sources:

  • PDF.
  • Word.
  • Markdown.
  • HTML.
  • Excel.
  • PPT.
  • Database records.
  • API documentation.

During parsing, pay attention to:

  1. The hierarchical relationship between headings and body text.
  2. Whether table content is complete.
  3. Whether code blocks are preserved.
  4. Whether headers and footers contaminate the body text.
  5. Whether text in images is recognized.
  6. Whether the reading order of multi-column documents is correct.
  7. Whether metadata is preserved.

For example, a technical document:

# KV Cache

## How It Works

KV Cache is used to cache the Key and Value of historical Tokens.

## Advantages

Reduce redundant computation during the Decode phase.

If headings are lost during parsing, it may affect the semantic completeness of subsequent Chunks.

If the table structure is broken, data that originally had clear correspondences may become incomprehensible text.

6.2 Optimize the Chunking Strategy

Chunking determines the smallest content unit retrieved from the knowledge base.

Common strategies:

Strategy Characteristics Use Cases
Fixed-length splitting Split by Token count General text
Recursive character splitting Prefer splitting along structural boundaries Markdown, general documents
Semantic splitting Divide based on semantic shifts Long text
Structured splitting Split based on structures such as headings, paragraphs, and tables Technical documents
Parent-child Chunk Small chunks for retrieval, large chunks to supplement context Complex knowledge bases

If a Chunk is too short:

  • Semantics are incomplete.
  • Preceding definitions are missing.
  • Contextual relationships may be lost.
  • More fragments are needed to answer complex questions.

If a Chunk is too long:

  • Topics are mixed.
  • Relevant content is diluted by a large amount of irrelevant content.
  • The Embedding representation may not be focused enough.
  • Context cost increases.

There is no universal optimal Chunk size; it should be evaluated based on document type, query pattern, Embedding model, and context window.

6.3 Overlap Splitting

Overlap is the repeated text retained between adjacent Chunks.

For example:

Chunk A:
Transformer uses the Self-Attention mechanism,
calculating associations between different Tokens.

Chunk B:
calculating associations between different Tokens.
Multi-head attention can extract information from multiple representation subspaces.

Overlap can reduce information loss caused by splitting boundaries.

However, excessive Overlap leads to:

  • Increased duplicate content.
  • Vector index bloat.
  • Highly duplicated retrieval results.
  • The Reranker wastes computational resources.

Therefore, the overlap ratio needs to be controlled, and deduplication should be performed after retrieval.

6.4 Supplementing Headings and Context

If a Chunk contains only body text, it may lack information about the section to which it belongs.

For example:

Cache the Key and Value of historical Tokens to reduce redundant computation.

Without context, it is not easy to tell whether this sentence discusses KV Cache, database caching, or some other caching mechanism.

The title can be added:

Document: LLM Inference Optimization

Section: How KV Cache Works

Content:
Cache the Key and Value of historical Tokens to reduce redundant computation.

This approach can give the Embedding and subsequent ranking models more complete semantic information.

7. Stage Two: Optimizing Embedding and Vector Recall

Vector retrieval is an important component of modern RAG.

Its core is to use an Embedding model to map Query and Document into a vector space, and then retrieve by vector similarity.

7.1 Embedding Model Selection

Different Embedding models have different capabilities.

Consider the following when choosing:

  • Chinese and English support.
  • Multilingual capability.
  • Technical domain knowledge.
  • Long-text support.
  • Vector dimensions.
  • Inference latency.
  • Deployment cost.
  • Input format requirements for queries and documents.

Do not choose a model based solely on model size or public leaderboards.

A more reliable approach is to build an evaluation set using your own real queries and documents, and compare different models on Recall@K, NDCG@K, and latency.

7.2 Representation Consistency for Query and Document

Queries and documents must use compatible Embedding representations.

Note the following:

  • Use compatible model versions.
  • Follow the Query / Document prefixes required by the model.
  • Keep preprocessing rules consistent.
  • Use the same vector dimensions.
  • Clarify whether L2 normalization is performed.
  • Consider regenerating document vectors when upgrading the model.

If document vectors are generated with model A and query vectors are generated with an incompatible model B, even if the dimensions are the same, the two cannot be considered to lie in the same semantic space.

7.3 Similarity Metrics

Common vector similarity measures include:

Cosine similarity:

Cosine(A,B)= \frac{A\cdot B}{\|A\|\|B\|}

Inner product:

IP(A,B)=A\cdot B

Euclidean distance:

L2(A,B)= \sqrt{\sum_{i=1}^{d}(A_i-B_i)^2}

Note the following:

  • The larger the cosine similarity, the closer the directions usually are.
  • The larger the inner product, the higher the matching degree usually is, but it is also affected by vector magnitude.
  • The smaller the L2 distance, the closer the two vectors usually are.
  • For unit-normalized vectors, cosine similarity and inner product produce equivalent rankings.

Do not confuse the semantics of the scores returned by different vector databases.

7.4 ANN Parameter Optimization

As the knowledge base grows, ANN (Approximate Nearest Neighbor) algorithms are usually used.

Common index types:

  • HNSW.
  • IVF.
  • PQ.
  • IVF-PQ.

Taking HNSW as an example:

Parameter Purpose Tuning direction
M Controls the number of node connections in the graph Increasing it may improve search quality but also increases memory usage
efConstruction Controls the search width when building the index Increasing it usually improves graph construction quality but makes construction slower
efSearch Controls the search width during queries Increasing it usually improves recall but increases latency

Taking IVF as an example:

  • nlist: number of cluster centers.
  • nprobe: number of clusters probed during a query.

Increasing nprobe usually expands the search range but increases query cost.

ANN parameters should be tuned together with Recall@K and query latency, rather than solely pursuing higher vector similarity scores.

8. Stage Three: Introducing Hybrid Retrieval

Relying solely on vector retrieval may not reliably handle exact terminology, numbers, and technical parameters.

Relying solely on keyword retrieval, in turn, makes it difficult to fully capture semantic relationships.

Therefore, Hybrid Search can be used.

8.1 BM25 + Dense Search

BM25 handles keyword matching.

Dense Search handles semantic matching.

For example, for the query:

How do I enable a certain configuration in PostgreSQL 16.2?

Keyword retrieval can focus on matching:

  • PostgreSQL.
  • 16.2.
  • Configuration item names.
  • Parameter values.

Vector retrieval can then recall relevant configuration descriptions and operational documentation.

8.2 Hybrid Retrieval Architecture

flowchart TD
    A["User Query"] --> B["Query Preprocessing"]

    B --> C["BM25 Keyword Retrieval"]
    B --> D["Dense Vector Retrieval"]

    C --> E["Keyword Candidate Set"]
    D --> F["Semantic Candidate Set"]

    E --> G["Result Fusion"]
    F --> G

    G --> H["Deduplication and Filtering"]
    H --> I["Candidate Document Set"]

    style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style B fill:#EAF3EC,stroke:#83A995,color:#315D49
    style C fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style D fill:#EAE8FA,stroke:#A49BD5,color:#51477D
    style E fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style F fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style G fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
    style H fill:#EAF3EC,stroke:#83A995,color:#315D49
    style I fill:#DCEFE5,stroke:#4F8B70,color:#24563F

8.3 RRF Result Fusion

The raw scores from different retrievers usually cannot be added directly.

RRF (Reciprocal Rank Fusion) calculates a fused score based on the rankings of documents in different retrieval results.

RRF(d)= \sum_{i=1}^{n} \frac{1}{k+rank_i(d)}

Where:

  • d: candidate document.
  • rank_i(d): the rank of the document in the i-th retriever.
  • k: smoothing constant.
  • n: number of retrievers.

RRF does not need to compare the raw scores of BM25 and vector retrieval, making it suitable for fusing ranking results from different retrievers.

However, it also does not consider the actual gap between raw scores, so it is not the optimal choice in all scenarios.

IX. Stage 4: Improving Accuracy with Reranker

Initial recall primarily addresses coverage.

However, the candidate set may still contain a large amount of content that is topically similar but actually irrelevant.

The role of the Reranker is to make a more fine-grained relevance judgment on candidate documents.

9.1 Why Can Reranker Improve Accuracy?

Vector retrieval usually uses a Bi-Encoder:

  • Query is encoded separately.
  • Document is encoded separately.
  • Finally, matching is performed via vector similarity.

A common Reranker implementation uses a Cross-Encoder:

  • Query and Document are input jointly.
  • Joint Self-Attention is performed inside the Transformer.
  • A relevance score is output.

Therefore, the Reranker can more fully determine whether the query conditions match the document content.

9.2 Rerank Workflow

flowchart TD
    A["Initial Retrieval Candidate Set"] --> B["Construct Query-Document Pair"]
    B --> C["Cross-Encoder"]

    C --> D["Joint Attention Computation"]
    D --> E["Relevance Prediction"]
    E --> F["Relevance Score"]

    F --> G["Sort by Score Descending"]
    G --> H["Select Final Top-K"]

    style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style B fill:#EAF3EC,stroke:#83A995,color:#315D49
    style C fill:#EAE8FA,stroke:#A49BD5,color:#51477D
    style D fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style E fill:#EAF3EC,stroke:#83A995,color:#315D49
    style F fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
    style G fill:#EAF3EC,stroke:#83A995,color:#315D49
    style H fill:#DCEFE5,stroke:#4F8B70,color:#24563F

9.3 Reranker Optimization Priorities

  • Use a model that matches the language of the business.
  • Use domain-relevant evaluation data.
  • Control the number of candidate documents.
  • Optimize Batch inference.
  • Control the input Token length.
  • Improve ranking capability through hard negative samples.
  • Validate the validity of model outputs.

Pay special attention:

A Reranker can only re-rank candidate documents that have already been recalled; it cannot recover evidence that was not recalled in the first stage.

Therefore, you cannot compensate for insufficient initial recall capability by continuously increasing the size of the Reranker model.

10. Stage 5: Optimizing Query Understanding and Query Rewriting

Many retrieval failures are not because the knowledge base lacks content, but because the user's query and the knowledge base documents are expressed differently.

For example:

User query:

Why does AI remember previous content when answering questions?

Knowledge base document:

KV Cache is used to cache the Keys and Values of historical Tokens during autoregressive generation.

If retrieval is performed directly, colloquial expressions in the query may fail to accurately match professional documents.

Therefore, Query Understanding and Query Rewriting can be used to improve the retrieval input.

10.1 Query Rewrite

Convert user questions into expressions more suitable for retrieval.

Original question:

Why does AI remember previous content when answering questions?

After rewriting:

During autoregressive generation by large language models,
how does KV Cache cache the Keys and Values of historical Tokens?

Rewriting can add professional terminology and clarify the query topic.

However, avoid introducing constraints during rewriting that the user did not express.

10.2 Query Expansion

Query Expansion broadens retrieval coverage by adding synonyms, related terms, or associated concepts.

For example:

Original query:
Python multithreading performance issues

Expanded queries:
Python multithreading performance
GIL Global Interpreter Lock
CPU-intensive tasks
Python multiprocessing
Concurrent thread execution

Expanded queries can be retrieved separately, and then the results can be fused.

However, more expansion terms are not always better. Irrelevant expansions may increase noise in the candidate set.

10.3 Multi-Query Retrieval

For complex questions, multiple queries from different perspectives can be generated.

For example:

Original question:

Why does RAG sometimes fail to find the correct answer?

Break it down into:

  1. Why does vector retrieval in RAG miss relevant documents?
  2. How does chunk splitting affect RAG recall?
  3. How do embedding models affect semantic retrieval?
  4. How does a reranker affect RAG ranking quality?

Execute the retrievals separately, then fuse the results.

10.4 Query Rewriting Process

flowchart TD
    A["Original user Query"] --> B["Query Understanding"]
    B --> C["Identify topic and key entities"]
    C --> D{"Is query rewriting needed?"}

    D -->|"No"| E["Use original Query"]
    D -->|"Yes"| F["Query Rewrite"]

    F --> G["Query Expansion"]
    G --> H["Generate multiple retrieval Queries"]

    E --> I["Execute retrieval"]
    H --> I

    I --> J["Fuse multi-path results"]
    J --> K["Candidate document set"]

    style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style B fill:#EAF3EC,stroke:#83A995,color:#315D49
    style C fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style D fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
    style E fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style F fill:#EAE8FA,stroke:#A49BD5,color:#51477D
    style G fill:#EAE8FA,stroke:#A49BD5,color:#51477D
    style H fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style I fill:#EAF3EC,stroke:#83A995,color:#315D49
    style J fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
    style K fill:#DCEFE5,stroke:#4F8B70,color:#24563F

Whether to perform Query rewriting should be determined based on the query type.

Simple questions do not necessarily require additional rewriting; complex questions, colloquial questions, and questions with multiple constraints are usually more worth analyzing at the query level.

11. Stage 6: Optimizing Context Construction

Correct retrieval results do not mean that the context sent to the LLM is necessarily correct.

If truncation, duplication, ordering errors, or content loss occur during context construction, the final answer may still be incomplete.

11.1 Why is context quality important?

Suppose 5 Chunks are retrieved:

Chunk Content Relevance
A Definition of KV Cache High
B Computation mechanism of KV Cache High
C GPU memory overhead of KV Cache High
D History of Transformer development Low
E Basic introduction to GPU Low