I. What Is RAG?
RAG stands for Retrieval-Augmented Generation, known in Chinese as retrieval-augmented generation.
It is a technical architecture that combines information retrieval with the generative capabilities of large language models.
Traditional large language models mainly rely on parametric knowledge learned during training to answer questions, whereas RAG first retrieves relevant content from external knowledge sources before generating an answer, and then passes the retrieval results to the model as context.
Simply put:
- Retrieval: Find information relevant to the question from an external knowledge base.
- Augmented: Add the retrieved information to the model's input context.
- Generation: The large language model generates an answer based on the question and the retrieved content.
1.1 Why Is RAG Needed?
Large language models have several inherent limitations.
First, model knowledge has a temporal boundary.
After a model is trained, its parameters do not automatically absorb new events that occur afterward.
For example, if a company releases a new version of its product today and the model has not received the relevant information, it cannot accurately answer questions about the new version's feature changes based solely on its old training knowledge.
Second, the model does not know private data.
General-purpose large models usually do not have permission to access a company's internal documents, business databases, product manuals, and private knowledge bases.
For example:
- Internal company technical specifications.
- Product design documents.
- Customer service knowledge bases.
- Internal company policies.
- Private project code and technical materials.
This content usually does not appear in model training data.
Third, the model may hallucinate.
When the model lacks reliable information, it may generate fluent but factually incorrect content.
Fourth, the model's context is limited.
Even if a large number of documents are placed directly into the prompt, the system is still limited by the context window, token cost, and processing latency.
The goal of RAG is to enable the model to use information from external knowledge sources without having to retrain it.
1.2 Basic Workflow of RAG
Traditional model Q&A:
flowchart TB
A["User question"] --> B["Build Prompt"]
B --> C["Large language model"]
C --> D["Generate answer"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style D fill:#E5F0E9,stroke:#83A995,color:#315D49
RAG Q&A:
flowchart TB
A["User question"] --> B["Question processing"]
B --> C["Knowledge base retrieval"]
C --> D["Relevant document chunks"]
D --> E["Build augmented context"]
A --> E
E --> F["Large language model"]
F --> G["Generate answer"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style C fill:#EAF3EC,stroke:#83A995,color:#315D49
style D fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style E fill:#EAF3EC,stroke:#83A995,color:#315D49
style F fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style G fill:#E5F0E9,stroke:#83A995,color:#315D49
The core differences between the two approaches are:
| Comparison aspect | Traditional model Q&A | RAG Q&A |
|---|---|---|
| Knowledge source | Knowledge in model parameters | Model parameters and external knowledge |
| Private data | Usually cannot be accessed directly | Can retrieve authorized data |
| Knowledge updates | Usually requires updating the model or external input | Only the knowledge base needs to be updated |
| Real-time information | Depends on external input or tools | Can retrieve from the latest data sources |
| Traceability | Usually lacks external evidence | Can be linked to retrieval sources |
| System complexity | Relatively simple | Requires retrieval, indexing, and knowledge management |
Note: RAG does not make the model relearn knowledge; instead, it provides additional information to the model at inference time.
II. Overall Architecture of RAG
A complete RAG system usually includes two interrelated workflows:
- Offline knowledge processing workflow: transform raw data into a searchable knowledge index.
- Online Q&A workflow: retrieve knowledge based on the user's question and pass the results to the model to generate an answer.
These two workflows have different performance requirements and data lifecycles.
2.1 Overall Architecture Diagram
flowchart TB
subgraph OFF["Offline Knowledge Processing"]
A["Raw Knowledge Source"]
B["Document Parsing"]
C["Text Cleaning"]
D["Document Chunking"]
E["Embedding Vectorization"]
F["Vector Index and Metadata Storage"]
A --> B
B --> C
C --> D
D --> E
E --> F
end
subgraph ON["Online Q&A Processing"]
G["User Question"]
H["Question Vectorization"]
I["Retrieval and Filtering"]
J["Relevant Document Chunks"]
K["Build Prompt"]
L["Large Language Model"]
M["Answer and Citations"]
G --> H
H --> I
I --> J
J --> K
G --> K
K --> L
L --> M
end
F --> I
style A fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style D fill:#EAF3EC,stroke:#83A995,color:#315D49
style E fill:#EAF3EC,stroke:#83A995,color:#315D49
style F fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style G fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style H fill:#EAF3EC,stroke:#83A995,color:#315D49
style I fill:#EAF3EC,stroke:#83A995,color:#315D49
style J fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style K fill:#EAF3EC,stroke:#83A995,color:#315D49
style L fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style M fill:#E5F0E9,stroke:#83A995,color:#315D49
2.2 Offline Knowledge Processing Workflow
The core task of the offline stage is to convert raw data in various formats into structured, searchable knowledge chunks.
It mainly includes:
- Data collection: acquire data from PDF, Word, Markdown, HTML, databases, etc.
- Document parsing: extract body text, headings, tables, and structural information.
- Text cleaning: remove invalid content and normalize text formatting.
- Document chunking: split long documents into smaller chunks.
- Vectorization: convert text into vectors using an embedding model.
- Index storage: store vectors, text, and metadata.
Offline processing can be completed in advance, so the entire knowledge base does not need to be re-parsed every time a user asks a question.
2.3 Online Q&A Workflow
The online stage focuses primarily on response speed and retrieval quality.
Typical workflow:
- Receive the user's question.
- Normalize or rewrite the question.
- Generate the question vector.
- Search the knowledge base for relevant chunks.
- Filter results based on similarity, permissions, and business rules.
- Rank the candidate results.
- Combine the relevant content into context.
- Call the large language model.
- Return the answer and cited sources.
The offline stage determines knowledge base quality, while the online stage determines whether truly useful knowledge can be found for the current question.
3. Document Parsing and Data Preprocessing
The retrieval quality of RAG depends first on the quality of the knowledge sources.
If the original documents are parsed incorrectly, their structure is lost, or their content is disordered, even excellent embedding and retrieval algorithms later cannot compensate.
3.1 Supported Data Sources
Common knowledge sources include:
| Data Type | Typical Format | Processing Focus |
|---|---|---|
| Text documents | TXT, Markdown | Heading hierarchy, paragraph structure |
| Office documents | PDF, DOCX | Page layout, tables, images |
| Web pages | HTML | Body text extraction, navigation cleanup |
| Tables | CSV, XLSX | Row-column relationships, field semantics |
| Databases | MySQL, PostgreSQL | Query results, structured fields |
| Code | Python, Java, Go | File structure, class and function boundaries |
| Multimodal data | Images, audio, video | OCR, ASR, visual understanding |
Different data sources cannot simply use a uniform plain-text extraction method.
For example:
- PDFs may contain two-column layouts and complex tables.
- Word documents may contain headings, images, footnotes, and nested tables.
- Web pages may contain a large amount of navigation, advertisements, and irrelevant content.
- Code files need to preserve classes, functions, and dependencies.
- Tables need to preserve the semantic associations between column names and rows.
3.2 The Importance of Document Parsing
Suppose a technical document contains:
# Redis Cache Design
## 1. Cache Strategies
Redis supports multiple cache eviction strategies.
## 2. Eviction Mechanisms
Common strategies include LRU, LFU, and random eviction.
If the heading structure is lost during parsing, only the following may remain:
Redis Cache Design
Redis supports multiple cache eviction strategies.
Common strategies include LRU, LFU, and random eviction.
Although the text content is still there, the relationships between sections have been weakened.
In actual retrieval, preserving the heading path helps to understand the context of a Chunk.
Metadata can be added to each segment:
{
"document_id": "doc_001",
"title": "Redis 缓存设计",
"section": "淘汰机制",
"content": "常见策略包括 LRU、LFU 和随机淘汰。",
"source": "redis-guide.md",
"page": 12
}
3.3 Data Cleaning
Common cleaning operations:
- Remove duplicate headers and footers.
- Delete navigation bars and irrelevant page elements.
- Normalize spaces and line breaks.
- Handle garbled text and abnormal characters.
- Remove duplicate content.
- Preserve necessary headings, tables, and code structures.
- Mark document content with version and source information.
Note that not all special characters, numbers, symbols, and code should be treated as noise and removed.
For technical documents, function names, parameters, version numbers, and error codes may be exactly the key content for retrieval.
IV. Chunking: Document Segmentation Mechanism
Chunking is a very critical part of RAG.
Most embedding models and vector retrieval systems do not directly perform efficient and precise semantic retrieval on an entire long document; instead, they first split the document into multiple segments and then index each segment.
These segments are usually called Chunks.
4.1 Why Is Segmentation Needed?
Suppose a technical document has 100 pages and covers:
- Redis data structures.
- Cache eviction strategies.
- Persistence mechanisms.
- Primary-replica replication.
- Cluster architecture.
The user only asks:
How does Redis's LFU eviction strategy work?
If the entire document is directly used as a single retrieval unit, it is difficult for the system to accurately locate the relevant section.
After segmentation, we can get:
Chunk 1: Redis Data Structures
Chunk 2: Redis Cache Eviction Strategies
Chunk 3: Redis Persistence Mechanisms
Chunk 4: Redis Primary-Replica Replication
Chunk 5: Redis Cluster Architecture
In this way, finer-grained content can be retrieved for the user's question.
4.2 Fixed-Length Segmentation
The simplest method is to split by a fixed number of Tokens.
For example:
- Chunk Size: 500 Tokens.
- Chunk Overlap: 100 Tokens.
Suppose a text passage has 1200 Tokens:
Chunk 1: Token 1 - 500
Chunk 2: Token 401 - 900
Chunk 3: Token 801 - 1200
Adjacent Chunks have an overlap of 100 Tokens.
This method is simple to implement, but it may cut a complete sentence, paragraph, or even a code function in the middle.
4.3 The Role of Overlap
Overlap refers to a portion of content retained repeatedly between adjacent Chunks.
Its main purpose is to reduce semantic loss caused by segmentation boundaries.
For example:
Chunk A:
Redis uses multiple strategies to manage memory.
Among them, LRU is a common cache eviction strategy.
Chunk B:
Among them, LRU is a common cache eviction strategy.
It selects eviction targets based on the most recent access time.
The second Chunk retains part of the first Chunk's content, which can make the semantics across the segmentation boundary more complete.
But Overlap also has costs:
- Increases the number of vectors.
- Increases storage space.
- Increases the probability of duplicate retrieval.
- May cause context duplication.
Therefore, it cannot be assumed that the larger the Overlap, the better.
4.4 Semantic Segmentation
A further step beyond fixed-length segmentation is to split based on document structure and semantics.
Common strategies:
- Split by heading hierarchy.
- Split by paragraph.
- Split by sentence boundaries.
- Split by code functions or classes.
- Identify topic boundaries based on semantic similarity.
For example, a technical document:
# Transformer
## Self-Attention
Introduces the attention computation mechanism.
## Multi-Head Attention
Introduces the multi-head attention structure.
## Feed Forward Network
Introduces the feed-forward neural network.
Each second-level heading can be used as a main segmentation boundary.
This method can better preserve the document structure.
4.5 Parent-Child Chunk
In complex knowledge bases, a parent-child Chunk structure can also be used.
- Parent Chunk: a larger context unit.
- Child Chunk: a smaller retrieval unit.
During retrieval, Child Chunks are used to improve locating precision; during generation, the full context of the corresponding Parent Chunk is then fetched.
flowchart TB
A["Original document"] --> B["Parent Chunk"]
B --> C["Child Chunk 1"]
B --> D["Child Chunk 2"]
B --> E["Child Chunk 3"]
C --> F["Build child chunk index"]
D --> F
E --> F
F --> G["Retrieval hits Child Chunk"]
G --> H["Link to Parent Chunk"]
H --> I["Add full context"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style D fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style E fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style F fill:#EAF3EC,stroke:#83A995,color:#315D49
style G fill:#EAF3EC,stroke:#83A995,color:#315D49
style H fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style I fill:#DCEFE5,stroke:#4F8B70,color:#24563F
This approach balances retrieval precision and context completeness.
4.6 How Should Chunk Size Be Chosen?
There is no fixed Chunk Size suitable for all scenarios.
| Chunking Strategy | Advantages | Limitations |
|---|---|---|
| Fixed length | Simple and stable | May break semantic boundaries |
| Fixed length + Overlap | Mitigates loss of boundary information | Increases duplicate content |
| Paragraph-based chunking | Preserves natural semantics | Paragraph lengths vary greatly |
| Heading-based chunking | Preserves document structure | Sections may be too long |
| Semantic chunking | Closer to topic boundaries | Computation and implementation are complex |
| Parent-Child | Balances precision and context | Index and relationship management are more complex |
In engineering practice, chunking strategies can be designed separately according to document type, and parameters can then be determined through a retrieval evaluation dataset, rather than directly applying a fixed value.
5. Embedding: Converting Text into Vectors
Embedding is the foundational technology for implementing semantic retrieval in RAG.
It maps text to high-dimensional vectors, so that texts with similar semantics have a certain degree of proximity in vector space.
5.1 What Is Embedding?
Suppose the input text is:
Redis is a high-performance in-memory database.
The Embedding model converts it into a vector:
Where:
- x: input text.
- E: Embedding model.
- d: vector dimension.
- v_i: the i-th value in the vector.
For example, a vector may have 768 or 1,536 dimensions, depending on the model.
These values are not directly interpretable keywords; rather, they are continuous semantic representations learned by the model.
5.2 Why Is Vectorization Needed?
Consider two questions:
How does Redis perform cache eviction?
How does Redis remove data when memory is insufficient?
Their keywords are not exactly the same, but their semantics are highly related.
If only simple string matching is used, this relationship may not be fully recognized.
Embedding can map them into vector space and use vector similarity to find semantically related content.
5.3 Vector Similarity
Common similarity calculation methods include:
- Cosine Similarity: cosine similarity.
- Dot Product: dot product.
- Euclidean Distance: Euclidean distance.
Cosine similarity:
Where:
- A and B: two vectors.
- A\cdot B: dot product of vectors.
- \|A\|, \|B\|: vector magnitudes.
Cosine similarity mainly measures how close two vectors are in direction.
When vectors are normalized:
Therefore, for normalized vectors, the dot product can be used directly to calculate similarity.
Note that whether to use cosine, dot product, or a distance metric must match the training objective of the Embedding model and the index configuration of the vector database.
5.4 Embedding Dimensions and Storage
Assumptions:
- The vector dimension is 1536.
- Each dimension uses Float32.
- The number of vectors is 1 million.
A single vector occupies:
The pure vector data for 1 million vectors is approximately:
Actual storage also needs to account for:
- Vector index structures.
- Metadata.
- Text content.
- Database pages and storage overhead.
- Replicas and redundancy.
Therefore, large-scale knowledge bases require careful evaluation of vector dimensions, index types, and storage costs.
5.5 Document Vectors and Query Vectors
RAG needs to vectorize at least two types of text:
- Offline stage: convert each Chunk in the knowledge base into a vector.
- Online stage: convert the user's question into a query vector.
The two must use compatible Embedding spaces.
If different models, different vector dimensions, or incompatible model versions are used, vector retrieval may fail.
For Embedding models trained with asymmetric retrieval, documents and queries may also need different task instructions or encoding modes; the model's usage guidelines should be followed.
6. Vector Databases and Indexing
After completing Embedding, vectors need to be stored and support efficient similarity retrieval.
This is where vector databases and vector indexes come into play.
6.1 What Is a Vector Database?
A vector database is a type of data system that supports vector storage, similarity search, and vector index management.
Common solutions include:
- Milvus.
- Qdrant.
- Weaviate.
- Chroma.
- PostgreSQL + pgvector.
- Vector retrieval capabilities of Elasticsearch / OpenSearch.
They differ in indexing algorithms, filtering capabilities, distributed architecture, operational complexity, and cost.
6.2 Vector Data Structure
A knowledge chunk usually stores not only the vector but also text and metadata.
For example:
{
"id": "chunk_001",
"vector": [0.12, -0.34, 0.56],
"text": "Redis 支持多种缓存淘汰策略。",
"metadata": {
"document_id": "doc_001",
"title": "Redis 缓存设计",
"section": "淘汰机制",
"source": "redis-guide.md",
"page": 12,
"tenant_id": "tenant_001"
}
}
Actual vectors may have hundreds or even thousands of dimensions; the above only illustrates the data structure.
6.3 Why Are Indexes Needed?
Suppose there are 10 million vectors in the knowledge base.
If each retrieval computes the similarity between the query vector and all 10 million vectors, a large amount of computation is required.
Vector indexes reduce the candidate search range and improve query efficiency.
Common indexes include:
| Index | Core Idea | Characteristics |
|---|---|---|
| Flat | Brute-force exact search | Exact results, high computational cost |
| HNSW | Hierarchical small-world graph | Fast queries, relatively high memory overhead |
| IVF | Clustering-based inverted index | Narrows the search range through clustering |
| IVF-PQ | Clustering and product quantization | Reduces storage cost, may lose accuracy |
6.4 ANN Approximate Nearest Neighbor Search
ANN stands for Approximate Nearest Neighbor, known in Chinese as approximate nearest neighbor search.
Its goal is to strike a balance between search efficiency and retrieval accuracy.
Unlike exact search, ANN does not guarantee finding the mathematically absolute nearest neighbor every time; instead, it quickly finds candidate results that are highly likely to be relevant through the index structure.
For large-scale vector libraries, ANN is a very common retrieval approach.
However, note that:
The nearest neighbors returned by a vector index are not equivalent to the knowledge that can actually answer the user's question.
Vector similarity is only a retrieval signal and cannot be directly equated with answer relevance or factual correctness.
7. Retrieval Strategies: How to Find Truly Relevant Content?
Retrieval is one of the core stages of RAG.
Even if the knowledge base has already been built, if retrieval results are inaccurate, it is difficult for the model to generate reliable answers.
7.1 Top-K Retrieval
The most basic retrieval method is Top-K.
The system computes the similarity between the query vector and knowledge base vectors, then selects the top K results.
For example:
User question:
How does Redis's LFU eviction strategy work?
Retrieval results:
1. Redis LFU eviction mechanism
Similarity: 0.92
2. Redis LRU eviction mechanism
Similarity: 0.86
3. Redis memory management
Similarity: 0.81
4. Redis persistence mechanism
Similarity: 0.54
The system can select Top-3 or Top-5 as candidate context.
However, it cannot simply be assumed that the larger K is, the better.
If K is too small, key evidence may be missed; if K is too large, irrelevant content may be introduced, increasing context cost and interfering with the model's judgment.
7.2 Similarity Threshold
In addition to Top-K, a similarity threshold can also be set.
For example:
Where:
- q: user query.
- d_i: candidate document.
- score: similarity score.
- \tau: threshold.
Only candidate results that exceed the threshold proceed to subsequent processing.
However, similarity scores are usually not calibrated probabilities; score distributions may differ under different embedding models, datasets, and distance metrics.
Therefore, one cannot directly assume that 0.8 means 80% relevance, nor can the same threshold be shared across all models.
7.3 Metadata Filtering
Metadata Filtering uses document metadata for filtering.
For example:
{
"tenant_id": "tenant_001",
"department": "engineering",
"document_type": "technical",
"version": "2.0"
}
When a user queries, additional filter conditions can be added:
tenant_id = tenant_001
AND department = engineering
AND document_type = technical
This can restrict the retrieval scope.
Typical applications:
- Multi-tenant knowledge bases.
- Department permission isolation.
- Specified product versions.
- Specified document types.
- Time range filtering.
- Data classification and access control.
Note that permission filtering cannot rely solely on the model or prompt.
It must be enforced by the backend and retrieval layer, and prevent cross-tenant data leakage caused by missing filter conditions, misconfiguration, or cache reuse.
7.4 Hybrid Search
Pure vector retrieval is good at understanding semantic similarity, but it does not necessarily perform best on exact keywords, product model numbers, code symbols, and error codes.
For example:
ERR_CONNECTION_RESET
Or:
PostgreSQL 16.2
Such queries often require exact matching.
Hybrid retrieval typically combines:
- Dense Retrieval: dense vector retrieval.
- Sparse Retrieval: sparse retrieval, such as BM25.
BM25 is a classic information retrieval ranking algorithm that calculates relevance mainly based on factors such as term matching, term frequency, inverse document frequency, and document length.
Vector retrieval and BM25 each have their advantages:
| Method | Advantages | Limitations |
|---|---|---|
| Vector retrieval | Semantic matching, synonymous expressions | May be insufficiently sensitive to exact keywords |
| BM25 | Keyword, term, and identifier matching | Limited understanding of semantic paraphrasing |
| Hybrid retrieval | Combines both types of retrieval signals | Requires fusion and parameter tuning |
One way to implement hybrid retrieval is to perform the two retrievals separately and then fuse their rankings.
For example, Reciprocal Rank Fusion (RRF):
Where:
- d: candidate document.
- R: ranking lists from different retrieval systems.
- rank_r(d): the document's rank in a given retrieval list.
- k: smoothing constant.
RRF does not directly compare the raw scores of different retrievers; instead, it fuses based on rankings.
7.5 Reranking
After initial retrieval is complete, the candidate results may not yet be the content most suitable for generating the answer.
Reranking uses a more fine-grained relevance model to reorder the candidate documents.
Typical architecture:
flowchart TB
A["User Query"] --> B["Vector Retrieval"]
A --> C["BM25 Retrieval"]
B --> D["Merge Candidate Results"]
C --> D
D --> E["Deduplication and Candidate Filtering"]
E --> F["Reranker Re-ranking"]
F --> G["Select Final Top-K"]
G --> H["Build Generation Context"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#EAF3EC,stroke:#83A995,color:#315D49
style D fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style E fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style F fill:#EAF3EC,stroke:#83A995,color:#315D49
style G fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style H fill:#E5F0E9,stroke:#83A995,color:#315D49
Common Rerankers include:
- Cross-Encoder.
- Dedicated ranking models.
- LLM-based relevance assessment.
A Cross-Encoder usually takes the query and candidate document as input together and directly predicts relevance.
Unlike dual-tower Embedding, it cannot precompute a final relevance score independently for each document in advance like a vector model, so it is generally used on smaller candidate sets.
Reranking can usually improve ranking quality, but it also increases computational overhead and request latency.
7.6 Query Rewrite: Query Rewriting
A user's question may have:
- Vague expression.
- Unclear references.
- Missing context.
- Incomplete keywords.
- Colloquial expressions.
For example:
Original question:
What is its eviction mechanism?
Without prior context, the system can hardly determine what “it” refers to.
It can be completed through query rewriting:
Rewritten question:
What is Redis's cache eviction mechanism?
Common methods:
- Rule-based keyword expansion.
- Coreference resolution based on conversation history.
- LLM Query Rewrite.
- Multi-Query Retrieval.
Multi-Query Retrieval can generate multiple queries phrased differently, retrieve from multiple perspectives, and then fuse the results.
However, query rewriting may also introduce new semantic bias. The original question should be retained, and rewrite results should be used as auxiliary retrieval input rather than unconditionally replacing the user's original intent.
VIII. Generation Stage: Passing Retrieval Results to the LLM
After retrieval is complete, the candidate knowledge snippets need to be organized into a context that the model can understand.
This step is usually called Context Construction or Prompt Assembly.
8.1 Building an Augmented Prompt
Suppose the user asks:
How does Redis's LFU eviction policy work?
Retrieved knowledge snippets:
[Source 1]
Redis supports the LFU eviction policy, determining eviction targets by tracking the access frequency of keys.
[Source 2]
LFU uses an approximate counting mechanism; access frequency information is not a simple counter that grows without bound.
It can be organized as:
System:
You are a professional technical assistant.
Please prioritize answering questions based on the provided reference materials.
If the materials are insufficient, clearly state so and do not fabricate facts.
Context:
[Source 1]
Redis supports the LFU eviction policy, determining eviction targets by tracking the access frequency of keys.
[Source 2]
LFU uses an approximate counting mechanism; access frequency information is not a simple counter that grows without bound.
Question:
How does Redis's LFU eviction policy work?
Here, Context is the external knowledge that RAG adds for the model.
8.2 Why Restrict the Model to Answer Based on Retrieved Content?
If explicit generation constraints are not set, the model may:
- Ignore retrieval results.
- Use training knowledge to supplement unverified information.
- Mix materials from different versions.
- Stitch together irrelevant documents into incorrect conclusions.
- Speculate about content not covered by the materials.
Therefore, you can explicitly require in the Prompt:
- Prioritize the provided reference materials.
- Do not disguise information not present in the materials as facts.
- Point out conflicts when materials conflict.
- Clearly state when no supporting evidence can be found.
- Provide source citations for key conclusions.
However, Prompt constraints can only improve behavior; they cannot guarantee that the model will always be faithful to the evidence. High-reliability scenarios also require independent citation verification, answer validation, and human review mechanisms.
8.3 Citing Sources
To make answers traceable, document identifiers, titles, page numbers, and snippet IDs need to be retained during the retrieval stage.
For example:
{
"answer": "Redis's LFU policy performs approximate eviction based on key access frequency.",
"citations": [
{
"document_id": "doc_001",
"chunk_id": "chunk_008",
"title": "Redis Cache Design",
"page": 12
}
]
}