1. Why Do AI Applications Need Knowledge Bases and Long-Term Memory?
Large language models themselves do not possess reliable, unlimited, and automatically persistent memory capabilities.
Each time a model receives a request, it reasons primarily based on the current input context. Even if the model can handle a long context window, that does not mean it can automatically remember all previous conversations, continuously update business knowledge, or accurately manage a user's long-term preferences.
In real-world applications, there are usually three types of information:
- Model built-in knowledge: general knowledge learned during the model's training phase.
- External knowledge: enterprise documents, databases, product materials, business rules, and real-time information.
- Long-term memory: user preferences, historical interactions, ongoing task states, and confirmed personalized information.
The lifecycle, update methods, and access policies of these three types of information are completely different.
1.1 Model Context Is Not the Same as Long-Term Memory
The context window is the range of information a model can receive and process in a single inference.
For example:
User: I am a Java backend engineer.
Assistant: Understood.
User: I mainly use Spring Boot.
Assistant: Understood.
User: From now on, when explaining technical solutions, prioritize Java examples.
If this information is only kept in the current conversation context, then when the session ends, the context is truncated, or a new session starts, the model may no longer be able to use it.
Long-term memory requires extracting information with lasting value, processing it, storing it in external storage, and retrieving it on demand in subsequent requests.
1.2 The Difference Between Knowledge Bases and Long-Term Memory
Knowledge bases and long-term memory can both use databases, vector databases, and retrieval techniques, but they solve different problems.
| Comparison Dimension | AI Knowledge Base | Long-Term Memory System |
|---|---|---|
| Core objective | Provide queryable external knowledge | Preserve historical information that remains valid |
| Data source | Documents, databases, business systems | Conversations, user behavior, task states |
| Main content | Facts, rules, processes, materials | Preferences, experiences, goals, relationships, states |
| Data granularity | Documents, paragraphs, entities, records | User-level, session-level, task-level memories |
| Update method | Document sync, data updates, re-indexing | Extraction, merging, correction, forgetting |
| Retrieval basis | Semantics, keywords, filter conditions | User identity, semantics, time, importance |
| Lifecycle | Determined by knowledge validity period and business rules | Determined by memory value, timeliness, and user authorization |
| Permission model | Document and business data permissions | Ownership of users, sessions, and memories |
| Typical scenarios | Enterprise Q&A, technical document retrieval | Personalized assistants, cross-session tasks |
The two can share the same underlying infrastructure, but they should not be conflated into a single data management model.
1.3 Why Can't We Rely on Vector Databases Alone?
Vector databases excel at semantic similarity retrieval, but they cannot automatically resolve:
- Whether the information is true.
- Whether the information has expired.
- Whether two memories conflict with each other.
- Whether a given memory should be overwritten.
- Whether the user has authorized storage.
- How long the information should be kept.
- Whether a memory belongs to the current user.
- How to clean up all copies after the user requests deletion.
Vector retrieval only solves part of the information lookup problem.
A complete knowledge and memory system also requires data governance, permission control, version management, lifecycle management, and consistency mechanisms.
2. The Overall Architecture of an AI Knowledge Base
An AI knowledge base typically consists of data ingestion, document parsing, knowledge processing, index construction, retrieval services, and generation services.
2.1 Core Architecture of a Knowledge Base
flowchart TD
A["External Knowledge Sources"] --> B["Data Ingestion Layer"]
B --> C["Document Parsing and Content Extraction"]
C --> D["Knowledge Cleaning and Standardization"]
D --> E["Document Chunking and Metadata Generation"]
E --> F["Embedding Vectorization"]
E --> G["Keyword Indexing"]
F --> H["Vector Database"]
G --> I["Full-Text Search Engine"]
H --> J["Knowledge Retrieval Service"]
I --> J
J --> K["Reranking and Context Assembly"]
K --> L["LLM Answer Generation"]
L --> M["Result Return and Citation"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#EAF3EC,stroke:#83A995,color:#315D49
style D fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style E fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style F fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style G fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style H fill:#EAE8FA,stroke:#A49BD5,color:#51477D
style I fill:#EAE8FA,stroke:#A49BD5,color:#51477D
style J fill:#EAF3EC,stroke:#83A995,color:#315D49
style K fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style L fill:#EAF3EC,stroke:#83A995,color:#315D49
style M fill:#DCEFE5,stroke:#4F8B70,color:#24563F
2.2 Data Ingestion Layer
Responsible for acquiring knowledge from different data sources.
Common data sources:
| Data Source | Data Content |
|---|---|
| File System | PDF, Word, Excel, Markdown |
| Enterprise Documents | Wiki, knowledge management platform |
| Database | MySQL, PostgreSQL |
| Object Storage | Images, documents, attachments |
| Business API | Product information, order rules, service data |
| Web Pages | Technical documentation, public materials |
| Messaging System | Business events, knowledge change notifications |
The data ingestion layer needs to handle:
- Data source connection management.
- Authentication.
- Incremental synchronization.
- Full synchronization.
- Data change detection.
- Retry on failure.
- Data provenance records.
- Synchronization task status management.
In particular, it is necessary to distinguish between full synchronization and incremental synchronization.
Full synchronization is suitable for initializing the knowledge base or rebuilding the index; incremental synchronization is suitable for continuous updates, avoiding reprocessing all documents each time.
2.3 Document Parsing Layer
Different types of documents require different parsing methods.
For example:
- PDF: extract text, pages, tables, and layout.
- Word: extract paragraphs, headings, tables, and hierarchy.
- Excel: extract worksheets, cells, and table relationships.
- HTML: extract body text, headings, and links.
- Markdown: preserve heading hierarchy and code blocks.
- Images: use OCR or vision models to extract content.
- Audio: extract text through speech recognition.
Document parsing cannot be simply understood as extracting plain text.
For content such as technical documentation, contracts, and financial reports, table structures, section relationships, footnotes, and context can all affect the final knowledge quality.
2.4 Knowledge Cleaning and Standardization
Raw data usually has problems such as duplication, noise, inconsistent formats, and missing information.
Common processing:
- Remove invalid characters.
- Clean up duplicate content.
- Unify date and time formats.
- Unify units and field naming.
- Filter out navigation bars and footers.
- Identify the document language.
- Preserve source and update time.
- Identify sensitive information.
- Record data permissions.
The cleaning stage should not arbitrarily delete key context from the original documents.
It is recommended to retain the original files, parsing results, and cleaning results so that knowledge errors can be traced.
2.5 Metadata Management
Metadata is used to describe the knowledge content itself and its source.
For example:
{
"document_id": "doc_10001",
"title": "模型网关架构设计",
"source": "engineering_wiki",
"department": "engineering",
"category": "architecture",
"version": "2.1",
"created_at": "2026-10-10",
"updated_at": "2026-11-15",
"access_level": "internal",
"tenant_id": "tenant_001"
}
Metadata can be used for:
- Retrieval filtering.
- Permission checks.
- Document deduplication.
- Version management.
- Time-range queries.
- Source tracking.
- Result ranking.
For enterprise knowledge bases, metadata is not only a retrieval optimization field but also an important foundation for access control.
3. Knowledge Chunking and Index Construction
Knowledge chunking is a key part of RAG systems.
Documents are usually long, and it is impossible to put all their content directly into the model context, so documents need to be split into smaller knowledge chunks.
3.1 Why Are Chunks Needed?
Suppose there is a 100-page technical document and the user only asks about the parameter definition of one interface.
If the entire document is passed to the model every time, it will cause:
- A large amount of irrelevant context.
- Higher token costs.
- Longer inference latency.
- Key content being obscured by noise.
- Risk of exceeding the context window.
With chunking, only content relevant to the question needs to be retrieved.
3.2 Common Chunking Strategies
Fixed-length chunking
Split text according to a fixed number of tokens.
Advantages:
- Simple to implement.
- Controllable chunk length.
- Suitable for quickly building a basic index.
Disadvantages:
- May break semantic completeness.
- May split a complete paragraph.
- Does not understand document structure.
Recursive character chunking
Split layer by layer using separators such as paragraphs, line breaks, sentences, and characters.
Advantages:
- Can preserve natural text structure as much as possible.
- Highly general-purpose.
Disadvantages:
- Not necessarily suitable for complex document structures.
- Handling tables and code requires additional strategies.
Semantic chunking
Determine split positions based on semantic shifts in the text.
Advantages:
- Easier to preserve semantic completeness.
- Suitable for long texts with clear topic changes.
Disadvantages:
- Higher computational cost.
- Split boundaries depend on the model or algorithm.
- Requires additional validation of chunking quality.
Structured chunking
Split according to document structure such as headings, sections, tables, and code blocks.
Advantages:
- Preserves document hierarchy.
- Facilitates citation and traceability.
- Suitable for technical documents and enterprise knowledge.
Disadvantages:
- Depends on high-quality document parsing.
- Requires designing strategies for different document formats.
3.3 Chunk Overlap
Adjacent chunks can retain some overlapping content.
For example:
Original text:
A B C D E F G H I J
Chunk 1:
A B C D E F
Chunk 2:
E F G H I J
The overlapping portion can mitigate context loss caused by chunk boundaries.
However, more overlap is not always better.
An excessively high overlap ratio can lead to:
- Duplicate retrieval.
- Increased token costs.
- Concentration of similar content.
- Lower effective knowledge density.
The exact chunk length and overlap ratio should be determined experimentally based on document type, embedding model, retrieval performance, and context budget.
3.4 Relationship Between Chunks and Parent Documents
In practice, it is not advisable to store only isolated chunks.
A parent-child structure can be established:
- Document: original document.
- Section: document section.
- Chunk: minimum retrieval unit.
For example:
{
"chunk_id": "chunk_001",
"document_id": "doc_10001",
"section_id": "section_03",
"content": "模型网关负责统一模型访问与治理。",
"chunk_index": 12,
"token_count": 18,
"metadata": {
"title": "模型网关架构设计",
"section": "系统职责"
}
}
During retrieval, chunks are matched first, and then parent sections or neighboring chunks are fetched as needed; this balances precise recall and context completeness.
3.5 Embedding and Vector Indexing
Embedding models map text into high-dimensional vectors.
Let the text be x, and the embedding function be:
where:
d represents the vector dimension.
For a user query q and a knowledge snippet c, their semantic similarity can be calculated using vector similarity.
Common methods include:
Cosine similarity
Dot product
Euclidean distance
The specific metric used must be consistent with how the embedding model was trained and with the vector index configuration.
Note that high vector similarity does not mean the content is necessarily correct, nor does it mean the two texts are completely equivalent in a business sense.
3.6 Vector Indexes and Full-Text Indexes
A knowledge base usually needs to support both semantic retrieval and keyword retrieval.
Vector index
Suitable for:
- Synonymous expressions.
- Natural-language questions.
- Semantically similar content.
- Cases where user wording differs from document wording.
Full-text index
Suitable for:
- Technical terms.
- Product model numbers.
- Error codes.
- Function names.
- Exact keywords.
- Specific identifiers.
For example, when a user queries ERR_CONNECTION_RESET, pure semantic retrieval may be less accurate than keyword retrieval.
Therefore, hybrid search is usually better suited to complex knowledge bases than a single retrieval method.
4. Knowledge Retrieval and Generation Mechanisms
The core value of a knowledge base is not only storing knowledge, but also accurately finding relevant information when a user asks a question and providing it to the model.
4.1 Retrieval Execution Flow
flowchart TD
A["User submits a question"] --> B["Query understanding and rewriting"]
B --> C["Generate query vector"]
B --> D["Extract keywords"]
C --> E["Vector retrieval"]
D --> F["Full-text retrieval"]
E --> G["Merge candidate results"]
F --> G
G --> H["Metadata and permission filtering"]
H --> I["Reranking"]
I --> J["Select Top-K knowledge snippets"]
J --> K["Context assembly"]
K --> L["Model generates the answer"]
L --> M["Return answer and knowledge citations"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style D fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style E fill:#EAF3EC,stroke:#83A995,color:#315D49
style F fill:#EAF3EC,stroke:#83A995,color:#315D49
style G fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style H fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style I fill:#EAE8FA,stroke:#A49BD5,color:#51477D
style J fill:#EAF3EC,stroke:#83A995,color:#315D49
style K fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style L fill:#EAF3EC,stroke:#83A995,color:#315D49
style M fill:#DCEFE5,stroke:#4F8B70,color:#24563F
4.2 Query Rewriting
User input is not necessarily suitable for direct knowledge retrieval.
For example:
User question:
Does it support streaming?
Without context, the system cannot determine what “it” refers to.
Query rewriting can use the conversation history to convert the question into a more complete retrieval query:
Rewritten:
Does the model gateway support SSE streaming responses for large model APIs?
Common query rewriting methods:
- Coreference resolution.
- Question completion.
- Keyword expansion.
- Multi-query generation.
- Query decomposition.
- Query intent recognition.
However, query rewriting may also change the user's original meaning, so it is necessary to avoid treating information inferred by the model as fact.
4.3 Hybrid Search
Hybrid search combines vector retrieval with keyword retrieval.
For example:
- Vector search Top 20.
- Full-text search Top 20.
- Merge candidate results.
- Deduplicate.
- Unified ranking.
A common fusion method is Reciprocal Rank Fusion (RRF).
Where:
- d: the candidate document.
- rank_i(d): the document's rank in the i-th retrieval result.
- k: a smoothing parameter.
- n: the number of retrieval results participating in the fusion.
RRF does not depend on the raw scores of different retrieval algorithms sharing the same scale, which makes it suitable for fusing multiple ranked result lists.
4.4 Reranking
Initial retrieval usually prioritizes recall, so it inevitably introduces some noise.
A reranker uses a more refined relevance model to re-score the candidate documents.
The typical process:
- Use vector search and full-text search to recall candidate documents.
- Combine the question and the candidate documents into an input.
- Use the reranker to compute relevance scores.
- Re-sort according to the scores.
- Select the most relevant knowledge snippets.
Vector search is better suited to large-scale candidate recall, while reranking is better suited to fine-grained relevance judgments.
The two are usually complementary.
4.5 Context Assembly
The context ultimately fed to the model needs to be controlled in both length and quality.
Considerations include:
- The number of Top-K items.
- Document relevance.
- Token budget.
- Chunk deduplication.
- Document source.
- Timeliness.
- Permission scope.
- Content credibility.
- Conflicts between different pieces of knowledge.
More context is not always better.
Too much irrelevant content may dilute the key evidence, or even affect the model's attention to important information.
4.6 Handling No-Result and Low-Quality Retrieval
A knowledge base must be able to handle no-result situations.
Common strategies:
- Adjust the query phrasing and retrieve again.
- Expand the candidate recall scope.
- Try other retrieval methods.
- Return a notice that there is insufficient knowledge.
- Ask the user for additional information.
- Hand the matter over to a human.
The model should not be forced to generate an answer when there is insufficient evidence.
5. Basic Concepts of Long-Term Memory Systems
The goal of a long-term memory system is to enable an AI application to retain information of lasting value across different sessions and different tasks.
5.1 What Is Long-Term Memory?
Long-term memory is information extracted and persisted from user interactions, task execution, and external events.
For example:
User information:
The user primarily works on Java backend development.
Technical preferences:
Prefer Spring Boot examples.
Answer preferences:
Technical articles should go deep into the principles and avoid being overly introductory.
Ongoing task:
Currently building an AI technology learning column.
This information may be used repeatedly across multiple sessions, so it is worth keeping long term.
But not every conversation should become long-term memory.
For example:
User: Help me calculate 123 × 456.
This is usually just a one-off task, and there is no need to store it permanently.
5.2 Core Characteristics of Long-Term Memory
- Persistent: does not depend on the context of a single conversation.
- Retrievable: subsequent requests can find the relevant memory.
- Updatable: can be corrected when user information changes.
- Mergeable: consolidates the same information across multiple interactions.
- Time-aware: distinguishes between information that is valid long term and short term.
- Traceable: the source of a memory and when it was formed can be identified.
- Governable: supports permissions, deletion, expiration, and user control.
5.3 Short-Term Memory vs. Long-Term Memory
| Dimension | Short-term memory | Long-term memory |
|---|---|---|
| Main scope | Current session | Across sessions |
| What is stored | Current dialogue, temporary task state | Stable preferences, long-term facts, ongoing goals |
| Storage location | Context window, session cache | Databases, vector indexes, etc. |
| Lifecycle | During the session or for a limited time | Managed according to value and policy |
| Update method | Continuously appended as the dialogue proceeds | Extraction, merging, correction |
| Main purpose | Keep the current dialogue coherent | Personalization and long-term continuity |
Short-term memory is not necessarily kept only in the model context; it can also be persisted in a session database.
Long-term memory does not mean permanent storage either; it means that its lifecycle spans multiple sessions.
6. Taxonomy of Long-Term Memory
Long-term memory can be classified by content, purpose, and lifecycle.
6.1 Factual Memory
Records stable facts about the user or the business.
For example:
- The user's occupation.
- The tech stack the user uses.
- The team the user belongs to.
- The system's technical architecture.
- The project's basic configuration.
Factual memory needs to track source, confidence, and validity period.
In particular, information such as user identity, job title, and business status may change over time.
6.2 Preference Memory
Records the user's stable preferences during interactions.
For example:
- Whether the user prefers concise or detailed answers.
- Which programming language examples the user prefers.
- Which output formats the user prefers.
- Requirements for article depth and structure.
- Frequently used technical tools.
Preference memory can usually directly influence the style of subsequent responses.
However, a one-off expression request should not be extended indefinitely into a permanent preference.
6.3 Episodic Memory
Records a particular important event or interaction experience.
For example:
- The user has completed a certain project.
- A key issue was discovered during a certain task.
- A certain system version experienced a failure.
- The user once made a specific decision.
Episodic memory should retain the event time and source, and avoid mistaking historical states for current states.
6.4 Semantic Memory
Records knowledge that has been organized, abstracted, and summarized.
For example:
Original interaction:
The user repeatedly requested that technical articles include principles, flowcharts, and engineering practice.
Summarized memory:
The user prefers technical articles that are well-structured, explain principles in depth, and include engineering practice.
Semantic memory focuses on reusable abstract information rather than storing historical conversations verbatim.
6.5 Procedural Memory
Records task execution methods, operational processes, and stable rules.
For example:
- How to deploy a certain project.
- How to execute a data processing workflow.
- What steps an Agent should follow to complete a task.
- Operational constraints for a certain business task.
Note that procedural memory cannot replace permission verification and business rule validation during actual execution.
6.6 Working Memory
Working memory stores information that must be continuously used during the execution of the current task.
For example:
- Current task goal.
- Completed steps.
- Steps to be executed.
- Current execution status.
- Intermediate calculation results.
- Temporary variables and context.
Working memory usually has clear task boundaries; after a task ends, it can be archived, cleared, or converted into long-term memory.
6.7 Comparison of Memory Types
| Type | Main Content | Typical Use |
|---|---|---|
| Factual Memory | Stable facts | User information |
| Preference Memory | Interaction preferences | Personalized responses |
| Episodic Memory | Historical events | Review of past experiences |
| Semantic Memory | Abstract knowledge | Cross-task reuse |
| Procedural Memory | Methods and rules | Task execution |
| Working Memory | Current task status | Multi-step tasks |
These categories are not mutually exclusive. The same piece of information can have factual, episodic, and semantic attributes at the same time.
A real system should design memory types based on retrieval needs, rather than over-splitting them just for classification.
VII. Overall Architecture of a Long-Term Memory System
A production-grade long-term memory system usually includes modules such as memory extraction, memory processing, memory storage, memory retrieval, memory injection, and lifecycle management.
7.1 System Architecture
```mermaid flowchart TD A["User requests and interaction records"] --> B["Memory extraction module"] B --> C["Information filtering and classification"] C --> D["Entity recognition and relation extraction"] D --> E["Deduplication and conflict detection"] E --> F["Memory quality assessment"] F --> G["Memory write strategy"] G --> H["Long-term memory storage"] H --> I["Memory indexing and association"] J["New user request"] --> K["Memory retrieval module"] K --> L["Ranking by relevance and recency"] I --> K L --> M["Permission and validity checks"] M --> N["Memory context assembly"] N --> O["LLM reasoning and generation"] O --> P["Return result"] O --> Q["Memory update and feedback"] Q --> B style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F style B fill:#EAF3EC,stroke:#83A995,color:#315D49 style C fill:#F5F8F4,stroke:#B5CBBB,color:#40594A style D fill:#E0F0FE,stroke:#93BCE8,color:#1E4976 style E fill:#EAE8FA,stroke:#A49BD5,color:#51477D style F fill:#F5F8F4,stroke:#B5CBBB,color:#40594A style G fill:#EAF3EC,stroke:#83A995,color:#315D49 style H fill:#DCEFE5,stroke:#4F8B70,color:#24563F style I fill:#EAF3EC,stroke:#83A995,color:#315D49 style J fill:#DCEFE5,stroke:#4F8B70,color:#24563F style K fill:#EAF3EC,stroke:#83A995,color:#315D49 style L fill:#F5F8F4,stroke:#B5CBBB,color:#40594A style M fill:#F7F4E8,stroke:#C7B77A,color:#675A2D style N fill:#F5F8F4,stroke:#B5CBBB,color:#40594A style O fill:#E0F0FE,stroke:#93BCE8,color:#1E4976 style P fill:#DCEFE5,stroke:#4F8B70,color:#24563F style Q fill:#EAE8FA,stroke:#A49BD5,color:#51477D ```7.2 Memory Extraction Module
Memory extraction is responsible for identifying information of lasting value from raw interactions.
For example:
```text Original conversation: I'm a Java engineer, mainly using Spring Boot at work, and when I write technical articles I'd like more coverage of the underlying principles. ```Extraction result:
```json { "memories": [ { "type": "fact", "content": "The user is a Java engineer" }, { "type": "preference", "content": "The user mainly uses Spring Boot" }, { "type": "preference", "content": "The user prefers in-depth explanations of underlying technical principles" } ] } ```The extraction module needs to distinguish between:
- Information the user has explicitly stated.
- Information inferred from behavior.
- Guesses generated by the model.
- Temporary task information.
- Information that is not suitable for storage.
In particular, it must avoid treating model-inferred content as facts explicitly confirmed by the user.
7.3 Memory Filtering Module
Not all information is worth saving.
It can be evaluated along the following dimensions:
Importance
Will this information affect future interactions or tasks?
Stability
Is this information likely to remain valid over the long term?
Reuse value
Might it be used in multiple scenarios in the future?
Credibility
Does the information come from an explicit user statement or a reliable data source?
Sensitivity
Does storing this information involve privacy or sensitive data?
Authorization status
Does the user permit the system to store and use this information?
You can build a memory write decision flow:
```text Information extraction ↓ Does it have lasting value? ↓ Is it credible? ↓ Is storage permitted? ↓ Is there any sensitive risk? ↓ Write, pending confirmation, or discard ```In real systems, the above process should be implemented through an explicit policy engine rather than relying solely on the model's own judgment.
7.4 Memory Update Module
Memory update is more complex than memory writing.
For example:
Historical memory:
The user primarily uses Python.
New information:
The project has recently been fully migrated to Go.
The system cannot simply save both pieces of information as current facts.
It needs to determine:
- Whether the new information conflicts with the old information.
- Whether the new information represents a change in state.
- Whether the old information still has historical value.
- Whether an effective time should be set.
- Whether the old memory should be marked as invalid.
A reasonable way to record this:
{
"content": "The user primarily uses Go to develop the current project",
"valid_from": "2026-11-20",
"status": "active"
}
The old record can be retained as historical information instead of directly overwriting all history.
8. Memory Data Model and Storage Design
A long-term memory system cannot store only a piece of text; it also needs a data structure that can support retrieval, updating, deduplication, permissions, and lifecycle management.
8.1 Memory Record Structure
A basic memory record can include:
{
"memory_id": "mem_10001",
"user_id": "user_001",
"type": "preference",
"content": "The user prefers Java code examples",
"source": "conversation",
"source_id": "conversation_100",
"confidence": 0.98,
"importance": 0.85,
"created_at": "2026-11-20T10:00:00Z",
"updated_at": "2026-11-20T10:00:00Z",
"valid_from": "2026-11-20T10:00:00Z",
"valid_until": null,
"status": "active",
"version": 1
}
Core field descriptions:
| Field | Purpose |
|---|---|
| memory_id | Unique memory identifier |
| user_id | User to whom the memory belongs |
| type | Memory type |
| content | Memory content |
| source | Information source |
| source_id | Source record identifier |
| confidence | Confidence of the information |
| importance | Memory importance |
| created_at | Creation time |
| updated_at | Update time |
| valid_from | Effective time |
| valid_until | Expiration time |
| status | Memory status |
| version | Memory version |
The actual system can also add:
- tenant_id.
- embedding_model.
- vector_id.
- sensitivity_level.
- retention_policy.
- consent_scope.
- superseded_by.
- access_scope.
8.2 Layered Design of Memory Storage
It is recommended to split memory storage into multiple logical layers.
Relational Database
Store structured memories:
- User ID.
- Memory type.
- Memory status.
- Time information.
- Permission information.
- Source information.
- Version information.
Vector Database
Store the Embeddings required for semantic retrieval:
- Memory vector.
- Memory ID.
- User or tenant identifier.
- Retrieval filter metadata.
Object Storage
Store larger raw data or archived content:
- Original documents.
- Historical event records.
- Archive files.
- Large task artifacts.
Cache
Store frequently accessed memories:
- Core user preferences.
- Current task context.
- Frequently accessed memory retrieval results.
8.3 Why Do We Need to Combine Relational Databases and Vector Databases?
Vector databases are good at semantic retrieval, but they are not suitable for independently handling all business state management.
For example, when you need to find:
- All of a user's valid memories.
- The count of memories of a certain type.
- Memories created within a certain time range.
- Invalidated memories.
- Memories the user has requested to delete.
- The historical versions of a given memory.
These operations are better handled by a relational database.
The recommended setup is:
- A relational database as the authoritative store for memory metadata and lifecycle management.
- A vector database as the semantic retrieval index.
- A cache as an acceleration layer for high-frequency reads.
One point must be made explicit: the vector index is derived data and should not be the sole source of truth.
8.4 Consistency Issues in Memory Writes
A relational database and a vector database usually cannot achieve atomic cross-system commit through an ordinary transaction.
As a result, the write process may run into the following:
- The database write succeeds, but the vector index update fails.
- The vector index update succeeds, but the database transaction rolls back.
- The memory update succeeds, but the old vectors are not deleted.
- A delete operation cleans up only the main table, not the index.
An event-driven, eventual-consistency design is recommended:
- First save the memory record in the authoritative database.
- Write an Outbox event in the same transaction.
- A background consumer processes the vector index.
- Retry when indexing fails.
- Run periodic index consistency checks.
- Clean up the index for invalidated records.
For data that must be strictly deleted, the deletion state must also be propagated to caches, vector indexes, and archival systems, and the cleanup results must be verified.
IX. Long-Term Memory Retrieval Mechanisms
Memory retrieval is not simply a matter of searching for similar text; it requires ranking results by combining user identity, semantic relevance, time, importance, and the current task.
9.1 Basic Semantic Retrieval
After the user makes a request, the system converts the request into a vector and performs a similarity search against the vectors in the memory store.