Back to series
· 16 min read

AI Knowledge Base and Long-Term Memory System


1. Why Do AI Applications Need Knowledge Bases and Long-Term Memory?

Large language models themselves do not possess reliable, unlimited, and automatically persistent memory capabilities.

Each time a model receives a request, it reasons primarily based on the current input context. Even if the model can handle a long context window, that does not mean it can automatically remember all previous conversations, continuously update business knowledge, or accurately manage a user's long-term preferences.

In real-world applications, there are usually three types of information:

  1. Model built-in knowledge: general knowledge learned during the model's training phase.
  2. External knowledge: enterprise documents, databases, product materials, business rules, and real-time information.
  3. Long-term memory: user preferences, historical interactions, ongoing task states, and confirmed personalized information.

The lifecycle, update methods, and access policies of these three types of information are completely different.

1.1 Model Context Is Not the Same as Long-Term Memory

The context window is the range of information a model can receive and process in a single inference.

For example:

User: I am a Java backend engineer.
Assistant: Understood.

User: I mainly use Spring Boot.
Assistant: Understood.

User: From now on, when explaining technical solutions, prioritize Java examples.

If this information is only kept in the current conversation context, then when the session ends, the context is truncated, or a new session starts, the model may no longer be able to use it.

Long-term memory requires extracting information with lasting value, processing it, storing it in external storage, and retrieving it on demand in subsequent requests.

1.2 The Difference Between Knowledge Bases and Long-Term Memory

Knowledge bases and long-term memory can both use databases, vector databases, and retrieval techniques, but they solve different problems.

Comparison Dimension AI Knowledge Base Long-Term Memory System
Core objective Provide queryable external knowledge Preserve historical information that remains valid
Data source Documents, databases, business systems Conversations, user behavior, task states
Main content Facts, rules, processes, materials Preferences, experiences, goals, relationships, states
Data granularity Documents, paragraphs, entities, records User-level, session-level, task-level memories
Update method Document sync, data updates, re-indexing Extraction, merging, correction, forgetting
Retrieval basis Semantics, keywords, filter conditions User identity, semantics, time, importance
Lifecycle Determined by knowledge validity period and business rules Determined by memory value, timeliness, and user authorization
Permission model Document and business data permissions Ownership of users, sessions, and memories
Typical scenarios Enterprise Q&A, technical document retrieval Personalized assistants, cross-session tasks

The two can share the same underlying infrastructure, but they should not be conflated into a single data management model.

1.3 Why Can't We Rely on Vector Databases Alone?

Vector databases excel at semantic similarity retrieval, but they cannot automatically resolve:

  • Whether the information is true.
  • Whether the information has expired.
  • Whether two memories conflict with each other.
  • Whether a given memory should be overwritten.
  • Whether the user has authorized storage.
  • How long the information should be kept.
  • Whether a memory belongs to the current user.
  • How to clean up all copies after the user requests deletion.

Vector retrieval only solves part of the information lookup problem.

A complete knowledge and memory system also requires data governance, permission control, version management, lifecycle management, and consistency mechanisms.


2. The Overall Architecture of an AI Knowledge Base

An AI knowledge base typically consists of data ingestion, document parsing, knowledge processing, index construction, retrieval services, and generation services.

2.1 Core Architecture of a Knowledge Base

flowchart TD
    A["External Knowledge Sources"] --> B["Data Ingestion Layer"]
    B --> C["Document Parsing and Content Extraction"]
    C --> D["Knowledge Cleaning and Standardization"]
    D --> E["Document Chunking and Metadata Generation"]
    E --> F["Embedding Vectorization"]
    E --> G["Keyword Indexing"]
    F --> H["Vector Database"]
    G --> I["Full-Text Search Engine"]
    H --> J["Knowledge Retrieval Service"]
    I --> J
    J --> K["Reranking and Context Assembly"]
    K --> L["LLM Answer Generation"]
    L --> M["Result Return and Citation"]

    style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style B fill:#EAF3EC,stroke:#83A995,color:#315D49
    style C fill:#EAF3EC,stroke:#83A995,color:#315D49
    style D fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style E fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style F fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style G fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style H fill:#EAE8FA,stroke:#A49BD5,color:#51477D
    style I fill:#EAE8FA,stroke:#A49BD5,color:#51477D
    style J fill:#EAF3EC,stroke:#83A995,color:#315D49
    style K fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style L fill:#EAF3EC,stroke:#83A995,color:#315D49
    style M fill:#DCEFE5,stroke:#4F8B70,color:#24563F

2.2 Data Ingestion Layer

Responsible for acquiring knowledge from different data sources.

Common data sources:

Data Source Data Content
File System PDF, Word, Excel, Markdown
Enterprise Documents Wiki, knowledge management platform
Database MySQL, PostgreSQL
Object Storage Images, documents, attachments
Business API Product information, order rules, service data
Web Pages Technical documentation, public materials
Messaging System Business events, knowledge change notifications

The data ingestion layer needs to handle:

  • Data source connection management.
  • Authentication.
  • Incremental synchronization.
  • Full synchronization.
  • Data change detection.
  • Retry on failure.
  • Data provenance records.
  • Synchronization task status management.

In particular, it is necessary to distinguish between full synchronization and incremental synchronization.

Full synchronization is suitable for initializing the knowledge base or rebuilding the index; incremental synchronization is suitable for continuous updates, avoiding reprocessing all documents each time.

2.3 Document Parsing Layer

Different types of documents require different parsing methods.

For example:

  • PDF: extract text, pages, tables, and layout.
  • Word: extract paragraphs, headings, tables, and hierarchy.
  • Excel: extract worksheets, cells, and table relationships.
  • HTML: extract body text, headings, and links.
  • Markdown: preserve heading hierarchy and code blocks.
  • Images: use OCR or vision models to extract content.
  • Audio: extract text through speech recognition.

Document parsing cannot be simply understood as extracting plain text.

For content such as technical documentation, contracts, and financial reports, table structures, section relationships, footnotes, and context can all affect the final knowledge quality.

2.4 Knowledge Cleaning and Standardization

Raw data usually has problems such as duplication, noise, inconsistent formats, and missing information.

Common processing:

  • Remove invalid characters.
  • Clean up duplicate content.
  • Unify date and time formats.
  • Unify units and field naming.
  • Filter out navigation bars and footers.
  • Identify the document language.
  • Preserve source and update time.
  • Identify sensitive information.
  • Record data permissions.

The cleaning stage should not arbitrarily delete key context from the original documents.

It is recommended to retain the original files, parsing results, and cleaning results so that knowledge errors can be traced.

2.5 Metadata Management

Metadata is used to describe the knowledge content itself and its source.

For example:

{
  "document_id": "doc_10001",
  "title": "模型网关架构设计",
  "source": "engineering_wiki",
  "department": "engineering",
  "category": "architecture",
  "version": "2.1",
  "created_at": "2026-10-10",
  "updated_at": "2026-11-15",
  "access_level": "internal",
  "tenant_id": "tenant_001"
}

Metadata can be used for:

  • Retrieval filtering.
  • Permission checks.
  • Document deduplication.
  • Version management.
  • Time-range queries.
  • Source tracking.
  • Result ranking.

For enterprise knowledge bases, metadata is not only a retrieval optimization field but also an important foundation for access control.


3. Knowledge Chunking and Index Construction

Knowledge chunking is a key part of RAG systems.

Documents are usually long, and it is impossible to put all their content directly into the model context, so documents need to be split into smaller knowledge chunks.

3.1 Why Are Chunks Needed?

Suppose there is a 100-page technical document and the user only asks about the parameter definition of one interface.

If the entire document is passed to the model every time, it will cause:

  • A large amount of irrelevant context.
  • Higher token costs.
  • Longer inference latency.
  • Key content being obscured by noise.
  • Risk of exceeding the context window.

With chunking, only content relevant to the question needs to be retrieved.

3.2 Common Chunking Strategies

Fixed-length chunking

Split text according to a fixed number of tokens.

Advantages:

  • Simple to implement.
  • Controllable chunk length.
  • Suitable for quickly building a basic index.

Disadvantages:

  • May break semantic completeness.
  • May split a complete paragraph.
  • Does not understand document structure.

Recursive character chunking

Split layer by layer using separators such as paragraphs, line breaks, sentences, and characters.

Advantages:

  • Can preserve natural text structure as much as possible.
  • Highly general-purpose.

Disadvantages:

  • Not necessarily suitable for complex document structures.
  • Handling tables and code requires additional strategies.

Semantic chunking

Determine split positions based on semantic shifts in the text.

Advantages:

  • Easier to preserve semantic completeness.
  • Suitable for long texts with clear topic changes.

Disadvantages:

  • Higher computational cost.
  • Split boundaries depend on the model or algorithm.
  • Requires additional validation of chunking quality.

Structured chunking

Split according to document structure such as headings, sections, tables, and code blocks.

Advantages:

  • Preserves document hierarchy.
  • Facilitates citation and traceability.
  • Suitable for technical documents and enterprise knowledge.

Disadvantages:

  • Depends on high-quality document parsing.
  • Requires designing strategies for different document formats.

3.3 Chunk Overlap

Adjacent chunks can retain some overlapping content.

For example:

Original text:
A B C D E F G H I J

Chunk 1:
A B C D E F

Chunk 2:
E F G H I J

The overlapping portion can mitigate context loss caused by chunk boundaries.

However, more overlap is not always better.

An excessively high overlap ratio can lead to:

  • Duplicate retrieval.
  • Increased token costs.
  • Concentration of similar content.
  • Lower effective knowledge density.

The exact chunk length and overlap ratio should be determined experimentally based on document type, embedding model, retrieval performance, and context budget.

3.4 Relationship Between Chunks and Parent Documents

In practice, it is not advisable to store only isolated chunks.

A parent-child structure can be established:

  • Document: original document.
  • Section: document section.
  • Chunk: minimum retrieval unit.

For example:

{
  "chunk_id": "chunk_001",
  "document_id": "doc_10001",
  "section_id": "section_03",
  "content": "模型网关负责统一模型访问与治理。",
  "chunk_index": 12,
  "token_count": 18,
  "metadata": {
    "title": "模型网关架构设计",
    "section": "系统职责"
  }
}

During retrieval, chunks are matched first, and then parent sections or neighboring chunks are fetched as needed; this balances precise recall and context completeness.

3.5 Embedding and Vector Indexing

Embedding models map text into high-dimensional vectors.

Let the text be x, and the embedding function be:

f(x) = \mathbf{v}

where:

\mathbf{v} \in \mathbb{R}^{d}

d represents the vector dimension.

For a user query q and a knowledge snippet c, their semantic similarity can be calculated using vector similarity.

Common methods include:

Cosine similarity

Cosine(A,B)= \frac{A\cdot B}{\|A\|\|B\|}

Dot product

Dot(A,B)=\sum_{i=1}^{d}A_iB_i

Euclidean distance

Distance(A,B)= \sqrt{\sum_{i=1}^{d}(A_i-B_i)^2}

The specific metric used must be consistent with how the embedding model was trained and with the vector index configuration.

Note that high vector similarity does not mean the content is necessarily correct, nor does it mean the two texts are completely equivalent in a business sense.

3.6 Vector Indexes and Full-Text Indexes

A knowledge base usually needs to support both semantic retrieval and keyword retrieval.

Vector index

Suitable for:

  • Synonymous expressions.
  • Natural-language questions.
  • Semantically similar content.
  • Cases where user wording differs from document wording.

Full-text index

Suitable for:

  • Technical terms.
  • Product model numbers.
  • Error codes.
  • Function names.
  • Exact keywords.
  • Specific identifiers.

For example, when a user queries ERR_CONNECTION_RESET, pure semantic retrieval may be less accurate than keyword retrieval.

Therefore, hybrid search is usually better suited to complex knowledge bases than a single retrieval method.


4. Knowledge Retrieval and Generation Mechanisms

The core value of a knowledge base is not only storing knowledge, but also accurately finding relevant information when a user asks a question and providing it to the model.

4.1 Retrieval Execution Flow

flowchart TD
    A["User submits a question"] --> B["Query understanding and rewriting"]
    B --> C["Generate query vector"]
    B --> D["Extract keywords"]
    C --> E["Vector retrieval"]
    D --> F["Full-text retrieval"]
    E --> G["Merge candidate results"]
    F --> G
    G --> H["Metadata and permission filtering"]
    H --> I["Reranking"]
    I --> J["Select Top-K knowledge snippets"]
    J --> K["Context assembly"]
    K --> L["Model generates the answer"]
    L --> M["Return answer and knowledge citations"]

    style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style B fill:#EAF3EC,stroke:#83A995,color:#315D49
    style C fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style D fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style E fill:#EAF3EC,stroke:#83A995,color:#315D49
    style F fill:#EAF3EC,stroke:#83A995,color:#315D49
    style G fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style H fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style I fill:#EAE8FA,stroke:#A49BD5,color:#51477D
    style J fill:#EAF3EC,stroke:#83A995,color:#315D49
    style K fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style L fill:#EAF3EC,stroke:#83A995,color:#315D49
    style M fill:#DCEFE5,stroke:#4F8B70,color:#24563F

4.2 Query Rewriting

User input is not necessarily suitable for direct knowledge retrieval.

For example:

User question:
Does it support streaming?

Without context, the system cannot determine what “it” refers to.

Query rewriting can use the conversation history to convert the question into a more complete retrieval query:

Rewritten:
Does the model gateway support SSE streaming responses for large model APIs?

Common query rewriting methods:

  • Coreference resolution.
  • Question completion.
  • Keyword expansion.
  • Multi-query generation.
  • Query decomposition.
  • Query intent recognition.

However, query rewriting may also change the user's original meaning, so it is necessary to avoid treating information inferred by the model as fact.

4.3 Hybrid Search

Hybrid search combines vector retrieval with keyword retrieval.

For example:

  • Vector search Top 20.
  • Full-text search Top 20.
  • Merge candidate results.
  • Deduplicate.
  • Unified ranking.

A common fusion method is Reciprocal Rank Fusion (RRF).

RRF(d)= \sum_{i=1}^{n} \frac{1}{k+rank_i(d)}

Where:

  • d: the candidate document.
  • rank_i(d): the document's rank in the i-th retrieval result.
  • k: a smoothing parameter.
  • n: the number of retrieval results participating in the fusion.

RRF does not depend on the raw scores of different retrieval algorithms sharing the same scale, which makes it suitable for fusing multiple ranked result lists.

4.4 Reranking

Initial retrieval usually prioritizes recall, so it inevitably introduces some noise.

A reranker uses a more refined relevance model to re-score the candidate documents.

The typical process:

  1. Use vector search and full-text search to recall candidate documents.
  2. Combine the question and the candidate documents into an input.
  3. Use the reranker to compute relevance scores.
  4. Re-sort according to the scores.
  5. Select the most relevant knowledge snippets.

Vector search is better suited to large-scale candidate recall, while reranking is better suited to fine-grained relevance judgments.

The two are usually complementary.

4.5 Context Assembly

The context ultimately fed to the model needs to be controlled in both length and quality.

Considerations include:

  • The number of Top-K items.
  • Document relevance.
  • Token budget.
  • Chunk deduplication.
  • Document source.
  • Timeliness.
  • Permission scope.
  • Content credibility.
  • Conflicts between different pieces of knowledge.

More context is not always better.

Too much irrelevant content may dilute the key evidence, or even affect the model's attention to important information.

4.6 Handling No-Result and Low-Quality Retrieval

A knowledge base must be able to handle no-result situations.

Common strategies:

  • Adjust the query phrasing and retrieve again.
  • Expand the candidate recall scope.
  • Try other retrieval methods.
  • Return a notice that there is insufficient knowledge.
  • Ask the user for additional information.
  • Hand the matter over to a human.

The model should not be forced to generate an answer when there is insufficient evidence.


5. Basic Concepts of Long-Term Memory Systems

The goal of a long-term memory system is to enable an AI application to retain information of lasting value across different sessions and different tasks.

5.1 What Is Long-Term Memory?

Long-term memory is information extracted and persisted from user interactions, task execution, and external events.

For example:

User information:
The user primarily works on Java backend development.

Technical preferences:
Prefer Spring Boot examples.

Answer preferences:
Technical articles should go deep into the principles and avoid being overly introductory.

Ongoing task:
Currently building an AI technology learning column.

This information may be used repeatedly across multiple sessions, so it is worth keeping long term.

But not every conversation should become long-term memory.

For example:

User: Help me calculate 123 × 456.

This is usually just a one-off task, and there is no need to store it permanently.

5.2 Core Characteristics of Long-Term Memory

  • Persistent: does not depend on the context of a single conversation.
  • Retrievable: subsequent requests can find the relevant memory.
  • Updatable: can be corrected when user information changes.
  • Mergeable: consolidates the same information across multiple interactions.
  • Time-aware: distinguishes between information that is valid long term and short term.
  • Traceable: the source of a memory and when it was formed can be identified.
  • Governable: supports permissions, deletion, expiration, and user control.

5.3 Short-Term Memory vs. Long-Term Memory

Dimension Short-term memory Long-term memory
Main scope Current session Across sessions
What is stored Current dialogue, temporary task state Stable preferences, long-term facts, ongoing goals
Storage location Context window, session cache Databases, vector indexes, etc.
Lifecycle During the session or for a limited time Managed according to value and policy
Update method Continuously appended as the dialogue proceeds Extraction, merging, correction
Main purpose Keep the current dialogue coherent Personalization and long-term continuity

Short-term memory is not necessarily kept only in the model context; it can also be persisted in a session database.

Long-term memory does not mean permanent storage either; it means that its lifecycle spans multiple sessions.


6. Taxonomy of Long-Term Memory

Long-term memory can be classified by content, purpose, and lifecycle.

6.1 Factual Memory

Records stable facts about the user or the business.

For example:

  • The user's occupation.
  • The tech stack the user uses.
  • The team the user belongs to.
  • The system's technical architecture.
  • The project's basic configuration.

Factual memory needs to track source, confidence, and validity period.

In particular, information such as user identity, job title, and business status may change over time.

6.2 Preference Memory

Records the user's stable preferences during interactions.

For example:

  • Whether the user prefers concise or detailed answers.
  • Which programming language examples the user prefers.
  • Which output formats the user prefers.
  • Requirements for article depth and structure.
  • Frequently used technical tools.

Preference memory can usually directly influence the style of subsequent responses.

However, a one-off expression request should not be extended indefinitely into a permanent preference.

6.3 Episodic Memory

Records a particular important event or interaction experience.

For example:

  • The user has completed a certain project.
  • A key issue was discovered during a certain task.
  • A certain system version experienced a failure.
  • The user once made a specific decision.

Episodic memory should retain the event time and source, and avoid mistaking historical states for current states.

6.4 Semantic Memory

Records knowledge that has been organized, abstracted, and summarized.

For example:

Original interaction:
The user repeatedly requested that technical articles include principles, flowcharts, and engineering practice.

Summarized memory:
The user prefers technical articles that are well-structured, explain principles in depth, and include engineering practice.

Semantic memory focuses on reusable abstract information rather than storing historical conversations verbatim.

6.5 Procedural Memory

Records task execution methods, operational processes, and stable rules.

For example:

  • How to deploy a certain project.
  • How to execute a data processing workflow.
  • What steps an Agent should follow to complete a task.
  • Operational constraints for a certain business task.

Note that procedural memory cannot replace permission verification and business rule validation during actual execution.

6.6 Working Memory

Working memory stores information that must be continuously used during the execution of the current task.

For example:

  • Current task goal.
  • Completed steps.
  • Steps to be executed.
  • Current execution status.
  • Intermediate calculation results.
  • Temporary variables and context.

Working memory usually has clear task boundaries; after a task ends, it can be archived, cleared, or converted into long-term memory.

6.7 Comparison of Memory Types

Type Main Content Typical Use
Factual Memory Stable facts User information
Preference Memory Interaction preferences Personalized responses
Episodic Memory Historical events Review of past experiences
Semantic Memory Abstract knowledge Cross-task reuse
Procedural Memory Methods and rules Task execution
Working Memory Current task status Multi-step tasks

These categories are not mutually exclusive. The same piece of information can have factual, episodic, and semantic attributes at the same time.

A real system should design memory types based on retrieval needs, rather than over-splitting them just for classification.


VII. Overall Architecture of a Long-Term Memory System

A production-grade long-term memory system usually includes modules such as memory extraction, memory processing, memory storage, memory retrieval, memory injection, and lifecycle management.

7.1 System Architecture

```mermaid flowchart TD A["User requests and interaction records"] --> B["Memory extraction module"] B --> C["Information filtering and classification"] C --> D["Entity recognition and relation extraction"] D --> E["Deduplication and conflict detection"] E --> F["Memory quality assessment"] F --> G["Memory write strategy"] G --> H["Long-term memory storage"] H --> I["Memory indexing and association"] J["New user request"] --> K["Memory retrieval module"] K --> L["Ranking by relevance and recency"] I --> K L --> M["Permission and validity checks"] M --> N["Memory context assembly"] N --> O["LLM reasoning and generation"] O --> P["Return result"] O --> Q["Memory update and feedback"] Q --> B style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F style B fill:#EAF3EC,stroke:#83A995,color:#315D49 style C fill:#F5F8F4,stroke:#B5CBBB,color:#40594A style D fill:#E0F0FE,stroke:#93BCE8,color:#1E4976 style E fill:#EAE8FA,stroke:#A49BD5,color:#51477D style F fill:#F5F8F4,stroke:#B5CBBB,color:#40594A style G fill:#EAF3EC,stroke:#83A995,color:#315D49 style H fill:#DCEFE5,stroke:#4F8B70,color:#24563F style I fill:#EAF3EC,stroke:#83A995,color:#315D49 style J fill:#DCEFE5,stroke:#4F8B70,color:#24563F style K fill:#EAF3EC,stroke:#83A995,color:#315D49 style L fill:#F5F8F4,stroke:#B5CBBB,color:#40594A style M fill:#F7F4E8,stroke:#C7B77A,color:#675A2D style N fill:#F5F8F4,stroke:#B5CBBB,color:#40594A style O fill:#E0F0FE,stroke:#93BCE8,color:#1E4976 style P fill:#DCEFE5,stroke:#4F8B70,color:#24563F style Q fill:#EAE8FA,stroke:#A49BD5,color:#51477D ```

7.2 Memory Extraction Module

Memory extraction is responsible for identifying information of lasting value from raw interactions.

For example:

```text Original conversation: I'm a Java engineer, mainly using Spring Boot at work, and when I write technical articles I'd like more coverage of the underlying principles. ```

Extraction result:

```json { "memories": [ { "type": "fact", "content": "The user is a Java engineer" }, { "type": "preference", "content": "The user mainly uses Spring Boot" }, { "type": "preference", "content": "The user prefers in-depth explanations of underlying technical principles" } ] } ```

The extraction module needs to distinguish between:

  • Information the user has explicitly stated.
  • Information inferred from behavior.
  • Guesses generated by the model.
  • Temporary task information.
  • Information that is not suitable for storage.

In particular, it must avoid treating model-inferred content as facts explicitly confirmed by the user.

7.3 Memory Filtering Module

Not all information is worth saving.

It can be evaluated along the following dimensions:

Importance

Will this information affect future interactions or tasks?

Stability

Is this information likely to remain valid over the long term?

Reuse value

Might it be used in multiple scenarios in the future?

Credibility

Does the information come from an explicit user statement or a reliable data source?

Sensitivity

Does storing this information involve privacy or sensitive data?

Authorization status

Does the user permit the system to store and use this information?

You can build a memory write decision flow:

```text Information extraction ↓ Does it have lasting value? ↓ Is it credible? ↓ Is storage permitted? ↓ Is there any sensitive risk? ↓ Write, pending confirmation, or discard ```

In real systems, the above process should be implemented through an explicit policy engine rather than relying solely on the model's own judgment.

7.4 Memory Update Module

Memory update is more complex than memory writing.

For example:

Historical memory:
The user primarily uses Python.

New information:
The project has recently been fully migrated to Go.

The system cannot simply save both pieces of information as current facts.

It needs to determine:

  • Whether the new information conflicts with the old information.
  • Whether the new information represents a change in state.
  • Whether the old information still has historical value.
  • Whether an effective time should be set.
  • Whether the old memory should be marked as invalid.

A reasonable way to record this:

{
  "content": "The user primarily uses Go to develop the current project",
  "valid_from": "2026-11-20",
  "status": "active"
}

The old record can be retained as historical information instead of directly overwriting all history.


8. Memory Data Model and Storage Design

A long-term memory system cannot store only a piece of text; it also needs a data structure that can support retrieval, updating, deduplication, permissions, and lifecycle management.

8.1 Memory Record Structure

A basic memory record can include:

{
  "memory_id": "mem_10001",
  "user_id": "user_001",
  "type": "preference",
  "content": "The user prefers Java code examples",
  "source": "conversation",
  "source_id": "conversation_100",
  "confidence": 0.98,
  "importance": 0.85,
  "created_at": "2026-11-20T10:00:00Z",
  "updated_at": "2026-11-20T10:00:00Z",
  "valid_from": "2026-11-20T10:00:00Z",
  "valid_until": null,
  "status": "active",
  "version": 1
}

Core field descriptions:

Field Purpose
memory_id Unique memory identifier
user_id User to whom the memory belongs
type Memory type
content Memory content
source Information source
source_id Source record identifier
confidence Confidence of the information
importance Memory importance
created_at Creation time
updated_at Update time
valid_from Effective time
valid_until Expiration time
status Memory status
version Memory version

The actual system can also add:

  • tenant_id.
  • embedding_model.
  • vector_id.
  • sensitivity_level.
  • retention_policy.
  • consent_scope.
  • superseded_by.
  • access_scope.

8.2 Layered Design of Memory Storage

It is recommended to split memory storage into multiple logical layers.

Relational Database

Store structured memories:

  • User ID.
  • Memory type.
  • Memory status.
  • Time information.
  • Permission information.
  • Source information.
  • Version information.

Vector Database

Store the Embeddings required for semantic retrieval:

  • Memory vector.
  • Memory ID.
  • User or tenant identifier.
  • Retrieval filter metadata.

Object Storage

Store larger raw data or archived content:

  • Original documents.
  • Historical event records.
  • Archive files.
  • Large task artifacts.

Cache

Store frequently accessed memories:

  • Core user preferences.
  • Current task context.
  • Frequently accessed memory retrieval results.

8.3 Why Do We Need to Combine Relational Databases and Vector Databases?

Vector databases are good at semantic retrieval, but they are not suitable for independently handling all business state management.

For example, when you need to find:

  • All of a user's valid memories.
  • The count of memories of a certain type.
  • Memories created within a certain time range.
  • Invalidated memories.
  • Memories the user has requested to delete.
  • The historical versions of a given memory.

These operations are better handled by a relational database.

The recommended setup is:

  • A relational database as the authoritative store for memory metadata and lifecycle management.
  • A vector database as the semantic retrieval index.
  • A cache as an acceleration layer for high-frequency reads.

One point must be made explicit: the vector index is derived data and should not be the sole source of truth.

8.4 Consistency Issues in Memory Writes

A relational database and a vector database usually cannot achieve atomic cross-system commit through an ordinary transaction.

As a result, the write process may run into the following:

  • The database write succeeds, but the vector index update fails.
  • The vector index update succeeds, but the database transaction rolls back.
  • The memory update succeeds, but the old vectors are not deleted.
  • A delete operation cleans up only the main table, not the index.

An event-driven, eventual-consistency design is recommended:

  1. First save the memory record in the authoritative database.
  2. Write an Outbox event in the same transaction.
  3. A background consumer processes the vector index.
  4. Retry when indexing fails.
  5. Run periodic index consistency checks.
  6. Clean up the index for invalidated records.

For data that must be strictly deleted, the deletion state must also be propagated to caches, vector indexes, and archival systems, and the cleanup results must be verified.


IX. Long-Term Memory Retrieval Mechanisms

Memory retrieval is not simply a matter of searching for similar text; it requires ranking results by combining user identity, semantic relevance, time, importance, and the current task.

9.1 Basic Semantic Retrieval

After the user makes a request, the system converts the request into a vector and performs a similarity search against the vectors in the memory store.