I. What Is a Vector Database?
A vector database (Vector Database) is a database system specifically designed for storing, indexing, retrieving, and managing high-dimensional vector data.
Unlike traditional relational databases, which mainly find data through primary keys, field values, and conditional expressions, the core task of a vector database is:
Find the data closest to the query vector based on the distance or similarity between vectors.
For example, a knowledge base contains three text passages:
Text A: Redis is an in-memory data structure store.
Text B: Redis supports data types such as String, Hash, and List.
Text C: Transformer uses the Self-Attention mechanism.
Through an Embedding model, text can be converted into vectors:
When a user queries:
What data types does Redis support?
The system converts the query into a vector:
It then calculates the similarity between the query vector and each vector in the database, and returns the most relevant text.
Vector databases are typically responsible for:
- Vector storage.
- Vector index construction.
- Similarity calculation.
- Nearest-neighbor retrieval.
- Metadata filtering.
- Data updates and deletion.
- Ranking of retrieval results.
- Persistence of vector data.
Note that a vector database is generally not responsible for understanding the raw text, nor is it necessarily responsible for generating embeddings. The Embedding model and the vector database are two different components.
II. Where Vector Databases Fit in RAG
In a RAG architecture, the vector database is mainly responsible for knowledge storage and similarity retrieval.
The overall architecture is divided into two stages: offline knowledge ingestion and online querying.
2.1 Offline Knowledge Ingestion
flowchart TD
A["Raw documents"] --> B["Document parsing"]
B --> C["Text chunking"]
C --> D["Generate Embedding"]
D --> E["Build vector records"]
E --> F["Write to vector database"]
F --> G["Build or update vector index"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#E5EAF8,stroke:#9AA8D2,color:#3F4F7A
style D fill:#EAF3EC,stroke:#83A995,color:#315D49
style E fill:#EAF3EC,stroke:#83A995,color:#315D49
style F fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style G fill:#E5F0E9,stroke:#83A995,color:#315D49
2.2 Online Querying
flowchart TD
A["User question"] --> B["Query Embedding"]
B --> C["Generate query vector"]
C --> D["Vector database"]
D --> E["Similarity retrieval"]
E --> F["Metadata filtering"]
F --> G["Return Top-K chunks"]
G --> H["Reranking"]
H --> I["Build context"]
I --> J["Large model generates answer"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#E5EAF8,stroke:#9AA8D2,color:#3F4F7A
style D fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style E fill:#EAF3EC,stroke:#83A995,color:#315D49
style F fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style G fill:#EAF3EC,stroke:#83A995,color:#315D49
style H fill:#E5EAF8,stroke:#9AA8D2,color:#3F4F7A
style I fill:#EAF3EC,stroke:#83A995,color:#315D49
style J fill:#DCEFE5,stroke:#4F8B70,color:#24563F
Here, several concepts need to be distinguished:
| Component | Primary Responsibility |
|---|---|
| Embedding Model | Converts text or other data into vectors |
| Vector Database | Stores, indexes, and retrieves vectors |
| ANN Index | Accelerates nearest-neighbor search in high-dimensional vectors |
| Reranker | Performs fine-grained ranking of initial retrieval results |
| LLM | Generates answers based on retrieved context |
A vector database solves the vector retrieval problem; it is not equivalent to a complete RAG system.
3. Basic Structure of Vector Data
To understand a vector database, you first need to understand the data structure it stores.
3.1 What Is a Vector?
A vector is an ordered array composed of multiple numerical values.
For example:
This is a 4-dimensional vector.
An embedding model typically maps input data into a high-dimensional vector space.
For example:
Where:
- text: input text.
- d: vector dimension.
- \mathbb{R}^{d}: d-dimensional real vector space.
Common embedding dimensions may be 384, 768, 1024, 1536, or other values, depending on the model.
Note the following:
- Vector dimension is not the number of tokens.
- Vector elements are not words or characters.
- Vector coordinates usually do not have directly interpretable independent semantics.
- Vector spaces generated by different embedding models usually cannot be directly mixed or compared.
3.2 Vector Record
A vector database usually stores not only vectors but also the corresponding identifiers, original content, or metadata.
For example:
{
"id": "chunk_001",
"vector": [0.12, 0.35, 0.81, 0.24],
"metadata": {
"document_id": "doc_001",
"title": "Redis 技术文档",
"section": "数据类型",
"page": 12,
"content": "Redis 支持 String、Hash、List 等数据类型。",
"version": "2.0"
}
}
A record usually contains three core parts:
| Field | Purpose |
|---|---|
| ID | Uniquely identifies a vector record |
| Vector | Used for similarity calculation |
| Metadata | Stores business attributes and associated information |
Some vector databases support separate text fields, sparse vectors, multiple vector fields, and multimodal data.
The actual data structure depends on the database's product design.
3.3 Relationship Between Vectors and Original Text
A vector itself usually cannot be directly reconstructed into the original text.
Therefore, the system needs to maintain the association between vectors and the original knowledge.
There are two common designs:
Option 1: The vector database stores the complete text
Vector Record
├── ID
├── Vector
├── Text
└── Metadata
The advantage is that the text can be returned directly after retrieval.
Option 2: The vector database stores only vectors and associated IDs
Vector Record
├── ID
├── Vector
└── Metadata
└── document_id
After a retrieval hit, the original content is then read from the document database or object storage using the ID.
This design can separate vector index storage from original content storage, making it easier to scale them independently.
4. Principles of Vector Similarity Calculation
The core operation of a vector database is calculating the distance or similarity between vectors.
Common methods include:
- Cosine Similarity: cosine similarity.
- Euclidean Distance: Euclidean distance.
- Dot Product: dot product.
- Manhattan Distance: Manhattan distance.
Different distance functions affect vector ranking results, so they need to be consistent with the embedding model's training method and usage conventions.
4.1 Cosine Similarity
Cosine similarity measures the degree of directional similarity through the angle between two vectors.
Formula:
Where:
- A: vector A.
- B: vector B.
- A\cdot B: vector dot product.
- \|A\|: magnitude of vector A.
- \|B\|: magnitude of vector B.
The computed result usually lies in:
For non-zero vectors:
- The closer to 1, the more similar the directions.
- The closer to 0, the closer the directions are to being orthogonal.
- The closer to -1, the more opposite the directions.
Example:
Then:
Therefore:
Cosine similarity focuses mainly on direction, not directly on vector length.
Applicable scenarios:
- Semantic text retrieval.
- Document similarity.
- Semantic search.
- Embedding vector matching.
4.2 Euclidean Distance
Euclidean distance is the straight-line distance between two vectors in space.
Formula:
For example:
Then:
The smaller the Euclidean distance, the closer the two vectors.
Characteristics:
- Considers both vector direction and length.
- Relatively sensitive to vector scale.
- Commonly used for spatial distance calculations.
- Very common in certain clustering and retrieval tasks.
4.3 Dot Product
The dot product is the sum of the products of the corresponding elements of two vectors.
Formula:
For example:
Then:
The dot product is affected by both vector direction and length.
When the vectors have already been L2-normalized:
In this case:
Therefore, on unit-normalized vectors, the dot product can be used to compute cosine similarity, and it is usually easier to implement efficiently.
4.4 Manhattan Distance
Manhattan distance computes the sum of the absolute differences across all dimensions of two vectors.
Formula:
For example:
Then:
It is suitable for certain tasks that are sensitive to differences along coordinate axes, but it is not the most common default choice in typical text Embedding retrieval.
4.5 Differences Between Distance Metrics
| Metric | Focus | Optimization Direction |
|---|---|---|
| Cosine | Vector direction | Larger means more similar |
| Euclidean | Straight-line distance in space | Smaller means more similar |
| Dot Product | Direction and vector length | Larger usually means more similar |
| Manhattan | Absolute differences across dimensions | Smaller means more similar |
An easily overlooked issue:
Different distance functions cannot be substituted arbitrarily.
Even if two Embedding models have the same dimensionality, that does not mean they are suited to the same distance metric.
You need to refer to the model's training objective, the vector normalization method, and the database's specific definition of distance.
V. Why Are Vector Indexes Needed?
If a vector database only stores vectors without an index, a query may require scanning all vectors.
Suppose the database has 10 million vectors, each with 768 dimensions.
Brute-force search requires computing distances one by one against a large number of vectors.
5.1 Brute-force Search
Brute-force search is also called Exact Nearest Neighbor Search.
Suppose the database contains:
The query vector is q.
The system computes:
Then it sorts by distance or similarity and takes the Top-K.
Basic flow:
flowchart TD
A["Input query vector q"] --> B["Read all vectors"]
B --> C["Compute distances one by one"]
C --> D["Obtain distance set"]
D --> E["Sort"]
E --> F["Select Top-K"]
F --> G["Return search results"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#E5EAF8,stroke:#9AA8D2,color:#3F4F7A
style D fill:#EAF3EC,stroke:#83A995,color:#315D49
style E fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style F fill:#EAF3EC,stroke:#83A995,color:#315D49
style G fill:#DCEFE5,stroke:#4F8B70,color:#24563F
For N d-dimensional vectors:
- The time complexity of computing distances is usually O(Nd).
- Full sorting usually requires O(N\log N).
- If only maintaining Top-K, you can use a heap of size K, which usually requires O(N\log K).
When the data scale is small, brute-force search may perform well and can produce exact results.
However, as the data scale grows, the cost of computing one by one increases significantly.
5.2 Exact Search and Approximate Search
The core goal of a vector index is to reduce the number of vectors that need to be accessed during search.
There are two important concepts here:
Exact Nearest Neighbor
Return the Top-K vectors in the database with the truly closest distances.
Approximate Nearest Neighbor
Reduce the search scope through an index structure and return results that are highly likely to be close to the true nearest neighbors.
Approximate search does not guarantee finding all the truly nearest vectors in the mathematical sense every time, but it can usually achieve a good balance between retrieval quality and performance.
Common index algorithms:
- HNSW.
- IVF.
- PQ.
- IVF-PQ.
- DiskANN.
- ScaNN.
Different algorithms use different data organization methods and search mechanisms.
6. HNSW Index Principles
HNSW (Hierarchical Navigable Small World) is a very common approximate nearest neighbor index algorithm in vector databases.
Its core idea is:
Build a multi-layer neighbor graph, use the upper layers to quickly locate the target region, and then descend layer by layer to the bottom layer for fine-grained search.
6.1 What Is a Neighbor Graph?
A neighbor graph is a graph structure.
Each vector corresponds to a node, and nodes are connected by edges.
Edges usually indicate that there is some proximity relationship between two vectors.
For example:
A
/ \
B---C
/ \ \
D---E---F
When querying a new vector, you can start from an entry node and keep visiting neighbors that are closer to the query vector.
6.2 Why Are Layers Needed?
If there is only one layer of the graph, a search may need to pass through many nodes to find the target region.
HNSW uses a multi-layer structure:
- Upper layers: fewer nodes, responsible for fast jumps.
- Middle layers: responsible for narrowing the search scope.
- Bottom layer: contains all data nodes and performs finer-grained search.
This can be understood as:
Layer 2: A -------- G
Layer 1: A --- C --- G --- J
Layer 0: A-B-C-D-E-F-G-H-I-J-K-L
The upper layers have fewer nodes and can help the search quickly cover larger regions.
6.3 HNSW Search Process
flowchart TD
A["Input query vector"] --> B["Start from the top-layer entry node"]
B --> C["Compute distances from the current node and its neighbors"]
C --> D{"Is there a closer neighbor?"}
D -->|"Yes"| E["Move to the closer neighbor"]
E --> C
D -->|"No"| F{"Is there another layer?"}
F -->|"Yes"| G["Enter the next layer"]
G --> C
F -->|"No"| H["Bottom-layer candidate set search"]
H --> I["Maintain the candidate node set"]
I --> J["Return Top-K"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#EAF3EC,stroke:#83A995,color:#315D49
style D fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style E fill:#E5EAF8,stroke:#9AA8D2,color:#3F4F7A
style F fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style G fill:#EAF3EC,stroke:#83A995,color:#315D49
style H fill:#E5EAF8,stroke:#9AA8D2,color:#3F4F7A
style I fill:#EAF3EC,stroke:#83A995,color:#315D49
style J fill:#DCEFE5,stroke:#4F8B70,color:#24563F
The actual HNSW search is more complex than the simplified flow shown in the figure.
It usually maintains:
- The current search node.
- The set of visited nodes.
- The candidate node queue.
- The current best result set.
- Search expansion parameters.
At upper layers, greedy search is usually used to keep finding closer nodes until no further improvement is possible, then it moves to the next layer.
At the bottom layer, the algorithm uses a larger candidate search range to achieve better recall.
6.4 HNSW Construction Process
HNSW is not just a search algorithm; it also includes an index construction process.
Each time a new vector is inserted, it usually requires:
- Randomly assign the highest layer to the new node.
- Start from the entry node and search for neighboring nodes.
- Find candidate neighbors at each layer.
- Select suitable neighbors to establish connections.
- Prune some edges according to connection limits.
- Add the new node to the graph structure.
- Update the entry node if necessary.
Layers are usually generated using a probability distribution, so that nodes at higher layers are sparser.
6.5 Key Parameters of HNSW
| Parameter | Meaning | Impact |
|---|---|---|
| M | Parameter related to the number of neighbor connections allowed per node | Affects graph density and memory |
| ef_construction | Search candidate size when building the index | Affects construction quality and speed |
| ef_search | Search candidate size during querying | Affects recall and latency |
| max_elements | Maximum number of elements supported by the index | Affects capacity planning |
Specifically:
The larger M is:
- Graph connections are usually richer.
- Search may find effective paths more easily.
- Memory usage increases.
- Index construction cost may increase.
The larger ef_construction is:
- More candidate neighbors are considered during construction.
- This is usually beneficial to index quality.
- Construction speed decreases.
- Construction resource consumption increases.
The larger ef_search is:
- More candidate nodes are explored during queries.
- This usually helps improve Recall@K.
- Query latency and computational overhead increase.
HNSW parameters need to be tested according to data distribution, dimensionality, data scale, and recall targets; you cannot rely only on default values.
6.6 Advantages and Disadvantages of HNSW
Advantages:
- Fast retrieval speed.
- Recall is usually high.
- Supports dynamic insertion.
- Suitable for scenarios with sufficient memory.
- Performs well on many general vector retrieval tasks.
Disadvantages:
- The graph structure requires additional memory.
- Index construction incurs computational overhead.
- Large-scale data may require a high memory budget.
- Deletion operations and graph maintenance are relatively complex.
- Parameter tuning requires experimentation.
7. Principles of IVF Inverted Index
IVF (Inverted File Index) is a vector indexing method based on clustering.
Its core idea is:
First divide the vector space into multiple regions, and at query time search only the most relevant subset of regions.
7.1 Clustering and Partitioning
Suppose there are a large number of vectors:
Use a clustering algorithm to divide them into K clusters:
Each cluster has a center vector:
These centroids can usually be trained with K-Means.
Illustration:
Vector space
● ● ●
● C1 ●
● ●
● C2 ●
● ●
● C3 ●
● ● ●
● C4 ●
7.2 Building an IVF Index
When building an IVF index:
- Train the cluster centroids.
- Compute the distance between each vector and the centroids.
- Assign each vector to its nearest cluster.
- Maintain a vector list for each cluster.
For example:
Cluster 1:
Vector A
Vector B
Vector C
Cluster 2:
Vector D
Vector E
Cluster 3:
Vector F
Vector G
7.3 IVF Query Flow
At query time, there is no need to search all clusters.
First compute the distance between the query vector and the centroids, then select the nearest few clusters to search.
This number is usually controlled by nprobe.
flowchart TD
A["Input query vector"] --> B["Compute distance to cluster centroids"]
B --> C["Select the nearest nprobe clusters"]
C --> D["Read the vectors in the selected clusters"]
D --> E["Compute exact distances"]
E --> F["Merge candidate results"]
F --> G["Return Top-K"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style D fill:#E5EAF8,stroke:#9AA8D2,color:#3F4F7A
style E fill:#EAF3EC,stroke:#83A995,color:#315D49
style F fill:#EAF3EC,stroke:#83A995,color:#315D49
style G fill:#DCEFE5,stroke:#4F8B70,color:#24563F
7.4 nlist and nprobe
IVF has two important parameters:
| Parameter | Meaning |
|---|---|
| nlist | Number of cluster centroids, i.e. the number of partitions |
| nprobe | Number of clusters actually probed during a query |
Assume:
Then a query probes only 16 clusters instead of searching all 1024 clusters.
Generally:
- The larger nlist is, the finer the partitions.
- The larger nprobe is, the wider the search scope.
- The larger nprobe is, the higher the recall usually is.
- The larger nprobe is, the higher the query overhead usually is.
However, an excessively large nlist may make cluster training and index maintenance complicated.
7.5 Pros and Cons of IVF
Advantages:
- It can reduce the search scope.
- It suits large-scale vector data.
- It can be combined with quantization and compression.
- The search scope is easy to tune via nprobe.
Disadvantages:
- It depends on clustering quality.
- Cluster boundaries may cause misses.
- When the distribution of new data changes, retraining may be required.
- nlist and nprobe must be tuned together with the data scale.
8. Principles of PQ Vector Quantization
PQ (Product Quantization) is mainly used to reduce vector storage cost and distance computation overhead.
It is not a simple vector index but a vector compression technique, which can usually be combined with indexes such as IVF.
8.1 Why Is Quantization Needed?
Suppose a vector has 1536 dimensions.
If each dimension uses FP32:
That is, each vector takes about 6 KB.
If there are 10 million vectors:
The raw vector data alone requires about 61.44 GB, without yet counting the index, metadata and other system overhead.
When the data scale is very large, vector memory becomes a significant cost.
8.2 The Core Idea of PQ
PQ splits a high-dimensional vector into multiple low-dimensional sub-vectors.
For example, an 8-dimensional vector:
can be split into 4 two-dimensional sub-vectors:
Then clustering is performed separately in each subspace to build the corresponding codebook.
Each sub-vector no longer stores full floating-point numbers; instead, it stores the ID of the centroid it belongs to.
8.3 PQ Encoding Process
flowchart TD
A["Input high-dimensional vector"] --> B["Split into multiple subvectors"]
B --> C["Cluster each subspace independently"]
C --> D["Generate multiple Codebooks"]
D --> E["Find the nearest center for each subvector"]
E --> F["Store the center ID"]
F --> G["Obtain the compressed vector encoding"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#E5EAF8,stroke:#9AA8D2,color:#3F4F7A
style D fill:#EAF3EC,stroke:#83A995,color:#315D49
style E fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style F fill:#EAF3EC,stroke:#83A995,color:#315D49
style G fill:#DCEFE5,stroke:#4F8B70,color:#24563F
8.4 PQ Compression Effect
Assume:
- The original dimension is d=1536.
- It is split into m=96 subvectors.
- Each subspace uses 256 cluster centers.
Each subvector only needs to store one 8-bit code:
Therefore, the encoded size of each vector is:
The original FP32 vector is approximately:
In theory, the vector encoding part is compressed to:
Actual systems also need to store the Codebook, IDs, index structures, and other metadata, so the overall storage compression ratio will not be exactly equal to this value.
8.5 How Does PQ Compute Distance?
It is not always necessary to fully decode the query vector and every original vector before computing distances.
You can precompute the distances between each subspace of the query vector and the cluster centers in the Codebook to build a distance lookup table.
Then, approximate distances are accumulated through table lookups.
This can reduce computation and memory access overhead.
However, because the original vectors are quantized and compressed, distance calculations are approximate and may cause recall loss.
8.6 IVF-PQ
IVF and PQ can be used together:
- IVF narrows the search region.
- PQ compresses vectors.
- During a query, compressed vectors are searched only in some partitions.
- Approximate distances are calculated for candidate results.
- When necessary, the original vectors are used for exact reranking.
This combination is suitable for scenarios with high requirements for storage cost and large-scale retrieval performance.
10. Other Common Vector Indexing Technologies
9.1 DiskANN
DiskANN is a class of graph indexing technologies for large-scale vector retrieval.
Its important characteristics are:
- It uses external storage such as SSDs to store large amounts of index data.
- It keeps some key structures in memory.
- It accesses relevant nodes through graph search.
- It reduces the cost of relying entirely on memory.
It is suitable for scenarios where memory cannot hold all vector data but relatively high retrieval performance is still required.
Note that DiskANN is an umbrella term for a series of related algorithms and implementations; different implementations may differ in index structure, caching, and disk access strategies.
9.2 ScaNN
ScaNN is a vector similarity search system proposed by Google.
It uses techniques such as partitioning, candidate filtering, and quantization to accelerate retrieval.
Its core goal is to achieve efficient approximate nearest neighbor search in large-scale vector data.
9.3 Comparison of Common Indexes
| Index Technology | Core Idea | Main Advantages | Main Limitations |
|---|---|---|---|
| Flat | Brute-force exact search | Exact results and simple implementation | High query cost at large scale |
| HNSW | Multi-layer nearest-neighbor graph | Low latency and relatively high recall | Large memory footprint |
| IVF | Clustering-based partitioning | Reduces the search scope | Depends on clustering quality |
| PQ | Subspace quantization | Compressed storage | Approximate distances introduce error |
| IVF-PQ | Combination of partitioning and quantization | Suitable for large-scale data | Complex trade-off between parameters and accuracy |
| DiskANN | Disk-assisted graph search | Reduces memory pressure | Disk access introduces extra latency |
| ScaNN | Partitioning and quantization optimization | Efficient approximate search | Depends on the specific implementation and data distribution |
No single index is best for every use case.
Actual selection needs to consider all of the following:
- Data scale.
- Vector dimensions.
- Memory capacity.
- Disk performance.
- Query QPS.
- Recall@K.
- Query latency.
- Data update frequency.
10. Complete Internal Architecture of a Vector Database
A vector database is not just an ANN index.
A production-grade vector database usually also includes modules such as data ingestion, storage engine, index management, query execution, metadata filtering, and transaction or consistency management.
10.1 Logical Architecture
flowchart TD
A["Client / Application"] --> B["API Access Layer"]
B --> C["Query and Write Coordinator"]
C --> D["Schema and Collection Management"]
C --> E["Query Execution Engine"]
C --> F["Write Processing Engine"]
E --> G["Vector Index Module"]
E --> H["Metadata Filtering Module"]
E --> I["Distance Calculation Module"]
F --> J["Data Buffering and Persistence"]
F --> G
J --> K["Disk Storage"]
G --> L["Index Storage"]
H --> M["Metadata Index"]
K --> N["Recovery and Consistency Management"]
L --> N
M --> N
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#EAF3EC,stroke:#83A995,color:#315D49
style D fill:#E5EAF8,stroke:#9AA8D2,color:#3F4F7A
style E fill:#E5EAF8,stroke:#9AA8D2,color:#3F4F7A
style F fill:#E5EAF8,stroke:#9AA8D2,color:#3F4F7A
style G fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style H fill:#EAF3EC,stroke:#83A995,color:#315D49
style I fill:#EAF3EC,stroke:#83A995,color:#315D49
style J fill:#EAF3EC,stroke:#83A995,color:#315D49
style K fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style L fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style M fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style N fill:#E5F0E9,stroke:#83A995,color:#315D49
The actual internal architecture of different products is not exactly the same.
For example, some systems use a memory-first architecture, some use a tiered disk-and-memory architecture, and some integrate vector indexes into existing database engines.
The diagram above describes logical modules and does not mean that all vector databases use the same physical implementation.
10.2 Collection
A Collection can be understood as a logical collection of vector data, similar to a table in a relational database.
A Collection usually has:
- Collection Name.
- Vector Dimension.
- Distance Metric.
- ID.
- Vector Field.
- Metadata Schema.
- Index Configuration.
For example:
{
"collection": "knowledge_base",
"dimension": 768,
"metric_type": "COSINE",
"fields": {
"id": "string",
"vector": "float_vector",
"content": "string",
"document_id": "string",
"category": "string"
}
}
Vectors in the same Collection usually need to satisfy the same vector dimension requirement.
Some databases support multiple vector fields, but each vector field still needs to satisfy its own dimension and configuration constraints.
10.3 Schema Management
Schema is used to define the field structure and data types of vector records.
Common field types:
- String.
- Integer.
- Float.
- Boolean.
- Array.
- JSON.
- Vector.
- Sparse Vector.
Schema serves the following purposes:
- Constrain the data structure.
- Validate written content.
- Support metadata filtering.
- Manage vector dimensions.
- Provide field information for index construction.
10.4 Storage Engine
A vector database needs to store:
- Raw vectors.
- Vector IDs.
- Metadata.
- Index structures.
- Data versions.
- Persistent logs or snapshots.
Different systems may use:
- In-memory storage.
- Local SSD.
- Distributed file systems.
- Object storage.
- WAL (Write-Ahead Log).
- Segment files.
In many production systems, memory and disk use a tiered design.
For example:
- Hot indexes are kept in memory.
- Large-scale raw vectors are stored on disk.
- Cold data is stored in object storage.
- Metadata uses a separate index.
The specific implementation depends on the product architecture.
11. Vector Write and Index Construction Mechanisms
A vector write does not necessarily mean the data immediately enters the final ANN index.
In production systems, writes and index construction may be handled asynchronously.
11.1 Vector Write Process
flowchart TD
A["Receive write request"] --> B["Validate ID and vector dimensions"]
B --> C{"Is the data valid?"}
C -->|"No"| D["Return parameter error"]
C -->|"Yes"| E["Write to buffer or log"]
E --> F["Persist data"]
F --> G["Build or update index"]
G --> H["Update searchable status"]
H --> I["Write complete"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style D fill:#F7EAE7,stroke:#C98D7C,color:#754438
style E fill:#EAF3EC,stroke:#83A995,color:#315D49
style F fill:#EAF3EC,stroke:#83A995,color:#315D49
style G fill:#E5EAF8,stroke:#9AA8D2,color:#3F4F7A
style H fill:#EAF3EC,stroke:#83A995,color:#315D49
style I fill:#DCEFE5,stroke:#4F8B70,color:#24563F
Here, three points in time need to be distinguished:
- The request is received by the system.
- The data has been persisted.
- The data has become queryable.
They do not necessarily occur at the same time.
For example, some systems first write data to a WAL or buffer, and then build the index asynchronously.
Therefore, applications need to understand the database's write acknowledgment, visibility, and consistency semantics.
11.2 Segments and Index Construction
To improve write throughput, some vector databases use a segment mechanism.
A segment is a group of data records organized together logically.
Common processing approach:
- New data first enters a writable buffer.
- Once certain conditions are met, a data segment is formed.
- A vector index is built for the data segment.
- After the index is built, it participates in queries.
- Multiple data segments can be merged through Compaction.
flowchart TD
A["New vector write"] --> B["WAL / buffer"]
B --> C["Form Growing Segment"]
C --> D["Data meets build conditions"]
D --> E["Build ANN Index"]
E --> F["Convert to Sealed Segment"]
F --> G["Participate in queries"]
G --> H["Background Compaction"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#E5EAF8,stroke:#9AA8D2,color:#3F4F7A
style D fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style E fill:#EAF3EC,stroke:#83A995,color:#315D49
style F fill:#EAF3EC,stroke:#83A995,color:#315D49
style G fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style H fill:#E5F0E9,stroke:#83A995,color:#315D49
This is a common approach to segmented index management; specific products may define concepts such as Growing, Sealed, and Compaction differently.
12. Vector Query Execution Mechanisms
After a vector database receives a query request, it usually needs to execute multiple stages.
12.1 Query Request
A typical query includes:
{
"collection": "knowledge_base",
"vector": [0.12, 0.35, 0.81, 0.24],
"top_k": 5,
"filter": {
"category": "database",
"version": "2.0"
}
}
Main parameters:
| Parameter | Meaning |
|---|---|
| collection | Target collection to query |
| vector | Query vector |
| top_k | Number of results to return |
| filter | Metadata filter conditions |
| metric | Distance or similarity metric |
| search_params | ANN search parameters |
Request structures are not uniform across different products.
12.2 Query Execution Process
flowchart TD
A["Receive query request"] --> B["Parse query parameters"]
B --> C["Validate vector dimensions"]
C --> D["Process Metadata Filter"]
D --> E["Select relevant indexes and data segments"]
E --> F["Execute ANN search"]
F --> G["Merge candidate results"]
G --> H["Compute or correct distances"]
H --> I["Sort and select Top-K"]
I --> J["Return IDs, vectors, and metadata"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style D fill:#EAF3EC,stroke:#83A995,color:#315D49
style E fill:#E5EAF8,stroke:#9AA8D2,color:#3F4F7A
style F fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style G fill:#EAF3EC,stroke:#83A995,color:#315D49
style H fill:#EAF3EC,stroke:#83A995,color:#315D49
style I fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style J fill:#DCEFE5,stroke:#4F8B70,color:#24563F
The actual query execution plan may vary depending on the index type, filter conditions, and data distribution.
12.3 Top-K Retrieval
Top-K means returning the K records with the highest similarity or the shortest distance.
For example:
This means returning the top five candidate results.
Note the following:
Top-K does not mean that all five results are necessarily relevant.
If the query is outside the coverage of the knowledge base, the database may still return five relatively close records.
Therefore, real systems usually also need to configure:
- Similarity thresholds.
- Minimum relevance requirements.
- Metadata filtering.
- Reranking.
- No-result handling strategies.
12.4 Metadata Filtering
Vector retrieval does not necessarily rely only on vector distance.
Real systems usually also need to filter based on business conditions.
For example:
{
"category": "database",
"department": "engineering",
"version": "2.0",
"status": "published"
}
Search only records that meet the conditions.
Metadata filtering can help enable:
- Search by department.
- Search by knowledge category.
- Search by time range.
- Search by document version.
- Search by user permissions.
12.5 Execution Order of Filter and ANN
There is an important engineering issue between filter conditions and vector indexes:
Filter first or search first?
There are two common strategies.
Pre-filter: filter first, then perform vector retrieval.