I. What Is a Multi-Turn Conversation?
A multi-turn conversation (Multi-turn Conversation) refers to a process in which the user and a large language model interact repeatedly in succession around one or more topics.
Unlike single-turn Q&A, the current request in a multi-turn conversation often depends on the content of earlier exchanges.
For example:
User: What is a Transformer?
Model: A Transformer is a neural network architecture based on the attention mechanism.
User: Why is it better suited to parallel computation than an RNN?
Model: Because a Transformer can process multiple positions in a sequence in parallel...
User: Then what are its drawbacks?
Model: The main drawbacks of the Transformer include the high computational cost of attention...
The "it" in the last question can only be resolved to the Transformer by taking the preceding conversation into account.
If the model receives only the last sentence:
Then what are its drawbacks?
it may be unable to determine accurately what the user is referring to.
Therefore, a multi-turn conversation needs to solve two core problems:
- Context understanding: Which pieces of historical information does the current question depend on?
- Context management: How should the necessary historical information be provided to the model?
It should be made clear that:
A large language model does not usually acquire permanent memory automatically from a single conversation. The model is able to make use of prior dialogue mainly because the relevant information is reorganized and passed into the context of the current request.
II. Differences Between Single-Turn and Multi-Turn Conversations
1. Single-Turn Conversation
A single-turn conversation typically contains only one user input and one model response.
flowchart TD
A["User input"] --> B["Model receives Prompt"]
B --> C["Model generates response"]
C --> D["Returned to user"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#EAF3EC,stroke:#83A995,color:#315D49
style D fill:#DCEFE5,stroke:#4F8B70,color:#24563F
In a single-turn conversation, the model generates its result mainly from the current input and the system instructions.
2. Multi-Turn Conversation
A multi-turn conversation needs to carry the historical information relevant to the current task in subsequent requests.
flowchart TD
A["First-round user input"] --> B["Model generates first-round answer"]
B --> C["Save conversation history"]
C --> D["Second-round user input"]
D --> E["Assemble historical context"]
E --> F["Model generates second-round answer"]
F --> G["Update conversation history"]
G --> H["Third-round user input"]
H --> I["Assemble context again"]
I --> J["Model generates third-round answer"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#EAF3EC,stroke:#83A995,color:#315D49
style D fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style E fill:#EAF3EC,stroke:#83A995,color:#315D49
style F fill:#EAF3EC,stroke:#83A995,color:#315D49
style G fill:#EAF3EC,stroke:#83A995,color:#315D49
style H fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style I fill:#EAF3EC,stroke:#83A995,color:#315D49
style J fill:#DCEFE5,stroke:#4F8B70,color:#24563F
A multi-turn conversation does not mean the model is internally running one uninterrupted reasoning process.
Under the common stateless API call pattern, each round is a new model inference call. The application needs to reorganize the necessary historical messages into the current request.
3. Core Differences Between the Two Modes
| Aspect | Single-turn conversation | Multi-turn conversation |
|---|---|---|
| Input content | The current question | The current question plus relevant history |
| Context dependency | Relatively low | Relatively high |
| History management | Usually not required | Required |
| Coreference resolution | Relatively simple | Requires the history |
| Topic continuity | Not emphasized | Must be maintained |
| Token consumption | Relatively fixed | Grows with the history |
| Implementation complexity | Relatively low | Relatively high |
III. How Does the Model Understand a Multi-Turn Conversation?
The key to a multi-turn conversation is not simply storing the historical text, but enabling the model to identify, from the historical information, the content that the current question requires.
1. Conversation History as Context Input
Suppose the user has gone through three rounds of interaction:
First round:
User: I am planning to develop a web project in Python.
Second round:
User: Recommend a web framework.
Model: You could consider FastAPI, Django, and Flask.
Third round:
User: Is it suitable for building high-concurrency APIs?
The "it" in the third round is referentially ambiguous.
If the history does not further specify which framework the user chose, the model should not directly assume that the user has settled on a particular framework.
The application can pass the relevant historical messages to the model together:
[
{
"role": "user",
"content": "I am planning to develop a web project in Python."
},
{
"role": "user",
"content": "Recommend a web framework."
},
{
"role": "assistant",
"content": "You could consider FastAPI, Django, and Flask."
},
{
"role": "user",
"content": "Is it suitable for building high-concurrency APIs?"
}
]
The model understands the current question based on the entire message sequence.
But there is still a problem here: the user has not explicitly specified which framework "it" refers to.
Reasonable model behavior should be to make a judgment based on context and, when necessary, clarify with the user, rather than treating uncertain information as fact.
2. Context Is Not Simple Text Concatenation
The conversation received by the model is usually organized into a message structure with roles and content.
Common roles include:
| Role | Meaning | Purpose |
|---|---|---|
system |
System instructions | Define model behavior and constraints |
developer |
Developer instructions | Define application-layer behavior rules |
user |
User messages | Provide questions and tasks |
assistant |
Model messages | Provide answers or tool calls |
tool |
Tool results | Return external tool execution results |
A simplified multi-turn conversation structure:
[
{
"role": "system",
"content": "You are a professional Python technical assistant."
},
{
"role": "user",
"content": "What is FastAPI?"
},
{
"role": "assistant",
"content": "FastAPI is a modern Python web framework."
},
{
"role": "user",
"content": "What are its advantages?"
}
]
Role information is very important.
For example, even if a user message and a tool result contain the same text, they have different semantic origins and trust boundaries.
All historical content should not be simply merged into a single block of text without role distinctions.
3. How Does the Model Handle Anaphoric References?
Multi-turn conversations often contain the following:
- It.
- This.
- The solution mentioned above.
- The second approach.
- The question just asked.
- According to the previous requirements.
These expressions all require understanding in the context of the conversation history.
For example:
User: Compare PostgreSQL and MySQL.
Model: PostgreSQL has advantages in complex queries and scalability...
User: Then how is its JSON support?
The model needs to determine from the history which database "it" might refer to.
If the previous turn focused on PostgreSQL, the model may tend to treat it as the referent.
But if the context discusses two databases at the same time, ambiguity may arise.
From the perspective of model computation, historical messages are encoded by the tokenizer and then enter the Transformer as part of the current input sequence.
The self-attention mechanism allows the current token to compute attention weights based on other tokens in the context, thereby leveraging historical information.
Its core computation is as follows:
In a causal language model, the current position can attend only to historical positions that are allowed to be accessed and cannot directly access future tokens.
Therefore, the model can use entities, topics, and semantic associations in the context to understand the current expression.
Note: Attention is not an independent database retrieval mechanism. It performs neural network computation within a given context range, and it does not guarantee that the model will accurately find or use all historical information.
IV. Context Window
The context window is one of the most important foundational concepts in multi-turn conversation management.
1. What Is a Context Window?
The Context Window refers to the range of tokens that the model can process in a single inference pass.
It usually includes:
- System Prompt.
- Developer instructions.
- Historical user messages.
- Historical model responses.
- Tool calls and tool results.
- Current user input.
- Reserved space for model output.
For example, a model that supports a 128K token context window means that the input and output of a single request must fit within the context budget defined for that model.
Here, 128K refers to the number of tokens, not the number of characters, and not the number of conversation turns.
2. What Makes Up the Context Window
The token budget of a single request can be abstracted as:
Where:
- T_{\text{system}}: system instructions and developer instructions.
- T_{\text{history}}: historical conversation.
- T_{\text{current}}: current user input.
- T_{\text{output}}: tokens reserved for model output.
- T_{\text{overhead}}: message structure, special tokens, and other protocol overhead.
It must satisfy:
Real APIs may set an input limit, an output limit, or other constraints separately, so you cannot simply assume that all models use a single unified context limit.
3. Why Does the Context Window Fill Up?
Suppose a user keeps having a technical discussion with the model.
Each turn of the conversation may add:
- User input.
- Model responses.
- Tool call arguments.
- Tool execution results.
- Supplementary system context.
If the application retains the full history every turn, the number of input tokens will keep growing.
For example:
| Conversation turn | History tokens | Current input tokens | Reserved output tokens |
|---|---|---|---|
| Turn 1 | 0 | 500 | 2,000 |
| Turn 5 | 4,000 | 500 | 2,000 |
| Turn 10 | 12,000 | 500 | 2,000 |
| Turn 20 | 30,000 | 500 | 2,000 |
| Turn 40 | 65,000 | 500 | 2,000 |
This is only meant to illustrate the growth trend; the actual number of tokens depends on text length, the tokenizer, and the message structure.
4. Problems After the Context Window Is Exhausted
When a request exceeds the context range allowed by the model, the following may occur:
- The API rejects the request.
- The server requires the input to be shortened.
- The application deletes or compresses part of the history.
- The output budget is insufficient.
- Part of the history can no longer be retained.
Even if the hard context limit is not reached, an excessively long context may bring additional computational cost and attention allocation problems.
Therefore, context management is not simply about retaining as much history as possible, but about keeping the information that the current task actually needs.
V. Core Strategies for Context Management
Context management refers to selecting, organizing, compressing, and updating historical messages so that the model receives, within a limited context budget, the information needed to complete the current task.
Common strategies include:
- Full history retention.
- Sliding window.
- History summarization.
- Key information extraction.
- Long-term memory retrieval.
- Hybrid context management.
1. Full History Retention
Pass the entire conversation history to the model in full.
For example:
System Prompt
Turn 1 user message
Turn 1 model response
Turn 2 user message
Turn 2 model response
Turn 3 user message
Turn 3 model response
Current user message
Advantages:
- Preserves complete conversation details.
- Requires no additional summarization processing.
- Is very straightforward for short conversations.
- Information is not easily lost due to proactive deletion of history.
Disadvantages:
- Token consumption keeps increasing.
- Request latency may increase.
- The context window may become full.
- Historical noise may affect the current task.
Use cases:
- Short conversations.
- Tasks that require precise reference to previous text.
- Q&A systems with little historical content.
2. Sliding Window
A sliding window keeps only the most recent several turns of conversation and deletes earlier historical content.
For example, keep only the most recent 5 turns:
flowchart TD
A["Complete conversation history"] --> B["Sort chronologically"]
B --> C["Calculate the most recent window range"]
C --> D["Keep the most recent 5 turns"]
D --> E["Remove old messages outside the window"]
E --> F["Build the current model context"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#EAF3EC,stroke:#83A995,color:#315D49
style D fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style E fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style F fill:#E5F0E9,stroke:#83A995,color:#315D49
For example:
| History turn | Retained? |
|---|---|
| Turn 1 | No |
| Turn 2 | No |
| Turn 3 | No |
| Turn 4 | Yes |
| Turn 5 | Yes |
| Turn 6 | Yes |
| Turn 7 | Yes |
| Turn 8 | Yes |
Advantages:
- Simple to implement.
- Token consumption is easy to control.
- Suitable for continuous conversation scenarios.
- It can reduce the overhead caused by history growth.
Disadvantages:
- Important early information may be deleted.
- The current question may depend on content from a long time ago.
- Simply trimming by number of turns may break context integrity.
For example, in the first turn, the user explicitly specifies:
All subsequent code examples should use TypeScript.
If Turn 1 is deleted and this preference is not saved elsewhere, the model may use JavaScript again in subsequent responses.
Therefore, a sliding window usually needs to be combined with summarization or long-term memory.
3. Conversation Summarization
Conversation summarization compresses a long conversation into a structured summary that retains important information.
For example, the original history contains:
The user is developing a Python web project.
The user chose FastAPI as the backend framework.
The database uses PostgreSQL.
The project is deployed with Docker.
The user requests that code examples use asynchronous style whenever possible.
It can be compressed into:
Project background:
- Backend framework: FastAPI
- Database: PostgreSQL
- Deployment method: Docker
- Coding preference: prefer asynchronous style
Current goal:
Develop a Web API that supports user authentication and order management.
Conversation summarization can significantly reduce Token consumption while retaining the key background required for the task.
Typical flow:
flowchart TD
A["Read conversation history"] --> B["Analyze historical information"]
B --> C["Identify important facts and decisions"]
C --> D["Remove duplicates and low-value content"]
D --> E["Generate structured summary"]
E --> F["Save summary"]
F --> G["Load summary for subsequent requests"]
G --> H["Combine with recent messages"]
H --> I["Build model context"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#EAF3EC,stroke:#83A995,color:#315D49
style D fill:#EAF3EC,stroke:#83A995,color:#315D49
style E fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style F fill:#EAF3EC,stroke:#83A995,color:#315D49
style G fill:#EAF3EC,stroke:#83A995,color:#315D49
style H fill:#EAF3EC,stroke:#83A995,color:#315D49
style I fill:#DCEFE5,stroke:#4F8B70,color:#24563F
The key to a summary is not to make it as short as possible, but to retain the information actually needed for subsequent reasoning.
Pay special attention to retaining:
- Requirements explicitly stated by the user.
- Technical solutions that have already been determined.
- Important business rules.
- Current task status.
- Unresolved issues.
- Key entities and their relationships.
- Constraints that must be observed.
Summaries also carry risks:
- Details are lost.
- Numerical values are incorrectly generalized.
- Chronological order becomes confused.
- Negative conditions are omitted.
- The model mistakenly writes speculation as fact.
Therefore, important information should retain its original source wherever possible, and should not rely solely on model-generated summaries.
4. Key Information Extraction
Not all historical content needs to be saved in conversational form.
Important facts from history can be extracted into structured data.
For example:
{
"project": {
"language": "Python",
"framework": "FastAPI",
"database": "PostgreSQL",
"deployment": "Docker"
},
"preferences": {
"code_style": "异步优先",
"explanation_level": "技术细节充分"
},
"current_task": {
"goal": "实现订单管理 API",
"status": "开发中"
}
}
Compared with raw chat logs, structured information is easier to retrieve, update, and verify.
But structured extraction also needs to consider:
- Information source.
- Update time.
- Credibility.
- Scope of application.
- Whether it has become invalid.
- Whether there are conflicts.
5. Long-term Memory Retrieval
Long-term memory is used to store information outside the current conversation that may still be valuable in the future.
For example:
- The tech stack the user uses long term.
- Architectural constraints of the project.
- Output preferences explicitly stated by the user.
- Background of ongoing tasks.
- Business rules that remain valid long term.
When the user asks a new question, the system can retrieve relevant memories based on the current question, and then put the results into the model context.
Long-term memory usually does not need to be loaded in full.
A more reasonable approach is on-demand retrieval, to avoid irrelevant information consuming the context budget.
6. Short-term Context and Long-term Memory
Multi-turn dialogue systems usually need to distinguish between short-term context and long-term memory.
1. Short-term Context
Short-term context mainly serves the current session.
Typical content:
- The most recent few turns of dialogue.
- Current task requirements.
- Steps not yet completed.
- The result of the most recent tool execution.
- Temporary constraints in the current conversation.
It usually has the following characteristics:
- Strongly related to the current session.
- Updated frequently.
- Relatively short lifecycle.
- More emphasis on conversational continuity.
2. Long-term Memory
Long-term memory is used to store information that remains valuable across sessions.
Typical content:
- Long-term preferences.
- Stable project background.
- Information about ongoing tasks.
- Confirmed important facts.
- Frequently used ways of working.
It usually has the following characteristics:
- Can be used across sessions.
- Lower update frequency.
- Requires persistent storage.
- Needs to handle expiration and conflicts.
- Needs access control.
3. The Difference Between the Two
| Comparison Dimension | Short-term Context | Long-term Memory |
|---|---|---|
| Primary Role | Maintain continuity of the current conversation | Preserve important information across sessions |
| Lifecycle | Current session or task cycle | Can span multiple sessions |
| Content Scope | Recent messages and task status | Stable facts and long-term preferences |
| Update Frequency | High | Relatively low |
| Storage Method | Message history, cache, session state | Databases, vector stores, etc. |
| Loading Method | Organized according to the current conversation | Retrieved as needed |
| Main Risks | Context too long | Expiration, errors, and privacy issues |
4. How Do the Two Work Together?
A common architecture:
flowchart TD
A["User sends a new request"] --> B["Read the current session"]
B --> C["Load recent conversation"]
A --> D["Extract keywords and semantics from the current question"]
D --> E["Retrieve long-term memory"]
E --> F["Filter relevant memories"]
C --> G["Assemble context"]
F --> G
G --> H["Add system instructions"]
H --> I["Send to the large language model"]
I --> J["Generate response"]
J --> K["Update conversation history"]
K --> L["Determine whether long-term memory needs updating"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#EAF3EC,stroke:#83A995,color:#315D49
style D fill:#EAF3EC,stroke:#83A995,color:#315D49
style E fill:#EAF3EC,stroke:#83A995,color:#315D49
style F fill:#EAF3EC,stroke:#83A995,color:#315D49
style G fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style H fill:#EAF3EC,stroke:#83A995,color:#315D49
style I fill:#EAF3EC,stroke:#83A995,color:#315D49
style J fill:#EAF3EC,stroke:#83A995,color:#315D49
style K fill:#EAF3EC,stroke:#83A995,color:#315D49
style L fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
This distinction is needed here:
Historical messages are conversation records, while long-term memory is information that has been filtered and organized.
Historical messages should not all automatically become long-term memory.
For example, if a user briefly asks about the price of a product, it does not mean that the price needs to be stored permanently.
VII. Implementation Approaches for Memory Systems
Long-term memory is not just a piece of text; it also involves data storage, indexing, retrieval, updating, and deletion.
Common implementation approaches include:
1. Relational databases
You can use databases such as PostgreSQL and MySQL to store structured memory.
For example:
CREATE TABLE conversation_memory (
id BIGINT PRIMARY KEY,
user_id VARCHAR(128) NOT NULL,
memory_type VARCHAR(64) NOT NULL,
content TEXT NOT NULL,
source VARCHAR(128),
confidence FLOAT,
created_at TIMESTAMP,
updated_at TIMESTAMP
);
Field descriptions:
| Field | Purpose |
|---|---|
id |
Memory record ID |
user_id |
User to whom the memory belongs |
memory_type |
Memory type |
content |
Memory content |
source |
Information source |
confidence |
Credibility or confidence score |
created_at |
Creation time |
updated_at |
Update time |
Relational databases are suitable for storing:
- User preferences.
- Project configuration.
- Explicit business rules.
- Task status.
- Structured entity information.
Here, confidence is only a score field that the application can maintain; it does not mean that the model can inherently provide reliable factual probabilities.
2. Vector databases
For long-term memory in natural language form, you can use embeddings to convert text into vectors and then retrieve relevant information by similarity.
For example:
Memory A:
The user's backend project mainly uses FastAPI.
Memory B:
The user's database mainly uses PostgreSQL.
Memory C:
The user usually uses Docker to deploy services.
Current question:
What framework did my previous Python project use?
The system can convert the current question into a vector and retrieve semantically related content from the memory store.
Basic process:
flowchart TD
A["Historical information"] --> B["Memory content extraction"]
B --> C["Generate embedding"]
C --> D["Save vector and metadata"]
E["Current user question"] --> F["Generate query vector"]
F --> G["Vector similarity retrieval"]
D --> G
G --> H["Retrieve candidate memories"]
H --> I["Relevance filtering"]
I --> J["Add memories to context"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#EAF3EC,stroke:#83A995,color:#315D49
style D fill:#EAF3EC,stroke:#83A995,color:#315D49
style E fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style F fill:#EAF3EC,stroke:#83A995,color:#315D49
style G fill:#EAF3EC,stroke:#83A995,color:#315D49
style H fill:#EAF3EC,stroke:#83A995,color:#315D49
style I fill:#EAF3EC,stroke:#83A995,color:#315D49
style J fill:#DCEFE5,stroke:#4F8B70,color:#24563F
The core of vector retrieval is semantic similarity, not strict factual matching.
For example:
The user uses FastAPI.
and:
The user uses a Python web framework.
These may have a high degree of semantic similarity.
But this does not mean the system can determine every factual relationship on the strength of vector similarity alone.
3. Hybrid Retrieval
In practice, a system usually should not rely on vector retrieval alone.
It can combine:
- Keyword search.
- Vector similarity search.
- User ID filtering.
- Time filtering.
- Memory type filtering.
- Exact matching on business fields.
For example, if the user asks:
What was the order number we settled on last time?
This kind of question is better served by structured fields or exact keyword search than by semantic similarity alone.
Hybrid retrieval can improve the accuracy of memory recall.
8. Context Compression and Token Budget Management
Context management requires not only deciding which information to keep but also controlling the total volume of input tokens.
1. Token Budget Allocation
Suppose the context budget allowed by the model is 32K tokens.
The budget can be divided as follows:
| Content | Example budget |
|---|---|
| System Prompt | 2K |
| Developer instructions | 1K |
| Long-term memory | 2K |
| History summary | 4K |
| Recent conversation | 12K |
| Current user input | 2K |
| Output reserve | 8K |
| Other overhead | 1K |
| Total | 32K |
This is only an example of a budget design, not a fixed allocation ratio.
The specific allocation should be adjusted according to the task type, model capabilities, and output requirements.
For example:
- Code review needs to retain more code context.
- Customer service tasks need to retain order and user status.
- Ordinary knowledge Q&A does not need to load large amounts of long-term memory.
- Long document analysis needs to reserve more input budget for the original document.
2. Trimming Based on Token Count
You cannot control context simply by character count.
The token counts of different languages, code, numbers, and special symbols can differ significantly.
A more reasonable approach is to estimate the input length using the tokenizer that corresponds to the target model.
Pseudocode:
def build_context(
system_prompt,
memory,
summary,
recent_messages,
current_message,
max_tokens,
output_reserve
):
context = {
"system": system_prompt,
"memory": memory,
"summary": summary,
"messages": recent_messages,
"current": current_message
}
while count_tokens(context) + output_reserve > max_tokens:
if context["messages"]:
context["messages"].pop(0)
elif context["summary"]:
context["summary"] = compress_summary(
context["summary"]
)
elif context["memory"]:
context["memory"] = filter_memory(
context["memory"]
)
else:
raise ValueError("无法在当前预算内构建上下文")
return context