I. What Is Context Engineering?
Context Engineering—often referred to in Chinese simply as “context engineering”—is a set of engineering methods for designing, constructing, filtering, organizing, compressing, and managing the entire context fed into a large language model around the model’s operational requirements.
It focuses not on how a single Prompt should be written, but on:
- What information should the model receive?
- Which information should enter the context?
- How should relevant knowledge be retrieved from external data sources?
- How should historical information in multi-turn conversations be managed?
- How should tool definitions, tool results, and task state be organized?
- How should context length, information quality, and token cost be controlled?
- How can we ensure that the model continuously receives the right information in complex tasks?
Traditional Prompt Engineering focuses more on the instructions themselves, whereas Context Engineering goes further to focus on the entire information environment on which the model relies when executing tasks.
1.1 From Prompt Engineering to Context Engineering
Prompt Engineering usually focuses on how prompts are expressed, for example:
- How to clarify the task objective?
- How to design role instructions?
- How to constrain the model’s output format?
- Whether Few-shot examples are needed?
- How to use CoT to guide complex tasks?
Context Engineering has a broader scope. It includes not only Prompts but also other information the model needs at runtime.
| Comparison Dimension | Prompt Engineering | Context Engineering |
|---|---|---|
| Core Object | Prompts and instructions | The model’s complete context |
| Primary Goal | Improve instruction understanding and execution quality | Provide a complete, relevant, and reliable information environment |
| Focus | Role, task, constraints, examples | Instructions, knowledge, history, tools, state, memory |
| Data Sources | Mainly text constructed by the application | Internal data, external knowledge bases, tools, user input, etc. |
| Engineering Focus | Prompt design and optimization | Information acquisition, filtering, organization, updating, and management |
| Typical Applications | Classification, extraction, content generation | RAG, Agent, multi-turn conversations, complex workflows |
The two can be understood as:
- Prompt Engineering: designing how to make requests to the model.
- Context Engineering: designing what information the model can see when completing a task, and how that information is organized.
Prompt Engineering is an important part of Context Engineering, but it cannot represent the whole of context engineering.
1.2 Why Do We Need Context Engineering?
Large language models themselves have several important limitations.
First, models cannot automatically access all external information.
Models can use the parametric knowledge learned during training, but they cannot inherently read enterprise databases, internal documents, real-time business data, or users’ local files.
Relevant information needs to be provided to the model through retrieval, tool calls, and data interfaces.
Second, the model’s context window is limited.
The context window determines the total number of tokens the model can process in a single request.
Even if a model supports a longer context, it does not mean that putting all historical records and documents into it is the optimal approach.
An overly long context may lead to:
- Increased token cost.
- Increased Prefill computation.
- Increased response latency.
- Interference from irrelevant information.
- Key content being drowned out by a large amount of information.
- Decreased efficiency in utilizing information in long contexts.
Third, multi-turn tasks require state maintenance.
In complex conversations, the model not only needs to understand the current question but may also need to know:
- What requirements the user has previously made.
- Which steps have already been completed.
- What stage the current task is at.
- Which tools have been executed.
- Which results have been confirmed.
- What should be completed next.
Without state management, the model may repeat tasks, forget constraints, or make judgments based on outdated information.
Fourth, Agents need to acquire information dynamically.
An Agent does not merely receive an input once and generate an answer; it may also retrieve materials, call tools, observe results, and adjust plans during execution.
The context must change dynamically as the task progresses.
Therefore, the core value of Context Engineering is to turn the originally fragmented process of information acquisition and organization into an engineering system that can be designed, controlled, and evaluated.
II. What Does the Model Context Contain?
Context is not simply a piece of Prompt; it is the set of information that the model can access in a single inference request.
In different models and APIs, the specific message structure of the context may differ, but it can usually be understood from the following dimensions.
2.1 Instructions: Instruction Information
Instructions define what the model should do and which rules it should follow.
Common contents include:
- System Prompt: system-level behavior instructions.
- Developer Prompt: task and behavior constraints provided by the application developer.
- User Prompt: the user’s current input.
- Output format requirements.
- Task execution rules.
- Security and permission constraints.
For example, an enterprise knowledge base Q&A system may require:
Role: Enterprise Knowledge Base Assistant
Task:
Answer user questions based on the provided enterprise documents.
Constraints:
1. Prioritize retrieved documents as the basis.
2. Do not fabricate information that does not exist in the documents.
3. If the evidence is insufficient, clearly state that it cannot be confirmed.
4. Cite the relevant document sources when answering.
Output format:
Use Markdown.
Instructions address the issues of model behavior and task execution rules.
2.2 Knowledge: Knowledge Information
Knowledge refers to the facts, documents, and business knowledge that the model needs to complete the current task.
Common sources include:
- Internal enterprise documents.
- Product manuals.
- Technical documentation.
- Database records.
- Data returned by APIs.
- Search engine results.
- Vector database retrieval results.
- Files uploaded by users.
For example, when a user asks about the refund rules for one of the enterprise's products, the model needs to obtain the corresponding product policy rather than relying solely on general knowledge learned during training.
Knowledge information is typically obtained through RAG, database queries, or external APIs.
2.3 History: Conversation History
History refers to historical interaction information relevant to the current task.
It includes:
- Questions previously asked by the user.
- Answers previously given by the model.
- Constraints that have already been confirmed.
- Tool invocations that have already been executed.
- Results returned by tools.
- Unresolved issues.
Conversation history helps the model understand the relationship between the current input and previous interactions.
However, not all historical messages need to be retained permanently. Irrelevant, duplicate, and outdated content may need to be filtered or compressed.
2.4 Tools: Tool Information
Tool information typically includes the external capabilities available to the model and the rules for invoking them.
For example:
- Search tools.
- Database query tools.
- File reading tools.
- Code execution tools.
- Calendar tools.
- Order query tools.
- Internal enterprise business APIs.
Tool context typically includes:
- Tool name.
- Tool purpose.
- Parameter definitions.
- Parameter types.
- Required fields.
- Invocation constraints.
- Return data structure.
Tool definitions enable the model to determine when external capabilities need to be invoked and how invocation parameters should be constructed.
Note that a tool invocation request itself does not mean the tool has already been executed. Typically, the application or Agent Runtime is responsible for validating and executing the tool and providing the execution result back to the model.
2.5 State: Task State
State is a very important category of context in complex tasks.
It records the runtime state of the current task rather than simply repeating the conversation history.
For example, a travel booking Agent can maintain:
{
"task": "旅行预订",
"status": "in_progress",
"destination": "Tokyo",
"departure_date": "2026-12-10",
"return_date": "2026-12-15",
"hotel_selected": false,
"flight_selected": true,
"pending_actions": [
"选择酒店",
"确认订单"
]
}
Structured state is more reliable than re-inferring task progress from a large number of chat records.
A distinction needs to be made:
- History: what has happened.
- State: what state it is currently in.
- Plan: what to do next.
The three are interrelated but cannot simply be treated as equivalent.
2.6 Memory: Memory Information
Memory is typically used to store information that remains valuable across turns, tasks, or sessions.
For example:
- Long-term user preferences.
- Business constraints explicitly confirmed by the user.
- Long-term project background.
- Frequently used technology stacks.
- Established ways of working.
The difference between Memory and History is as follows:
| Comparison Dimension | History | Memory |
|---|---|---|
| Core purpose | Reconstruct the interaction process | Retain long-term valuable information |
| Time range | Current session or recent tasks | Cross-session or long-term |
| Data form | Message records | Summaries, facts, preferences, structured records |
| Update method | Generated with the conversation | Filtered, extracted, and updated |
| Main issue | History too long | Memory outdated or inaccurate |
Memory should not be a simple backup of all historical messages. From an engineering perspective, it is necessary to consider information validity periods, source credibility, update mechanisms, and conflict handling.
3. Overall Architecture of Context Engineering
In real systems, context engineering is usually not an independent module but is jointly composed of multiple components.
3.1 Typical Architecture
flowchart TB
A["User request"] --> B["Task identification and context construction"]
B --> C["Context data acquisition"]
C --> D["Instructions and rules"]
C --> E["Conversation history and memory"]
C --> F["Knowledge retrieval"]
C --> G["Tool definitions and task state"]
D --> H["Context filtering and integration"]
E --> H
F --> H
G --> H
H --> I["Context budgeting and compression"]
I --> J["Model inference"]
J --> K{"Is an external operation required?"}
K -->|"Yes"| L["Tool invocation and execution"]
L --> M["Result validation and state update"]
M --> H
K -->|"No"| N["Output result"]
N --> O["Response and context maintenance"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#EAF3EC,stroke:#83A995,color:#315D49
style D fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style E fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style F fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style G fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style H fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style I fill:#EAF3EC,stroke:#83A995,color:#315D49
style J fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style K fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style L fill:#EAF3EC,stroke:#83A995,color:#315D49
style M fill:#EAF3EC,stroke:#83A995,color:#315D49
style N fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style O fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
This architecture illustrates the main workflow of context engineering:
- Receive the user request and identify the task type.
- Retrieve the instructions, knowledge, history, memory, tools, and state required by the task.
- Filter for information relevant to the current task.
- Rank and compress it according to the context budget.
- Hand the constructed context to the model for inference.
- If a tool operation is needed, execute the tool and update the context.
- Generate the result and maintain subsequent state as the task requires.
The most important point here is that context is not assembled once and then left unchanged; it may be continuously updated as the task executes.
3.2 The Boundary Between Context Construction and Model Inference
Context Engineering is usually the responsibility of the application layer and the Agent Runtime.
The model is primarily responsible for:
- Understanding the context.
- Identifying task intent.
- Generating responses.
- Determining whether a tool needs to be called.
- Generating tool call parameters.
- Continuing inference based on tool results.
The application layer is usually responsible for:
- Retrieving user and business data.
- Retrieving external knowledge.
- Managing conversation history.
- Constructing context.
- Controlling the token budget.
- Validating tool parameters.
- Executing external operations.
- Updating task state.
- Saving and managing memory.
This division of responsibilities is very important.
Model-generated content must not be treated as an external operation already completed, and the model's own judgments must not be treated as permission checks.
4. Context Acquisition: How Do You Choose the Right Information?
The first core question in context engineering is information acquisition.
More information is not necessarily better; the key is to find information that is relevant to the current task, trustworthy, and sufficiently complete.
4.1 Static Context and Dynamic Context
From the perspective of how information is acquired, it can be divided into two categories.
Static context
Information that is relatively fixed when the application is running or a request is being processed.
For example:
- System instructions.
- Role definitions.
- Output specifications.
- Tool schemas.
- Fixed business rules.
- Product capability descriptions.
Static context is usually managed through application configuration or versioned templates.
Dynamic context
Information that changes with user input, task progress, and external data.
For example:
- The current user question.
- The latest conversation history.
- Retrieval results.
- Real-time database records.
- Tool execution results.
- The current task state.
- Files temporarily uploaded by the user.
Dynamic context needs to be re-acquired or updated for each task request.
| Type | Main Sources | Typical Characteristics |
|---|---|---|
| Static context | Configuration, templates, rule bases | Relatively stable and reusable |
| Dynamic context | User input, retrieval, tools, state | Changes with the task and requires updates |
4.2 Retrieve on Demand Instead of Loading All at Once
Suppose the user asks:
What is the current status of order 20261120001?
The system does not need to put the entire enterprise knowledge base, all order records, and the full conversation history into the context.
A more reasonable approach is:
- Identify that this is an order status query task.
- Extract the order number.
- Call the order query API.
- Retrieve the data for the specified order.
- Validate the query result.
- Provide the necessary fields to the model.
This approach reduces irrelevant information and lowers context cost.
For complex tasks, a staged retrieval strategy can also be used: first retrieve the information needed to complete the current step, and continue retrieving only when subsequent tasks actually require it.
4.3 Information Sources and Credibility
Information from different sources has different levels of credibility and timeliness.
For example:
| Information Source | Typical Characteristics | Issues to Note |
|---|---|---|
| System configuration | Relatively high authority | Configuration may be outdated |
| Enterprise official documentation | Clear business rules | Version and applicable scope |
| Database records | Structured and queryable | Data consistency and update time |
| External web pages | Broad coverage | Source quality and timeliness |
| Historical conversations | Contain interaction context | The user may have previously provided incorrect information |
| Model parameter knowledge | No external retrieval required | May be outdated or inaccurate |
In engineering practice, source, timestamp, version, and credibility markers can be retained for context information.
For example:
{
"content": "The standard refund period is 7 days",
"source": "official_refund_policy",
"version": "v3",
"updated_at": "2026-11-01",
"authority": "official"
}
The model can use this metadata to understand the source and timeliness of information, and the application can use it to enforce priority and validation rules.
Note that labeling information with credibility does not automatically enable the model to correctly distinguish true from false. Critical rules should still be controlled by the application layer.
5. Context Organization: How to Make Information Easier for the Model to Use?
After retrieving information, we still need to address how to organize it.
The same set of data may lead to different model performance when organized with different structures and orders.
5.1 Structured Context
Organize different types of information separately as much as possible, instead of mixing everything into one large block of natural language.
For example:
# Task
Answer the user's question about the order status.
# User Request
What is the current status of order 20261120001?
# Business Rules
- Only query orders that the current user has permission to access.
- Do not infer order information that was not returned.
- The order status must be based on the result returned by the order system.
# Retrieved Data
Order number: 20261120001
Order status: Shipped
Logistics status: In transit
Updated at: 2026-11-20 09:30
# Output Requirements
Explain the current order status and the most recent update time.
This organizational approach helps distinguish tasks, rules, data, and output constraints.
It should be emphasized that the headings here are only illustrative organizational devices; not all model APIs have a unified context partitioning protocol. The specific roles, message fields, and content formats should be based on the target model's interface.
5.2 Information Priority
There may be a large amount of information in the context, but not all information is equally important.
It can be organized according to task relevance and business importance:
| Priority | Information Type | Example |
|---|---|---|
| High | Core task and hard constraints | Current task, permission restrictions |
| High | Direct evidence | Current order query result |
| High | Key task status | Completed steps, pending steps |
| Medium | Relevant historical information | Conditions previously confirmed by the user |
| Medium | Supporting knowledge | Product background, operating instructions |
| Low | Irrelevant history | Chitchat unrelated to the current task |
Note that information priority is a context management strategy and is not equivalent to instruction priority in the model protocol.
For example, when security rules conflict with user requests, this cannot be resolved simply by placing certain text before or after other text. It should be handled according to the role hierarchy of the model interface and the permission policies on the application side.
5.3 Information Order
The order of information in the context may affect how the model uses it.
For example:
- State the task objective first, then provide relevant background.
- Place important constraints in a clear and stable position.
- Separate direct evidence from explanatory content.
- In long contexts, avoid surrounding key conditions with large amounts of irrelevant content.
Long-context models may also exhibit the so-called Lost in the Middle phenomenon: in some tasks, the model makes less effective use of information located in the middle of the context than information at the beginning or the end.
This is not a rule that necessarily holds for all models and tasks, but it shows that context window capacity does not equal the ability to effectively use information.
In engineering practice, you should test information ordering with your actual model and business tasks, rather than assuming the model is guaranteed to use information at every position equally well.
6. Context Budget: How Should Limited Tokens Be Allocated?
The context window is an important constraint in context engineering.
The total amount of information a model can process in a single request is limited, and the input context and output generation typically share the model's context length limit.
6.1 Composition of the Context Window
The context of a request may typically include:
- System / Developer instructions.
- Current user input.
- History messages.
- Retrieved documents.
- Tool definitions.
- Tool calls and returned results.
- The data or token representations corresponding to multimodal inputs.
- Context information required by other model APIs.
The token budget can be abstracted as:
Where:
- T_{\text{total}}: total token budget.
- T_{\text{instructions}}: instruction consumption.
- T_{\text{history}}: history conversation consumption.
- T_{\text{knowledge}}: retrieved knowledge consumption.
- T_{\text{tools}}: tool definitions and related information consumption.
- T_{\text{input}}: current input consumption.
- T_{\text{output}}: reserved output tokens.
Real systems must also account for additional overhead from the model API, special tokens, hidden formatting overhead, and how tokens are counted for multimodal data.
Therefore, you cannot accurately estimate the full context cost based solely on the length of the visible text.
6.2 Token Budget Allocation
Suppose a model request allows 32,000 tokens; you can design a budget based on the characteristics of the task.
| Context component | Example budget |
|---|---|
| System and developer instructions | 2,000 |
| Conversation history | 5,000 |
| Retrieved knowledge | 10,000 |
| Tool definitions and results | 3,000 |
| Current user input | 2,000 |
| Reserved output | 5,000 |
| Buffer space | 5,000 |
| Total | 32,000 |
This is only a budgeting example, not a universal optimal ratio.
The context composition of different tasks varies greatly:
- Simple Q&A requires less history and knowledge.
- Long document analysis requires more document content.
- Agent tasks need to account for tool definitions, execution results, and state.
- Code analysis may require a larger code context.
- Multimodal tasks also need to account for the data overhead of images, audio, or video.
Budget allocation should be adjusted dynamically according to the task type.
6.3 Context Compression
When the context approaches the window limit, some information needs to be compressed or evicted.
Common strategies include:
1. History truncation
Delete earlier messages that are only weakly related to the current task.
Suitable for simple conversations, but it may lose important early constraints.
2. History summarization
Condense a long conversation into a structured summary.
For example:
User goal:
Build an enterprise knowledge base question-answering system.
Confirmed requirements:
- Use Python.
- Use PostgreSQL.
- Use vector retrieval.
- Use FastAPI for the API.
Current progress:
Data import design is complete.
To be done:
Design the retrieval and reranking module.
3. Retrieval-based history management
Instead of putting all historical content into the current context, store the history in external storage and retrieve the relevant content when needed.
4. Structured state extraction
Extract the key facts, constraints, and task progress from the conversation into structured data.
5. Redundant information removal
Remove duplicate documents, duplicate tool results, irrelevant background, and information that is no longer valid.
6.4 Risks of Context Compression
Compression is not simply shortening the text.
If the compression process loses important information, it may cause the model to make incorrect judgments.
Common risks include:
- Key conditions are omitted from the summary.
- Numbers, times and entity names are altered.
- Confirmed facts are conflated with information still to be confirmed.
- Task states are incorrectly merged.
- Historical constraints are overwritten by new information.
- The chain of sources and evidence is lost.
Therefore, compression should follow one principle:
You can shorten the expression, but you must not arbitrarily change facts, constraints and task states.
For key fields such as amounts, order numbers, user permissions and time ranges, prefer structured storage rather than relying solely on natural-language summaries.
7. RAG: The Knowledge Acquisition Mechanism in Context Engineering
RAG (Retrieval-Augmented Generation) is one of the most typical applications of context engineering.
It addresses the question of how a model acquires external knowledge during reasoning.
7.1 The Core Idea of RAG
Traditional large language models rely mainly on the knowledge encoded in model parameters.
But parametric knowledge has drawbacks: updates are costly, information may become outdated, and it cannot naturally access private data.
RAG combines external knowledge retrieval with model generation.
Before the model generates an answer, the system first retrieves relevant content from a knowledge base, then adds the retrieval results to the context.
7.2 The Basic RAG Workflow
flowchart TB
A["Raw documents"] --> B["Document parsing"]
B --> C["Text cleaning"]
C --> D["Chunking"]
D --> E["Embedding vectorization"]
E --> F["Vector database"]
G["User question"] --> H["Question processing"]
H --> I["Retrieve relevant content"]
F --> I
I --> J["Reranking"]
J --> K["Context construction"]
K --> L["Large language model"]
L --> M["Generate answer"]
style A fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style B fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style C fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style D fill:#EAF3EC,stroke:#83A995,color:#315D49
style E fill:#EAF3EC,stroke:#83A995,color:#315D49
style F fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style G fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style H fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style I fill:#EAF3EC,stroke:#83A995,color:#315D49
style J fill:#EAF3EC,stroke:#83A995,color:#315D49
style K fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style L fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style M fill:#DCEFE5,stroke:#4F8B70,color:#24563F
7.3 The Relationship Between RAG and Context Engineering
RAG is mainly responsible for acquiring external knowledge, while Context Engineering is responsible for managing how that knowledge enters the model context.
For example, RAG may retrieve 20 document fragments, but that does not mean all of them should be handed directly to the model.
Context engineering also needs to consider:
- Which fragments are truly relevant?
- Is there duplicate content?
- Are there documents that conflict with one another?
- Have the documents expired?
- Should the original sources be retained?
- Is further reranking needed?
- How many tokens should ultimately be used?
Therefore, RAG is a context acquisition mechanism, while Context Engineering is a more complete context management system.
7.4 Key Optimization Points for RAG
Retrieval quality
Retrieval results must be relevant to the user's question. Vector retrieval can be used for semantic matching, but in scenarios involving exact numbers, product models and proper nouns, keyword retrieval is usually also needed.
Chunk strategy
If chunks are too small, semantic completeness may be lost; if chunks are too large, they may contain a lot of irrelevant information.
Chunk size, overlap ratio and chunking method should be evaluated according to document type and task.
Reranking
Initial retrieval can expand the candidate range, and then a reranking model sorts the candidates by relevance.
Metadata filtering
Filter by metadata such as document version, department, permissions and time.
Evidence retention
Retain source information such as document ID, title, page number and paragraph location to facilitate citation and traceability.
Permission control
User permission filtering must be enforced at the retrieval stage; it cannot rely solely on the model to judge which documents are accessible.
8. Memory: How to Manage Long-Term Context?
For cross-session and long-term tasks, relying only on the current conversation history is usually not enough.
This is when a Memory mechanism is needed.
8.1 The Core Role of Memory
Memory mainly solves two problems:
- How is information outside the current conversation window retained?
- When future tasks need that information, how is it provided to the model again?
Memory is not the same as a complete chat log, nor is it the same as model parameters.
It is usually a set of external information maintained at the application layer, implemented through databases, file storage, vector databases or other persistence systems.
8.2 Common Types of Memory
| Type | Description | Example |
|---|---|---|
| Profile Memory | User's long-term attributes and preferences | Preferred language, expression preferences |
| Semantic Memory | Long-term facts and knowledge | Project tech stack, business rules |
| Episodic Memory | Important events from the past | Completion status of a certain task |
| Procedural Memory | Long-term working methods or processes | Fixed code review standards |
| Task Memory | Intermediate state of the current task | Current progress, pending items |
These types are common engineering categorizations; not all frameworks use exactly the same definitions.
8.3 Memory Lifecycle
Memory is not simply writing and reading; its full lifecycle also needs to be managed.
flowchart TB
A["Generate interaction information"] --> B["Identify candidate memories"]
B --> C["Extract key facts"]
C --> D["Check quality and permissions"]
D --> E{"Is it worth saving?"}
E -->|"Yes"| F["Write to memory storage"]
E -->|"No"| G["Discard or keep only in history"]
F --> H["Retrieve and use"]
H --> I["Update or merge"]
I --> J["Validity check"]
J --> K{"Is it still valid?"}
K -->|"Yes"| F
K -->|"No"| L["Expire or delete"]
style A fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#EAF3EC,stroke:#83A995,color:#315D49
style D fill:#EAF3EC,stroke:#83A995,color:#315D49
style E fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style F fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style G fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style H fill:#EAF3EC,stroke:#83A995,color:#315D49
style I fill:#EAF3EC,stroke:#83A995,color:#315D49
style J fill:#EAF3EC,stroke:#83A995,color:#315D49
style K fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style L fill:#F7EAE7,stroke:#C98D7C,color:#754438
8.4 Problems Memory Needs to Solve
Memory Filtering
Not everything the user says is worth saving.
Filtering should be based on long-term value, task relevance, user authorization, and sensitivity.
Memory Conflicts
For example, a user previously used Java and later switched to Go.
The system needs to identify the temporal relationship between old and new information and avoid retaining two contradictory current facts at the same time.
Memory Expiration
Some information is time-sensitive, such as temporary project progress, order status, and short-term arrangements.
An expiration time or periodic revalidation is needed.
Memory Contamination
If incorrect content generated by a model is saved directly as long-term memory, the error may continue to propagate in the future.
Therefore, important memories should retain provenance and be verified when necessary.
Privacy and Permissions
A memory system needs to support access control, user authorization, data lifecycle management, and deletion mechanisms.
Just because a piece of information once appeared in a conversation does not mean it can be stored permanently and used across tasks by default.
9. Agent Context Engineering: Dynamic Task Context
When a system evolves from simple question answering into an Agent, the complexity of Context Engineering increases further.
This is because Agent context must not only describe user needs but also change continuously throughout the execution process.
9.1 Composition of Agent Context
At runtime, an Agent usually needs to maintain:
- System Instructions: behavior and execution rules.
- User Goal: the user's objective.
- Task Plan: the current task plan.
- Working Memory: current working information.
- Tool Definitions: callable tools.
- Tool Results: tool execution results.
- Environment State: external environment state.
- Execution History: execution history.
- Constraints: permission, security, and business constraints.
Together, these determine what the Agent should do next.
9.2 Agent Context Loop
flowchart TB
A["Receive task"] --> B["Build initial context"]
B --> C["Model analyzes current state"]
C --> D["Generate next action"]
D --> E{"Tool needed?"}
E -->|"Yes"| F["Validate tool call"]
F --> G["Execute tool"]
G --> H["Obtain execution result"]
H --> I["Update task status"]
I --> J["Update context"]
J --> C
E -->|"No"| K["Generate final result"]
K --> L["Save necessary task information"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#EAF3EC,stroke:#83A995,color:#315D49
style D fill:#EAF3EC,stroke:#83A995,color:#315D49
style E fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style F fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style G fill:#EAF3EC,stroke:#83A995,color:#315D49
style H fill:#EAF3EC,stroke:#83A995,color:#315D49
style I fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style J fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style K fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style L fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
This process can continue for multiple rounds.
After each tool execution, the Agent needs to update the context based on the latest result.
For example:
User request:
Help me analyze the sales performance over the past month,
identify the product with the most significant sales decline,
and generate an analysis report.
The Agent may need to:
- Query sales data for the most recent month.
- Query sales data for the previous month.
- Align by product dimension.
- Calculate sales changes.
- Identify the product with the most significant decline.
- Query related product information.
- Organize analysis conclusions.
- Generate a report.
Each stage produces new data and state.
If only the complete tool call history is retained without maintaining structured state, the context will quickly balloon as tasks become more complex.
9.3 Working Memory
Working Memory is the set of temporary information used by the Agent while executing the current task.
Unlike long-term Memory, it primarily serves the current task.
For example:
{
"goal": "Analyze sales performance over the past month",
"step": "compare_sales",
"completed": [
"query_current_month",
"query_previous_month"
],
"current_findings": {
"highest_decline_product": "Product A",
"decline_rate": 0.23
},
"pending": [
"query_product_details",
"generate_report"
]
}
This structured state helps the Agent keep track of current progress across multiple execution rounds.
Note that Working Memory is a concept in application architecture and does not necessarily correspond to a separate field in the model API.
9.4 Tool results should not enter the context indiscriminately
Tools may return large amounts of data.
For example, a database query may return 10,000 records, but the model may only need:
- Total sales.
- Product group summary.
- Month-over-month change.
- Anomalous products.
- Relevant statistical results.
A better approach is to have the program first perform deterministic calculations and data filtering, then provide the necessary results to the model.
For large-scale data analysis, the model should not be treated as a database or a high-precision calculation engine.
The model is better suited to interpreting results, organizing analysis, and generating reports; precise calculations should be delegated to deterministic programs wherever possible.