I. The Overall Architecture of an Agent
An AI Agent is not a single large model, but rather an intelligent task-execution system composed of components such as the model, runtime, tools, memory, state management, and execution control.
From a system-architecture perspective, an Agent can be divided into six core layers:
- Interaction layer: receives user goals and returns task results.
- Decision layer: understands goals, analyzes context, and formulates plans.
- Reasoning layer: calls the large model to perform reasoning and decision-making.
- Execution layer: schedules tools and performs external operations.
- State and memory layer: maintains task state and historical information.
- Control and governance layer: handles permissions, security, exceptions, timeouts, and resource limits.
1. Overall Architecture Diagram
flowchart TB
U["User / External System"] --> API["Agent API"]
API --> R["Agent Runtime"]
subgraph Decision["Decision and Reasoning Layer"]
G["Goal Manager"]
P["Planner"]
L["LLM Engine"]
D["Decision Engine"]
G --> P
P --> L
L --> D
end
subgraph Execution["Tool Execution Layer"]
TR["Tool Registry"]
TE["Tool Executor"]
EX["External Systems"]
TR --> TE
TE --> EX
end
subgraph Memory["State and Memory Layer"]
S["State Manager"]
M["Memory Manager"]
C["Context Manager"]
S --> C
M --> C
end
subgraph Governance["Control and Governance Layer"]
POL["Policy Engine"]
OBS["Observability"]
ERR["Error Handler"]
end
R --> G
D --> TR
EX --> S
C --> L
POL --> R
TE --> OBS
ERR --> R
R --> OUT["Result Validator"]
OUT --> RESP["Final Result"]
style U fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style API fill:#EAF3EC,stroke:#83A995,color:#315D49
style R fill:#EAE8FA,stroke:#A49BD5,color:#51477D
style G fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style P fill:#EAE8FA,stroke:#A49BD5,color:#51477D
style L fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style D fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style TR fill:#EAF3EC,stroke:#83A995,color:#315D49
style TE fill:#EAF3EC,stroke:#83A995,color:#315D49
style EX fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style S fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style M fill:#EAE8FA,stroke:#A49BD5,color:#51477D
style C fill:#EAF3EC,stroke:#83A995,color:#315D49
style POL fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style OBS fill:#EAF3EC,stroke:#83A995,color:#315D49
style ERR fill:#F7EAE7,stroke:#C98D7C,color:#754438
style OUT fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style RESP fill:#DCEFE5,stroke:#4F8B70,color:#24563F
The diagram shows a logical architecture; it does not mean that all components must be deployed independently. A small Agent can place multiple modules in the same process, while a complex system can be split into independent services.
2. The Core Components of an Agent
| Component | Core Responsibility | Key Question |
|---|---|---|
| Agent Runtime | Manages the Agent lifecycle and execution loop | How to keep a task running continuously |
| Goal Manager | Parses goals and constraints | How to determine what the task must accomplish |
| Planner | Task decomposition and path planning | What should be done next |
| LLM Engine | Reasoning, decision-making and content generation | How to obtain sound decisions |
| Tool Registry | Manages tool definitions | Which tools are available |
| Tool Executor | Executes external operations | How to execute tools safely |
| Memory Manager | Manages long-term memory | Which information needs to be retained |
| State Manager | Manages the current task state | Where the task execution has reached |
| Context Manager | Organizes the context for model input | What the model should currently see |
| Policy Engine | Permission and security control | Which operations are permitted |
| Error Handler | Handles exceptions and recovery | How to continue after a failure |
| Observability | Records the execution process | How to locate problems and evaluate results |
Three easily confused concepts need to be distinguished in particular:
- Runtime: drives the Agent's execution.
- LLM: provides reasoning and generation capabilities.
- Orchestrator: organizes collaboration among multiple execution components or Agents.
Different frameworks may divide responsibilities between the Runtime and the Orchestrator differently, but from an engineering standpoint it must be clear who handles scheduling, who handles execution, and who maintains state.
II. Agent Runtime: The Execution Core of the Entire System
Agent Runtime is the Agent's execution engine, responsible for turning model decisions into actual operations and continuously driving the task until the termination conditions are met.
It is not equivalent to the large model, nor should all business logic be stuffed into the Prompt.
1. Main Responsibilities of the Runtime
Task lifecycle management
- Create tasks.
- Initialize the execution context.
- Start the execution loop.
- Maintain task state.
- Handle interruption and recovery.
- End the task and return the result.
Model invocation management
- Construct model input.
- Select the model and parameters.
- Call the model API.
- Parse model output.
- Handle model invocation exceptions.
Tool execution management
- Parse tool invocation requests.
- Validate tool names and parameters.
- Check execution permissions.
- Invoke the tool executor.
- Collect execution results.
Execution control
- Limit the maximum number of execution steps.
- Limit the number of model invocations.
- Control task timeouts.
- Manage concurrency and resources.
- Determine whether human intervention is needed.
2. The Runtime's Execution Flow
flowchart TD
A["Create Agent Task"] --> B["Initialize Runtime"]
B --> C["Load task state"]
C --> D["Build model context"]
D --> E["Invoke LLM"]
E --> F["Parse model decision"]
F --> G{"Decision type"}
G -->|"Tool call"| H["Execute tool"]
G -->|"Continue reasoning"| I["Update context"]
G -->|"Final answer"| J["Validate task result"]
H --> K["Collect tool results"]
K --> I
I --> L{"Execution limit reached?"}
L -->|"No"| D
L -->|"Yes"| M["Terminate or request human intervention"]
J --> N{"Success criteria met?"}
N -->|"Yes"| O["Complete task"]
N -->|"No"| M
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style D fill:#EAF3EC,stroke:#83A995,color:#315D49
style E fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style F fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style G fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style H fill:#EAF3EC,stroke:#83A995,color:#315D49
style I fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style J fill:#EAE8FA,stroke:#A49BD5,color:#51477D
style K fill:#EAF3EC,stroke:#83A995,color:#315D49
style L fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style M fill:#F7EAE7,stroke:#C98D7C,color:#754438
style N fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style O fill:#DCEFE5,stroke:#4F8B70,color:#24563F
3. Why do we need an independent Runtime?
Suppose we use only a large model to complete an Agent task:
- The model generates tool calls.
- The model describes the execution results on its own.
- The model judges whether the task is complete on its own.
- The model decides whether to continue on its own.
This creates a serious problem: the content output by the model may be mistaken for something that has already been executed.
For example, the model generates:
{
"tool": "delete_file",
"arguments": {
"path": "/tmp/test.txt"
},
"status": "success"
}
This content is only data generated by the model; it does not prove that the file has been deleted.
Actual execution must involve the Runtime calling the tool and obtaining the tool's actual return result.
Therefore:
The model is responsible for proposing action suggestions, while the Runtime is responsible for controlling and carrying out actions.
III. Goal Manager: Goal Understanding and Task Constraints
The starting point for Agent execution is not tools, but the goal.
Users often describe requirements in natural language, and the Runtime needs to convert them into manageable task definitions.
1. Goal Parsing
For example:
Analyze the code quality in the project, identify serious issues, fix the parts that can be fixed automatically, and run tests.
The Goal Manager needs to extract:
| Dimension | Content |
|---|---|
| Goal | Improve project code quality |
| Analysis scope | Current project code |
| Task requirement | Identify serious issues |
| Execution action | Automatically fix fixable issues |
| Verification condition | Run tests |
| Result requirement | Output a report of issues and fixes |
| Execution constraint | Deleting files without authorization is not allowed |
2. Goal Structuring
The goal can be converted into a structured object:
{
"goal_id": "goal_001",
"description": "Analyze and fix project code issues",
"constraints": {
"scope": "current_project",
"allow_file_edit": true,
"allow_file_delete": false,
"require_test": true
},
"success_criteria": [
"Code inspection completed",
"Serious issues addressed",
"Test validation completed",
"Execution report generated"
],
"status": "pending"
}
Structured goals have several important functions:
- They give tasks clear boundaries.
- They give the Planner a reliable basis for planning.
- They enable the Policy Engine to check operation permissions.
- They enable the Result Validator to determine completion conditions.
3. Difference Between Goals and Plans
A goal describes the final desired state, while a plan describes the steps required to achieve the goal.
For example:
- Goal: Fix serious bugs in the project.
- Plan: Read the code, analyze the issues, modify files, run tests, and verify the results.
A goal should not be conflated with specific execution steps.
Otherwise, when a step fails, the Agent will find it difficult to replan.
IV. Planner: Task Decomposition and Execution Planning
The Planner is responsible for decomposing complex tasks into executable subtasks and determining the dependencies between tasks.
1. Task Decomposition
A complex task can be represented as a task tree:
Project Code Quality Analysis
├── 1. Project Scanning
│ ├── Read directory structure
│ ├── Identify language and framework
│ └── Analyze dependencies
├── 2. Code Analysis
│ ├── Static code inspection
│ ├── Security risk inspection
│ └── Logic issue analysis
├── 3. Issue Handling
│ ├── Classify issues
│ ├── Generate fix plan
│ └── Modify code
└── 4. Result Validation
├── Run tests
├── Analyze test results
└── Generate report
2. Common Planning Modes
| Planning Mode | Characteristics | Applicable Scenarios |
|---|---|---|
| Sequential | Execute in sequence | Tasks with clear sequential dependencies |
| Parallel | Execute in parallel | Mutually independent tasks |
| Conditional | Branch based on conditions | Need to choose a path based on results |
| Iterative | Multiple rounds of iteration | Tasks requiring repeated optimization |
| Hierarchical | Layered decomposition | Complex multi-level goals |
3. Sequential Planning
The result of the previous task is the input of the next task.
flowchart TD
A["Read project"] --> B["Analyze code"]
B --> C["Locate issues"]
C --> D["Modify code"]
D --> E["Run tests"]
E --> F["Generate report"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style D fill:#EAE8FA,stroke:#A49BD5,color:#51477D
style E fill:#EAF3EC,stroke:#83A995,color:#315D49
style F fill:#DCEFE5,stroke:#4F8B70,color:#24563F
4. Parallel Planning
When multiple tasks have no data dependencies, they can be executed simultaneously.
flowchart TD
A["Project analysis"] --> B["Security check"]
A --> C["Performance check"]
A --> D["Code style check"]
B --> E["Summarize analysis results"]
C --> E
D --> E
E --> F["Generate issue list"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style C fill:#EAE8FA,stroke:#A49BD5,color:#51477D
style D fill:#EAF3EC,stroke:#83A995,color:#315D49
style E fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style F fill:#DCEFE5,stroke:#4F8B70,color:#24563F
Parallel execution requires particular attention to: - Whether there are data dependencies between tasks. - Whether the same files will be modified simultaneously. - Whether there is resource contention. - Whether results need to be merged. - How to handle the failure of an individual subtask.
5. Conditional Planning
Select subsequent paths based on intermediate execution results.
flowchart TD
A["Run code inspection"] --> B["Analyze inspection results"]
B --> C{"Serious issues found?"}
C -->|"Yes"| D["Generate fix plan"]
C -->|"No"| E["Generate inspection report"]
D --> F["Apply fixes"]
F --> G["Run tests"]
G --> H{"Tests passed?"}
H -->|"Yes"| E
H -->|"No"| I["Record failure and request handling"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style D fill:#EAE8FA,stroke:#A49BD5,color:#51477D
style E fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style F fill:#EAF3EC,stroke:#83A995,color:#315D49
style G fill:#EAF3EC,stroke:#83A995,color:#315D49
style H fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style I fill:#F7EAE7,stroke:#C98D7C,color:#754438
6. Structured Representation of Planning Results
The Planner should not return only a natural-language plan. Engineering systems usually need to convert the plan into structured task objects.
{
"plan_id": "plan_001",
"tasks": [
{
"id": "task_1",
"name": "scan_project",
"dependencies": [],
"status": "pending"
},
{
"id": "task_2",
"name": "analyze_code",
"dependencies": ["task_1"],
"status": "pending"
},
{
"id": "task_3",
"name": "fix_issues",
"dependencies": ["task_2"],
"status": "pending"
},
{
"id": "task_4",
"name": "run_tests",
"dependencies": ["task_3"],
"status": "pending"
}
]
}
This allows the Runtime to schedule, retry, recover, and track tasks.
V. LLM Engine: The Agent's Reasoning Engine
The LLM is the core reasoning component of the Agent, responsible for understanding context, judging task progress, generating plans, and deciding the next action.
1. Responsibilities of the LLM in an Agent
These mainly include:
- Intent understanding: understand the user's goal.
- Context analysis: understand conversation history and tool results.
- Task planning: generate or adjust the execution plan.
- Tool selection: select appropriate tools.
- Parameter generation: generate tool-call parameters.
- Result analysis: interpret tool execution results.
- Decision generation: decide whether to continue execution or end the task.
2. Inputs to Model Invocation
An Agent does not simply send the user's question directly to the model.
A typical model request may include:
{
"system_prompt": "你是一个代码分析 Agent",
"user_goal": "检查项目并修复严重问题",
"conversation_history": [],
"task_state": {
"current_step": "analyze_code",
"completed_steps": ["scan_project"]
},
"memory_context": [],
"available_tools": [
{
"name": "read_file",
"description": "读取指定文件"
},
{
"name": "run_tests",
"description": "运行项目测试"
}
],
"constraints": {
"max_steps": 20,
"allow_file_delete": false
}
}
This information enters the model context only after it has been organized by the Context Manager.
3. Decision Types Output by the Model
An Agent usually needs to constrain model output to a limited set of decision types.
| Decision Type | Meaning |
|---|---|
| TOOL_CALL | Call a tool |
| CONTINUE | Continue reasoning or planning |
| FINAL | Return the final result |
| ASK_USER | Request additional information or confirmation from the user |
| HANDOFF | Hand off to another Agent or a human |
| ABORT | Abort the task |
A simplified structured decision:
{
"action": "TOOL_CALL",
"tool_name": "read_file",
"arguments": {
"path": "src/main.py"
},
"reason": "需要检查程序入口逻辑"
}
It should be emphasized that decisions output by the model must be validated by the Runtime and cannot be directly treated as trusted execution instructions.
6. Tool Registry and Tool Executor: Tool Management and Execution
Tools are the interface through which an Agent interacts with the external world.
However, tool definition, tool selection, and tool execution are three distinct stages.
1. Tool Registry
The Tool Registry is the registry responsible for managing the tools an Agent can use.
A tool usually needs to include:
| Field | Description |
|---|---|
| name | Tool name |
| description | Tool purpose description |
| parameters | Parameter schema |
| permissions | Required permissions |
| timeout | Maximum execution time |
| retry_policy | Retry policy |
| side_effects | Whether it has side effects |
| version | Tool version |
For example:
{
"name": "read_file",
"description": "读取指定路径的文件内容",
"parameters": {
"type": "object",
"properties": {
"path": {
"type": "string",
"description": "文件路径"
}
},
"required": ["path"]
},
"permissions": ["filesystem.read"],
"timeout": 10,
"side_effects": false
}
Tool definitions are usually provided to the model so it knows which tools are available and how each tool is called.
2. Tool Executor
The Tool Executor is responsible for converting tool call requests generated by the model into actual execution operations.
Its core responsibilities include:
- Parse the tool name.
- Look up tool registration information.
- Validate the parameter format.
- Check user and task permissions.
- Check resource limits.
- Execute the specific tool.
- Catch exceptions and timeouts.
- Normalize the returned result.
Execution flow:
flowchart TD
A["Receive tool invocation request"] --> B["Parse tool name"]
B --> C{"Does tool exist?"}
C -->|"No"| X["Return tool-not-found error"]
C -->|"Yes"| D["Validate parameter schema"]
D --> E{"Are parameters valid?"}
E -->|"No"| Y["Return parameter error"]
E -->|"Yes"| F["Check permissions and policy"]
F --> G{"Is execution allowed?"}
G -->|"No"| Z["Deny execution"]
G -->|"Yes"| H["Execute tool"]
H --> I{"Did execution succeed?"}
I -->|"Yes"| J["Normalize result"]
I -->|"No"| K["Handle exception"]
J --> L["Return tool result"]
K --> M["Classify error and decide whether to retry"]
M --> L
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style D fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style E fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style F fill:#EAE8FA,stroke:#A49BD5,color:#51477D
style G fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style H fill:#EAF3EC,stroke:#83A995,color:#315D49
style I fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style J fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style K fill:#F7EAE7,stroke:#C98D7C,color:#754438
style L fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style M fill:#EAE8FA,stroke:#A49BD5,color:#51477D
style X fill:#F7EAE7,stroke:#C98D7C,color:#754438
style Y fill:#F7EAE7,stroke:#C98D7C,color:#754438
style Z fill:#F7EAE7,stroke:#C98D7C,color:#754438
3. Boundary Between Tool Execution and Model Decision-Making
This is a very important boundary in Agent engineering:
- The model can issue requests to call tools.
- Runtime decides whether a request enters the execution phase.
- Policy Engine decides whether an operation is allowed.
- Tool Executor executes the specific operation.
- External tools return the actual execution result.
You cannot assume that a tool has executed successfully just because the model outputs success.
4. Tool Execution Results
It is best to use a unified structure for tool return results:
{
"tool_call_id": "call_001",
"tool_name": "read_file",
"status": "success",
"data": {
"content": "print('hello')"
},
"error": null,
"duration_ms": 25
}
A unified result structure allows Runtime to handle different tools consistently.
VII. Memory, State, and Context: The Three Most Easily Confused Components
Memory, State, and Context all relate to information management, but the three serve different responsibilities.
1. Memory: The Memory System
Memory is used to store information that subsequent tasks may need.
Common categories:
| Type | Content | Lifecycle |
|---|---|---|
| Working memory | Temporary information needed for the current task | During the task |
| Short-term memory | Historical information of the current session | During the session |
| Long-term memory | User preferences, stable knowledge, and historical experience | Across sessions |
| Episodic memory | Past tasks and execution experiences | Stored according to policy |
| Semantic memory | Organized facts and knowledge | Stored according to policy |
The key to Memory is not saving all conversations, but determining which information will be worth reusing in the future.
A typical memory processing flow:
flowchart TD
A["New information generated"] --> B["Assess information value"]
B --> C{"Worth remembering?"}
C -->|"No"| D["Do not save"]
C -->|"Yes"| E["Extract key content"]
E --> F["Organize and standardize"]
F --> G["Categorize and tag"]
G --> H["Write to memory storage"]
H --> I["Build index"]
I --> J["Retrieve for later tasks"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style C fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style D fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
style E fill:#EAE8FA,stroke:#A49BD5,color:#51477D
style F fill:#EAF3EC,stroke:#83A995,color:#315D49
style G fill:#EAF3EC,stroke:#83A995,color:#315D49
style H fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style I fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style J fill:#DCEFE5,stroke:#4F8B70,color:#24563F
2. State: Task state
State is used to record the execution status of the current task.
For example:
{
"task_id": "task_001",
"status": "running",
"current_node": "run_tests",
"completed_nodes": [
"scan_project",
"analyze_code",
"fix_code"
],
"retry_count": 1,
"started_at": "2026-11-20T10:00:00Z"
}
State focuses on:
- Where execution currently is.
- Which steps have been completed.
- Which steps are waiting to be executed.
- Whether the current task has failed.
- Whether continuation or recovery is allowed.
State usually needs a clear data structure, version, and persistence mechanism.
3. Context: Model context
Context is the set of information that the current model invocation can see.
It usually includes:
- System Prompt.
- User goals.
- Historical messages.
- Summary of the current task state.
- Relevant memories.
- Tool definitions.
- Tool execution results.
- Current execution constraints.
Context Manager is responsible for organizing this information into input that the model can understand.
4. Core differences among the three
| Comparison dimension | Memory | State | Context |
|---|---|---|---|
| Core purpose | Save reusable information | Record task progress | Provide input for model reasoning |
| Focus | What to remember | What is currently happening | What the model needs to know |
| Lifecycle | Can span tasks | Usually bound to a task | Bound to a single model invocation |
| Storage method | Databases, vector databases, etc. | State store, database | Prompt and message structure |
| Whether directly provided to the model | Retrieved on demand | Summarized on demand | Provided directly |
| Whether full persistence is required | Depends on memory strategy | Usually required | Usually not required |
The relationship among the three:
flowchart TD
A["Task Execution"] --> B["Produces State Changes"]
A --> C["Produces Reusable Information"]
B --> D["State Manager"]
C --> E["Memory Manager"]
D --> F["Context Manager"]
E --> G["Memory Retrieval"]
G --> F
F --> H["Build Model Input"]
H --> I["LLM Inference"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style C fill:#EAE8FA,stroke:#A49BD5,color:#51477D
style D fill:#EAF3EC,stroke:#83A995,color:#315D49
style E fill:#EAF3EC,stroke:#83A995,color:#315D49
style F fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style G fill:#EAE8FA,stroke:#A49BD5,color:#51477D
style H fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style I fill:#DCEFE5,stroke:#4F8B70,color:#24563F
In engineering practice, Memory, State, and Context cannot simply be merged into a single conversation history. Their data structures, update timing, and persistence requirements are all different.
8. Context Manager: Context Construction and Management
The Context Manager decides what information should be supplied for each model call.
This is especially important for Agents, because an Agent may run for dozens or even hundreds of rounds.
1. Why Is a Context Manager Needed?
Suppose an Agent is executing a complex task:
- It has read 100 files.
- It has called tools 30 times.
- It has produced a large number of intermediate results.
- It has gone through multiple rounds of plan adjustments.
If all historical information were passed in full on every model call, you would run into:
- Rapidly growing token consumption.
- Insufficient model context window.
- Important information drowned out by a mass of irrelevant information.
- Increased inference cost and latency.
2. Context Management Strategies
| Strategy | Purpose |
|---|---|
| Sliding Window | Keep only a recent portion of the history |
| Summarization | Compress old messages into summaries |
| Retrieval | Retrieve relevant memories on demand |
| Truncation | Remove low-value content |
| Context Partitioning | Manage different types of information in separate partitions |
| Structured State | Use structured state instead of lengthy conversations |
| Tool Result Filtering | Filter out irrelevant content from tool returns |
3. Context Construction Process
flowchart TD
A["Prepare model call"] --> B["Load system instructions"]
B --> C["Load current goals and constraints"]
C --> D["Load task state"]
D --> E["Retrieve relevant memories"]
E --> F["Filter historical messages"]
F --> G["Organize tool definitions and results"]
G --> H["Calculate token budget"]
H --> I{"Exceed context limit?"}
I -->|"Yes"| J["Compress or trim content"]
J --> H
I -->|"No"| K["Generate final context"]
K --> L["Send model request"]
style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style B fill:#EAF3EC,stroke:#83A995,color:#315D49
style C fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style D fill:#EAF3EC,stroke:#83A995,color:#315D49
style E fill:#EAE8FA,stroke:#A49BD5,color:#51477D
style F fill:#EAF3EC,stroke:#83A995,color:#315D49
style G fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style H fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style I fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
style J fill:#EAE8FA,stroke:#A49BD5,color:#51477D
style K fill:#DCEFE5,stroke:#4F8B70,color:#24563F
style L fill:#DCEFE5,stroke:#4F8B70,color:#24563F
4. Context Budget
The Context Manager needs to allocate token budgets to different types of content.
For example:
| Content | Example budget share |
|---|---|
| System instructions and constraints | 15% |
| User goals | 10% |
| Current task state | 15% |
| Historical summaries | 15% |
| Tool definitions | 15% |
| Materials relevant to the current task | 25% |
| Reserved space | 5% |
This is only an example allocation, not a fixed ratio. The actual budget needs to be adjusted dynamically based on the model's context window, the number of tools, and the task type.
IX. Agent Loop: The Core Execution Mechanism
The Agent Loop is the most core execution mechanism of an Agent.
It organizes model reasoning, tool execution, and environmental feedback into a continuous loop.
1. Standard Agent Loop
It can usually be abstracted into five stages:
| Stage | English | Core responsibility |
|---|---|---|
| Perceive | Observe | Obtain current environment information |
| Think | Think | Analyze goals and current state |
| Plan | Plan | Decide the next action |
| Act | Act | Call tools to perform operations |
| Evaluate | Evaluate | Determine whether the execution results meet the goals |
Complete execution flow:
```mermaid flowchart TD A["Observe: Get environment information"] --> B["Think: Analyze current state"] B --> C["Plan: Formulate an action plan"] C --> D["Act: Execute tool operations"] D --> E["Evaluate: Evaluate execution results"] E --> F{"Is the goal complete?"} F -->|"No"| A F -->|"Yes"| G["Return the final result"] style A fill:#E0F0FE,stroke:#93BCE8,color:#1E4976 style B fill:#EAE8FA,stroke:#A49BD5,color:#51477D style C fill:#EAE8FA,stroke:#A49BD5,color:#51477D style D fill:#EAF3EC,stroke:#83A995,color:#315D49 style E fill:#E0F0FE,stroke:#93BCE8,color:#1E4976 style F fill:#F7F4E8,stroke:#C7B77A,color:#675A2D style G fill:#DCEFE5,stroke:#4F8B70,color:#24563F ```2. Observe: Perceiving the Environment
Observe is responsible for obtaining information relevant to the current task.
Sources of information include:
- User input.
- Conversation history.
- Current task state.
- Tool execution results.
- External API responses.
- File and database data.
- Environment runtime logs.
The core question of Observe is: what facts does the Agent currently have?
Pay special attention to the fact that information returned by the external environment is not necessarily trustworthy. Content such as web pages, emails, and documents must be treated as untrusted data and must not directly override system instructions and security policies.
3. Think: Analyzing the Current State
The Think stage uses the LLM to analyze the current context.
Main judgments:
- What the user really wants to accomplish.
- What state the current task is in.
- What information has already been obtained.
- What is still missing.
- Whether there are execution risks.
- Whether a tool needs to be called in the next step.
The focus of Think is not to generate lengthy explanations, but to form judgments that can be used for decision-making.
4. Plan: Formulating an Action Plan
The Plan stage decides what operation to execute next.
Possible outcomes include:
- Calling a certain tool.
- Splitting off new subtasks.
- Adjusting an existing plan.
- Requesting additional information from the user.
- Waiting for an external task to complete.
- Ending the current task.
Simple tasks can directly generate the next action; complex tasks require maintaining an explicit task plan.
5. Act: Executing Actions
The Act stage is carried out by the Runtime and the Tool Executor.
Possible executions: - Querying a database. - Reading files. - Modifying files. - Calling an API. - Executing code. - Sending messages. - Calling other Agents.
Act is not the model generating a piece of text, but the system actually performing external operations.
6. Evaluate: Evaluating Results
The Evaluate stage needs to determine whether the action was effective.
For example:
- Did the tool execute successfully?
- Was the expected data obtained?
- Are the completion conditions of the current subtask satisfied?
- Does it need to be re-planned?
- Has the overall goal been completed?
Evaluation can combine: - Rule validation. - Unit tests. - Data consistency checks. - Independent model evaluation. - Human review.
For important tasks, deterministic verification methods should be used wherever possible, rather than relying entirely on the LLM's own judgment.
X. The Complete Operating Mechanism: From User Request to Task Completion
Combining the components described above yields a relatively complete Agent execution process.
Take an automated code-fixing Agent as an example.
1. Task Startup
The user submits a task:
Check the project for code issues, fix serious bugs, and make sure the tests pass.
The Runtime creates the task and initializes: - Task ID. - User goal. - Execution constraints. - Initial state. - Tool permissions. - Maximum execution budget.
2. Goal Parsing and Planning
The Goal Manager parses the goal.
The Planner breaks the task down into:
- Scan the project.
- Analyze the code.
- Locate the issues.
- Formulate a fix plan.
- Modify the code.
- Run the tests.
- Verify the results.
- Output a report.
3. Context Construction
The Context Manager loads: - System instructions. - User goal. - Current task plan. - Current state. - Relevant historical information. - Available tool definitions.
It then constructs the model request.
4. Model Decision-Making
The LLM determines that it currently needs to read project files and returns a tool call request.
The Runtime parses the request and hands it to the Tool Executor.
5. Tool Execution
The Tool Executor checks the tool, parameters, and permissions, performs the file read operation, and returns the result.
6. State Update
The State Manager updates the current task state and saves the completed steps and tool execution results.
7. Feedback and Re-Decision
The Runtime adds the tool result to the context and calls the model again.
Based on the new information, the model decides whether to continue analyzing, modify code, or run tests.
8. Result Verification and Completion
The Result Validator checks the test results and the task completion conditions.
If the requirements are met, the task ends; if the goal has not been achieved, processing continues or the failure recovery process is entered.