Back to series
· 11 min read

Chain of Thought and Reasoning Prompts: From Chain-of-Thought Mechanisms to Reasoning Optimization for Complex Tasks


1. What Is Chain of Thought?

Chain of Thought, abbreviated as CoT, is commonly referred to in Chinese as "思维链".

It is a prompting strategy that guides a large language model to generate or use intermediate reasoning steps, decomposing a complex problem into multiple interrelated subproblems and ultimately arriving at an answer.

Traditional prompting usually asks the model to give the result directly:

Question:
A product has an original price of 200 yuan. What is its price after a 20% discount?

Please give the answer.

The model needs to generate the final answer directly:

160 yuan.

After using CoT, the model can be guided to organize the solution process according to intermediate steps:

Question:
A product has an original price of 200 yuan. What is its price after a 20% discount?

Please analyze the calculation process step by step, then give the answer.

The model might generate:

Step 1: Understand what a 20% discount means.
A 20% discount means paying 80% of the original price.

Step 2: Calculate the discounted price.
200 × 0.8 = 160.

Final answer:
160 yuan.

The difference between the two approaches is that CoT not only asks the model to give an answer, but also guides it to explicitly organize intermediate reasoning steps.

1.1 Why Are Reasoning Prompts Needed?

Large language models can often generate answers directly for many tasks, but complex tasks usually require multiple consecutive judgment and computation steps.

For example:

  • Multi-step mathematical calculations.
  • Complex logical judgments.
  • Multi-condition rule matching.
  • Code problem analysis.
  • Synthesis of information across multiple documents.
  • Decomposition of complex problems.
  • Multi-stage planning.

If the final answer is generated directly, the model may make an error at some intermediate step without sufficiently unfolding the process for later inspection.

The value of CoT lies in breaking complex tasks into smaller reasoning steps, enabling the model to use the intermediate result from the previous step to continue processing subsequent problems.

1.2 Differences Between Standard Prompting and CoT

Comparison Dimension Standard Prompting Chain of Thought
Output Goal Give the result directly Give the result after organizing intermediate reasoning
Intermediate Steps Usually not explicitly required Usually explicitly generated
Applicable Tasks Simple Q&A, information extraction Multi-step reasoning, complex computation
Token Consumption Usually lower Usually higher
Reasoning Process Not required to be shown Can be required to be shown
Result Verification Mainly check the final result Can check intermediate steps and the final result

Note that CoT does not mean all problems will get more accurate answers.

Using CoT for simple tasks may add unnecessary output and latency; complex tasks may also reach incorrect conclusions because of erroneous intermediate steps.

2. The Underlying Mechanism of Chain of Thought

To understand CoT, we need to start with the generation mechanism of autoregressive language models.

2.1 How Do Large Language Models Generate Content?

Large language models usually generate Tokens in an autoregressive manner.

Given an input context, the model predicts the next Token based on the Tokens that have already appeared:

P(x_1,x_2,\ldots,x_T) = \prod_{t=1}^{T}P(x_t|x_{<t})

Where:

  • x_t: The Token currently being generated.
  • x_{<t}: The context before the current Token.
  • P(x_t|x_{<t}): The conditional probability of predicting the next Token given the existing context.

With standard prompting, the model generates the answer directly from the question context.

With CoT prompting, the model may first generate intermediate reasoning text, then use that text to continue generating subsequent steps and the final answer.

2.2 How Does CoT Affect the Generation Process?

Suppose a problem requires three steps to solve:

  • Identify the known conditions.
  • Perform intermediate calculations.
  • Obtain the final result.

Standard prompting:

Question context → Final answer

CoT prompting:

flowchart TD
    A["Question and known conditions"]
    B["Generate the first reasoning step"]
    C["Generate the second reasoning step"]
    D["Generate the third reasoning step"]
    E["Integrate intermediate results"]
    F["Generate the final answer"]

    A --> B
    B --> C
    C --> D
    D --> E
    E --> F

    style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style B fill:#EAF3EC,stroke:#83A995,color:#315D49
    style C fill:#EAF3EC,stroke:#83A995,color:#315D49
    style D fill:#EAF3EC,stroke:#83A995,color:#315D49
    style E fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style F fill:#DCEFE5,stroke:#4F8B70,color:#24563F

The arrows here represent contextual dependencies in the generation process; this does not mean that the model necessarily has three independent reasoning modules internally.

In practice, a Transformer performs computation at every generation step based on the current context.

2.3 Why might intermediate reasoning text improve task performance?

An important mechanism of CoT is to make intermediate results part of the context for subsequent generation.

For example:

Problem:
A warehouse has 120 items. 30 were sold in the morning,
and twice as many were sold in the afternoon as in the morning. How many are left?

If the model generates the answer directly, it must simultaneously identify the quantitative relationships, perform the multiplication, and perform the subtraction.

With CoT, the reasoning can be organized as:

Sold in the morning: 30 items.

Sold in the afternoon:
30 × 2 = 60 items.

Total sold:
30 + 60 = 90 items.

Remaining:
120 - 90 = 30 items.

The earlier computation results enter the context for subsequent generation, and the model can continue to use these intermediate results to complete the task.

However, this also introduces an important risk:

Once an intermediate step goes wrong, the error may keep propagating through the subsequent generation process.

CoT does not automatically verify the correctness of each step.

2.4 The relationship between CoT and model parameters

CoT is a prompting strategy at inference time; it is not the same as model training.

Conventional CoT prompting does not directly modify model weights.

It mainly changes:

  • The input context.
  • How the task is executed.
  • The form of intermediate outputs.
  • The conditional probability distribution of subsequent tokens.

That said, whether a model has strong multi-step reasoning ability also depends on factors such as pretraining, instruction tuning, preference alignment, and reasoning training.

Simply adding “please think step by step” to the prompt cannot conjure reasoning ability that the model did not originally have.

III. Basic CoT prompting methods

CoT can guide a model to organize its reasoning process in different ways.

3.1 Zero-shot CoT

Zero-shot CoT provides no reasoning examples and uses only a prompt phrase to guide the model to generate intermediate reasoning steps.

Classic prompts include:

Please think step by step, then give the answer.

Common English phrasing:

Let's think step by step.

For example:

Problem:

A team has 8 developers,
and each completes 5 tasks per day.
If they work for 6 days, how many tasks are completed in total?

Please analyze the calculation process step by step, then give the final answer.

The model might generate:

Number of tasks completed per day:
8 × 5 = 40

Number of tasks completed in 6 days:
40 × 6 = 240

Final answer:
240 tasks.

Characteristics of Zero-shot CoT:

  • No reasoning examples need to be prepared.
  • Only a reasoning instruction needs to be added.
  • Suitable for models that already have some reasoning ability.
  • Prompt cost is relatively low.
  • The quality of the reasoning process depends on the model's own ability.

3.2 Few-shot CoT

Few-shot CoT guides the model to learn a similar way of organizing reasoning by providing examples that include intermediate reasoning processes.

For example:

Example 1:

Problem:
Xiaoming has 10 apples and gives 3 to a friend. How many are left?

Reasoning:
The initial count is 10.
3 are given away.
The remaining count is 10 - 3 = 7.

Answer:
7.

Example 2:

Problem:
An item is priced at 100 yuan. What is the price after a 20% discount?

Reasoning:
A 20% discount means paying 80% of the original price.
100 × 0.8 = 80.

Answer:
80 yuan.

Now answer:

Problem:
An item is priced at 300 yuan. What is the price after a 30% discount?

The model can refer to the following in the examples:

  • How the problem is analyzed.
  • How the intermediate steps are organized.
  • The calculation process.
  • The format of the final answer.

Compared with Zero-shot CoT, Few-shot CoT provides not only a reasoning instruction but also concrete examples of reasoning patterns.

3.3 Zero-shot CoT vs. Few-shot CoT

Dimension Zero-shot CoT Few-shot CoT
Reasoning examples Not provided Provided
Main guidance method Natural-language instruction Instruction plus reasoning examples
Prompt complexity Relatively low Relatively high
Example preparation cost None Required
Token consumption Usually lower Usually higher
Applicable scenarios General reasoning tasks Specialized reasoning patterns or business rules

IV. Typical CoT reasoning strategies

CoT is an important class of reasoning prompting methods, but not every reasoning strategy is equivalent to CoT.

A variety of related strategies have taken shape around solving complex tasks.

4.1 Chain of Thought: linear chain of thought

The most basic form of CoT generates intermediate steps in sequence.

For example:

Problem:
An item is priced at 500 yuan. Its price is first raised by 20%, then lowered by 20%.
What is the final price?

Reasoning:

Step 1: Calculate the price after the increase.

500 × 1.2 = 600 yuan.

Step 2: Calculate the price after the decrease.

600 × 0.8 = 480 yuan.

Final answer:

480 yuan.

Note here that raising the price by 20% and then lowering it by 20% does not restore the original price.

The reason is that the base for the second calculation has already changed.

The value of CoT lies in breaking down the computation process, avoiding the direct application of incorrect intuition.

4.2 Least-to-Most Prompting: Easy-to-Hard Prompting

The core idea of Least-to-Most Prompting is:

First, decompose a complex problem into a set of simpler subproblems, then solve them one by one in order from easy to hard.

Its difference from ordinary linear CoT is that it places more emphasis on decomposing subproblems and their dependencies.

For example:

Problem:

A company has 3 departments.
Each department has 4 teams, and each team has 5 employees.
Each employee completes 8 tasks per week.
How many tasks does the entire company complete in one week?

It can be decomposed into:

Subproblem 1:
How many teams are there in the company in total?

3 × 4 = 12

Subproblem 2:
How many employees are there in the company in total?

12 × 5 = 60

Subproblem 3:
How many tasks does the company complete in one week?

60 × 8 = 480

Execution flow:

flowchart TD
    A["Receive complex problem"]
    B["Identify problem components"]
    C["Decompose into subproblems"]
    D["Determine subproblem dependencies"]
    E["Solve base subproblems"]
    F["Use intermediate results to solve subsequent problems"]
    G["Integrate all results"]
    H["Output final answer"]

    A --> B
    B --> C
    C --> D
    D --> E
    E --> F
    F --> G
    G --> H

    style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style B fill:#EAF3EC,stroke:#83A995,color:#315D49
    style C fill:#EAF3EC,stroke:#83A995,color:#315D49
    style D fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style E fill:#EAF3EC,stroke:#83A995,color:#315D49
    style F fill:#EAF3EC,stroke:#83A995,color:#315D49
    style G fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style H fill:#DCEFE5,stroke:#4F8B70,color:#24563F

Suitable for:

  • Multi-step calculations.
  • Complex rule analysis.
  • Multi-stage task planning.
  • Problems with clear dependencies.

4.3 Self-Consistency: Self-Consistency Reasoning

Self-Consistency is a strategy that generates multiple reasoning paths and then aggregates or selects among the results.

Its core idea is:

The same problem can be solved through different reasoning paths. If multiple paths reach the same answer, that answer may be more reliable.

For example:

Problem:
An item has an original price of 200 yuan. It is first discounted by 20%,
then reduced by 20 yuan.
What is the final price?

The model can be asked to independently generate multiple reasoning paths.

Path 1:
200 × 0.8 = 160
160 - 20 = 140

Answer: 140 yuan
Path 2:
First calculate the 20%-discounted price, 160 yuan,
then subtract 20 yuan.

Answer: 140 yuan
Path 3:
80% of the original price is 160 yuan,
then subtract 20 yuan after the discount.

Answer: 140 yuan

Count the results from multiple paths:

140 yuan: 3 times

Finally, select the answer that appears more frequently.

Execution process:

flowchart TD
    A["Input the same problem"]
    B["Generate multiple independent reasoning paths"]
    C["Extract final answer from each path"]
    D["Count answer distribution"]
    E["Select aggregated result"]
    F["Output final answer"]

    A --> B
    B --> C
    C --> D
    D --> E
    E --> F

    style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style B fill:#EAF3EC,stroke:#83A995,color:#315D49
    style C fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style D fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style E fill:#EAF3EC,stroke:#83A995,color:#315D49
    style F fill:#DCEFE5,stroke:#4F8B70,color:#24563F

The key to Self-Consistency is that the multiple reasoning paths have a certain degree of independence.

If all paths are affected by the same incorrect premise, even a high degree of answer agreement does not mean the answer is correct.

In addition, multiple sampling increases token consumption, computational cost, and response latency.

4.4 Tree of Thoughts: Tree of Thoughts

Tree of Thoughts, abbreviated as ToT, is a search strategy that expands the reasoning process from a single path into multiple candidate paths.

Ordinary CoT is closer to linear reasoning:

Step A → Step B → Step C → Final answer

ToT, by contrast, allows generating multiple candidate thoughts at a stage, evaluating them, and then choosing the direction to continue exploring.

For example, when designing a system architecture solution, you can first generate several candidate solutions:

  • Monolithic architecture.
  • Microservices architecture.
  • Modular monolith architecture.

Then evaluate them based on business scale, team capabilities, operations and maintenance costs, and scaling requirements.

The abstraction process is as follows:

flowchart TD
    A["Complex task"]
    B["Generate candidate thoughts"]
    C["Candidate solution A"]
    D["Candidate solution B"]
    E["Candidate solution C"]
    F["Evaluate candidate solutions"]
    G{"Choose the path to continue exploring"}
    H["Expand the next layer of reasoning"]
    I["Output final solution"]

    A --> B
    B --> C
    B --> D
    B --> E
    C --> F
    D --> F
    E --> F
    F --> G
    G -->|"Continue search"| H
    H --> F
    G -->|"Complete search"| I

    style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style B fill:#EAF3EC,stroke:#83A995,color:#315D49
    style C fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style D fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style E fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style F fill:#EAF3EC,stroke:#83A995,color:#315D49
    style G fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
    style H fill:#EAF3EC,stroke:#83A995,color:#315D49
    style I fill:#DCEFE5,stroke:#4F8B70,color:#24563F

ToT usually requires additional mechanisms for candidate generation, state evaluation, and search control.

It does not simply require the model to write more paragraphs of reasoning.

Suitable for:

  • Open-ended planning.
  • Comparison of multiple solutions.
  • Complex decision-making.
  • Problems that require exploring multiple candidate paths.

4.5 ReAct: Reasoning + Acting

ReAct combines Reasoning and Acting.

The model not only generates intermediate analysis, but can also issue tool-call requests as the task requires, obtaining information or performing actions through external tools.

For example, when a user asks about some real-time data:

  1. The model identifies that the question requires real-time information.
  2. The model generates a query tool-call request.
  3. The application executes the query.
  4. The tool returns the result.
  5. The model continues to compose the answer based on the result.

Typical structure:

Thought:
The current question requires obtaining external real-time data.

Action:
Call the query tool.

Observation:
The tool returns the query result.

Thought:
Analyze based on the returned result.

Final:
Compose the final answer.

Note that the Action in ReAct is not the model directly executing an external program.

After the model generates a tool-call request, the application still needs to parse, validate, and execute it.

Execution flow:

flowchart TD
    A["User asks a question"]
    B["Model analyzes the task"]
    C{"Is an external tool needed?"}
    D["Generate tool-call request"]
    E["Application validates parameters and permissions"]
    F["Execute external tool"]
    G["Return Observation"]
    H["Model continues analysis"]
    I["Generate final answer"]

    A --> B
    B --> C
    C -->|"Needed"| D
    D --> E
    E --> F
    F --> G
    G --> H
    H --> C
    C -->|"Not needed"| I

    style A fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style B fill:#EAF3EC,stroke:#83A995,color:#315D49
    style C fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
    style D fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style E fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
    style F fill:#EAF3EC,stroke:#83A995,color:#315D49
    style G fill:#F5F8F4,stroke:#B5CBBB,color:#40594A
    style H fill:#EAF3EC,stroke:#83A995,color:#315D49
    style I fill:#DCEFE5,stroke:#4F8B70,color:#24563F

ReAct is suitable for:

  • Real-time information queries.
  • Database retrieval.
  • API tool calls.
  • Multi-step business processing.
  • Tasks that require external feedback.

4.6 Differences Among Several Reasoning Strategies

Strategy Core Idea Key Features Use Cases
CoT Unfold the intermediate reasoning steps Single reasoning chain Multi-step problems
Zero-shot CoT Guide reasoning through instructions No examples required General reasoning
Few-shot CoT Provide examples of the reasoning process Learn the task pattern Specific reasoning formats
Least-to-Most Decompose first, then solve step by step Progressive sub-problems Complex compositional problems
Self-Consistency Multi-path sampling and aggregation Multiple reasoning runs Math and logic tasks
Tree of Thoughts Search multiple candidate paths Branch exploration and evaluation Planning and decision-making
ReAct Combine reasoning with tool actions External feedback loop Tool-based tasks

These strategies can be used in combination; they are not mutually exclusive technical systems.

V. How to Design Reasoning Prompts

A reasoning prompt is not simply a matter of adding a line like "please think carefully" to the Prompt.

To help the model handle complex tasks better, you need to clarify the task structure, the reasoning goal, and the constraints.

5.1 Define the Reasoning Goal

Not recommended:

Please think deeply about this problem.

This kind of instruction is too broad, and the model has no idea what analysis method it should use.

Recommended:

Please analyze the following technical problem.

1. Identify the known conditions in the problem.
2. Find the key constraints.
3. Analyze possible causes.
4. Compare different solutions.
5. Give a final recommendation.

If necessary information is missing, state this explicitly.

Explicit steps give the model a more concrete task structure.

5.2 Split Complex Tasks into Sub-problems

For example, a user asks:

Why does the response time of a Java service suddenly increase under high concurrency?

This is a complex problem that may involve:

  • CPU usage.
  • Memory and GC.
  • Thread pools.
  • Database connection pools.
  • Network latency.
  • Lock contention.
  • Downstream services.

You can ask the model to analyze each of these dimensions separately, instead of directly giving a single cause.

5.3 Provide Necessary Constraints

The reasoning process needs to follow explicit business and technical constraints.

For example:

Analysis requirements:

- Do not assume system configurations that were not provided.
- Distinguish known facts from speculation.
- Do not interpret correlation directly as causation.
- For each candidate cause, provide a verification method.
- If information is insufficient, list the data that needs to be supplemented.

Constraints can reduce baseless inference by the model when information is insufficient.

5.4 Require Structured Output

For complex analyses, you can divide the result into different sections:

Please output according to the following structure:

I. Understanding of the problem
II. Known conditions
III. Key analysis
IV. Candidate causes
V. Verification methods
VI. Final recommendation

This is easier to evaluate and use than asking the model to output a large amount of free-form reasoning text.

5.5 Use Verification Steps

For calculation and logic tasks, you can ask the model to check key results before producing output.

For example:

After completing the calculation, please check:

1. Whether all known conditions were correctly understood.
2. Whether there are any unit conversion errors.
3. Whether the intermediate calculations are consistent.
4. Whether the final result satisfies the problem's constraints.

But make it clear: having the model check itself is not the same as independent verification.