Back to series
· 10 min read

How Does a CPU Execute an Instruction?


I. The Essence of CPU Instruction Execution

The core task of the CPU (Central Processing Unit) is to execute machine instructions.

High-level language code written by programmers, such as C, Java, and Python, must ultimately be converted through compilation, interpretation, or runtime execution mechanisms into operations that the processor can execute.

Take C as an example:

int a = 10;
int b = 20;
int c = a + b;

This code expresses three basic operations:

  1. Define variable a.
  2. Define variable b.
  3. Add a and b, and store the result in c.

But the CPU does not directly understand int, +, or variable names.

At the machine level, these operations are represented as a series of machine instructions.

For example, a hypothetical instruction sequence:

MOV R1, #10
MOV R2, #20
ADD R3, R1, R2
STORE [ADDR], R3

Where:

  • MOV: data transfer.
  • ADD: integer addition.
  • STORE: write data to memory.
  • R1, R2, R3: general-purpose registers.
  • #10, #20: immediate values.
  • [ADDR]: memory address operand.

The above is pseudo-assembly for ease of understanding and does not correspond to the complete instruction encoding of any real processor.

The process of the CPU executing instructions is essentially:

Fetch the instruction, decode the instruction, read the operands, perform the operation, and save the result.

This process is commonly called the instruction cycle (Instruction Cycle).

II. The Basic Stages of Instruction Execution

Traditional computer organization principles usually divide the execution of an instruction into several stages:

  1. Instruction fetch (Fetch).
  2. Decode (Decode).
  3. Execute (Execute).
  4. Memory access (Memory Access).
  5. Write back (Write Back).

Different instructions do not necessarily go through all stages. For example, integer addition between registers typically does not require accessing data memory, whereas load instructions require memory access.

2.1 Instruction Execution Flow

flowchart TD
    A["Instruction Fetch (Fetch)"] --> B["Decode (Decode)"]
    B --> C["Read operands"]
    C --> D["Execute (Execute)"]
    D --> E{"Is memory access needed?"}
    E -->|Yes| F["Memory Access (Memory Access)"]
    E -->|No| G["Write Back (Write Back)"]
    F --> G
    G --> H["Update processor state"]
    H --> I["Continue to execute subsequent instructions"]

    classDef stage fill:#E8F1FF,stroke:#3973C6,color:#17365D
    classDef decision fill:#FFF4E5,stroke:#D99A35,color:#704510
    classDef result fill:#EAF7F0,stroke:#39966B,color:#174D35

    class A,B,C,D,F,G stage
    class E decision
    class H,I result

This is the traditional abstract model of instruction execution. Modern CPUs usually further split instructions into micro-operations and process multiple instructions in parallel through mechanisms such as pipelining and out-of-order execution.

III. The First Stage: Instruction Fetch (Fetch)

Instruction fetch is the first stage of CPU instruction execution.

The CPU needs to know where the next instruction to be executed is located.

This location is recorded by the program counter (Program Counter, PC).

3.1 What Is the PC?

The PC is a register in the CPU used to hold the address of the next instruction to be fetched.

In different processor architectures, the specific semantics and update timing of the PC may differ.

For example:

PC = 0x1000

This indicates that the processor currently needs to fetch an instruction from the corresponding address.

The basic process of the fetch stage:

  1. Determine the instruction address based on the PC.
  2. Fetch the instruction from the instruction cache or a lower storage level.
  3. Send the instruction to subsequent processing logic.
  4. Update the fetch address, usually to continue fetching the next sequential instruction.

3.2 How Does the PC Change?

Assume the program instructions are located at consecutive addresses:

Instruction Address Instruction
0x1000 MOV R1, #10
0x1004 MOV R2, #20
0x1008 ADD R3, R1, R2
0x100C STORE [ADDR], R3

If each instruction has a fixed length of 4 bytes, then during sequential execution, the PC typically increments by 4.

flowchart TD
    A["PC = 0x1000"] --> B["Fetch MOV R1, #10"]
    B --> C["PC = 0x1004"]
    C --> D["Fetch MOV R2, #20"]
    D --> E["PC = 0x1008"]
    E --> F["Fetch ADD R3, R1, R2"]
    F --> G["PC = 0x100C"]
    G --> H["Fetch STORE instruction"]

    classDef address fill:#E8F1FF,stroke:#3973C6,color:#17365D
    classDef fetch fill:#EAF7F0,stroke:#39966B,color:#174D35

    class A,C,E,G address
    class B,D,F,H fetch

Note:

Not all CPUs have 4-byte instructions.

For example:

  • x86/x86-64 instruction lengths are variable.
  • ARM A32 instructions are usually 4 bytes.
  • ARM Thumb instructions can use 16-bit or 32-bit encodings.
  • AArch64 instructions are usually 4 bytes.

Therefore, how the PC is updated depends on the specific instruction architecture.

3.3 Where are instructions read from?

CPUs do not always access main memory directly to fetch instructions every time.

Modern CPUs usually have an instruction cache (Instruction Cache, I-Cache).

The instruction fetch logic prioritizes fetching instructions from the I-Cache.

If the I-Cache misses, the corresponding data must be fetched from a lower-level cache or main memory.

flowchart TD
    A["PC provides the instruction address"] --> B["Query L1 I-Cache"]
    B --> C{"Cache hit?"}
    C -->|Yes| D["Fetch instruction"]
    C -->|No| E["Query lower-level cache"]
    E --> F{"Hit?"}
    F -->|Yes| G["Fetch instruction cache line"]
    F -->|No| H["Access main memory"]
    H --> I["Load instruction cache line"]
    G --> D
    I --> D
    D --> J["Send to instruction processing front end"]

    classDef access fill:#E8F1FF,stroke:#3973C6,color:#17365D
    classDef decision fill:#FFF4E5,stroke:#D99A35,color:#704510
    classDef result fill:#EAF7F0,stroke:#39966B,color:#174D35

    class A,B,E,H,I access
    class C,F decision
    class D,G,J result

3.4 Instruction Prefetch

To reduce waiting, modern CPUs also use prefetching mechanisms.

The prefetcher attempts to fetch instruction data that may be executed soon in advance, so that instructions are already in the cache or front-end buffer structures when they are actually needed.

Prefetching is usually based on program execution patterns, such as sequential instruction fetching and branch prediction results.

Prefetching is not guaranteed to be accurate every time. If prefetched data is not used, it may waste cache space and memory bandwidth.

4. Stage 2: Decode

The fetch stage obtains the machine instruction encoding.

The CPU also needs to parse these encodings to determine what operation to perform.

This process is called instruction decoding.

4.1 What is a machine instruction?

A machine instruction is a binary encoding defined by the processor's instruction set architecture (ISA).

For example, an instruction may represent:

ADD R3, R1, R2

The semantics it expresses are:

Add the values of registers R1 and R2, and write the result to R3.

But a real machine instruction does not directly store the string ADD R3, R1, R2; instead, it uses binary bits to represent information such as the opcode, register numbers, and addressing modes.

4.2 What information does an instruction usually contain?

A machine instruction may contain the following information:

Field Purpose
Opcode Opcode, indicating what operation to perform
Source Operand Source operand
Destination Operand Destination operand
Addressing Mode Addressing mode
Immediate Immediate value
Condition Information related to conditional execution or condition evaluation

The specific fields are determined by the ISA and instruction format; not all instructions contain all fields.

4.3 What does the decoder do?

The decoder identifies the instruction type and operand information based on the machine instruction encoding, and generates the control information required for subsequent execution.

For example:

ADD R3, R1, R2

After decoding, the processor needs to determine:

  • Operation type: integer addition.
  • Source operands: R1, R2.
  • Destination register: R3.
  • Execution resource: integer arithmetic unit.
  • Result handling: write the addition result to the destination register.
flowchart TD
    A["Machine instruction encoding"] --> B["Instruction decoder"]
    B --> C["Identify opcode"]
    B --> D["Parse source operand"]
    B --> E["Parse destination operand"]
    B --> F["Parse addressing mode"]
    C --> G["Generate control information"]
    D --> G
    E --> G
    F --> G
    G --> H["Enter subsequent execution flow"]

    classDef input fill:#E8F1FF,stroke:#3973C6,color:#17365D
    classDef decode fill:#FFF4E5,stroke:#D99A35,color:#704510
    classDef output fill:#EAF7F0,stroke:#39966B,color:#174D35

    class A input
    class B,C,D,E,F,G decode
    class H output

4.4 x86 Instructions and Micro-operations (μops)

Modern x86 CPUs have complex instruction sets, and some instructions may contain multiple operations.

To improve execution efficiency, many processors decode complex x86 instructions into simpler micro-operations (μops).

For example, a complex instruction with a memory operand may internally be split into multiple micro-operations such as load, arithmetic, and store.

flowchart TD
    A["x86 machine instruction"] --> B["Instruction decoding"]
    B --> C["Generate one or more μops"]
    C --> D["Scheduling and execution"]
    D --> E["Execution unit"]
    E --> F["Result handling"]

    classDef instruction fill:#E8F1FF,stroke:#3973C6,color:#17365D
    classDef decode fill:#FFF4E5,stroke:#D99A35,color:#704510
    classDef execute fill:#EAF7F0,stroke:#39966B,color:#174D35

    class A instruction
    class B,C decode
    class D,E,F execute

Note the following:

  • Not all x86 instructions are split into multiple μops.
  • Some simple instructions can correspond directly to a single micro-operation.
  • Some processors also support micro-op fusion.
  • Different CPU microarchitectures may implement decoding and micro-operations differently.

The ISA defines the software-visible semantics of instructions, while μops are part of the internal implementation of a specific processor microarchitecture.

V. Third Stage: Reading Operands

After decoding is complete, the CPU needs to obtain the data required to execute the instruction.

Operands may come from:

  • Registers.
  • Immediate values in the instruction.
  • Memory.

5.1 Register Operands

Registers are high-speed storage units inside the CPU.

For example:

ADD R3, R1, R2

The CPU needs to read the values of R1 and R2, then perform the addition.

flowchart TD
    A["Read instruction ADD R3, R1, R2"] --> B["Read register R1"]
    A --> C["Read register R2"]
    B --> D["Obtain operand A"]
    C --> E["Obtain operand B"]
    D --> F["Send to integer execution unit"]
    E --> F

    classDef source fill:#E8F1FF,stroke:#3973C6,color:#17365D
    classDef result fill:#EAF7F0,stroke:#39966B,color:#174D35

    class A,B,C source
    class D,E,F result

Register access is usually much faster than accessing main memory, so compilers use registers as effectively as possible.

5.2 Immediate Operands

An immediate is a constant encoded directly in the instruction.

For example:

MOV R1, #10

Here, 10 is the immediate value.

The CPU does not need to access memory separately to read this constant; it obtains it by decoding the instruction encoding.

5.3 Memory Operands

Some instructions need to access memory.

For example:

LOAD R1, [0x1000]

This means reading data from the specified memory address into R1.

A real processor may calculate the effective address through an Address Generation Unit (AGU), and then complete the memory access through a Load/Store Unit (LSU).

flowchart TD
    A["Memory Access Instruction"] --> B["Parse Address Expression"]
    B --> C["Address Generation Unit (AGU)"]
    C --> D["Calculate Effective Address"]
    D --> E["Access Data Cache"]
    E --> F{"L1 D-Cache Hit?"}
    F -->|Yes| G["Read Data"]
    F -->|No| H["Access Lower-Level Cache or Main Memory"]
    H --> G
    G --> I["Send Data into Execution Flow"]

    classDef address fill:#E8F1FF,stroke:#3973C6,color:#17365D
    classDef decision fill:#FFF4E5,stroke:#D99A35,color:#704510
    classDef result fill:#EAF7F0,stroke:#39966B,color:#174D35

    class A,B,C,D,E,H address
    class F decision
    class G,I result

Note:

A memory operand does not mean the CPU accesses main memory every time. The data may already be in L1, L2, or L3 cache.

6. Fourth Stage: Execute

The Execute stage is the stage where the instruction operation is actually completed.

The CPU selects the corresponding execution unit based on the instruction type.

Modern CPUs usually have multiple execution units with different functions.

6.1 Integer Arithmetic Unit (ALU)

The ALU (Arithmetic Logic Unit) is responsible for integer arithmetic and logic operations.

Common operations include:

  • Addition.
  • Subtraction.
  • AND operation.
  • OR operation.
  • XOR operation.
  • Comparison.
  • Shift.

For example:

ADD R3, R1, R2

The execution logic can be abstracted as:

R3=R1+R2

6.2 Floating-Point Unit (FPU)

The FPU (Floating Point Unit) is used to perform floating-point operations.

For example:

float a = 1.5f;
float b = 2.5f;
float c = a * b;

Floating-point operations usually follow floating-point standards such as IEEE 754, but the specific instruction support and execution details depend on the processor architecture.

6.3 SIMD Execution Unit

SIMD (Single Instruction Multiple Data) allows one instruction to perform the same type of operation on multiple data elements.

For example:

A = [1, 2, 3, 4]
B = [5, 6, 7, 8]

A + B = [6, 8, 10, 12]

On processors that support the corresponding SIMD instructions, vector registers and vector execution units can be used to process multiple data elements in parallel.

Common technologies include:

  • x86 SSE.
  • x86 AVX.
  • ARM NEON.
  • ARM SVE.

6.4 Branch Execution Unit

Branch instructions change the program execution order.

For example:

if (a > b) {
    c = a;
} else {
    c = b;
}

The CPU needs to determine whether the condition is true and decide which instruction should be executed next.

Modern processors usually use branch prediction mechanisms to predict the branch direction in advance and reduce pipeline stalls.

6.5 Relationship Between Different Execution Units

flowchart TD
    A["Instruction Scheduler"] --> B["Integer ALU"]
    A --> C["Floating-Point FPU"]
    A --> D["SIMD Vector Unit"]
    A --> E["Load/Store Unit (LSU)"]
    A --> F["Branch Execution Unit"]

    B --> G["Execution Result"]
    C --> G
    D --> G
    E --> G
    F --> G

    classDef dispatch fill:#E8F1FF,stroke:#3973C6,color:#17365D
    classDef unit fill:#FFF4E5,stroke:#D99A35,color:#704510
    classDef result fill:#EAF7F0,stroke:#39966B,color:#174D35

    class A dispatch
    class B,C,D,E,F unit
    class G result

Different processors have different numbers of execution ports and execution units, and instructions of the same type may also be handled by multiple execution units.

7. Fifth Stage: Memory Access

The Memory Access stage mainly deals with instructions that need to read from or write to memory.

For example:

LOAD R1, [0x1000]
STORE [0x2000], R1

The first instruction reads data from memory.

The second instruction writes register data to memory.

7.1 Load Operation

Load means reading data from memory.

LOAD R1, [ADDR]

Basic process:

  1. Calculate the effective address.
  2. Query the data cache.
  3. If the cache hits, retrieve the data.
  4. If the cache misses, request data from lower-level storage.
  5. Provide the read result to subsequent execution logic.

7.2 Store Operation

Store means writing data to memory.

STORE [ADDR], R1

The CPU needs to:

  1. Calculate the effective address.
  2. Obtain the data to be written.
  3. Perform cache writes and coherence-related operations.
  4. Handle lower-level storage updates according to the cache policy.

Modern processors usually have structures such as a Store Buffer to temporarily hold writes that have not yet completed lower-level storage handling.

Therefore, instruction completion and the final writing of data to DRAM do not necessarily occur at the same moment.

VIII. Stage 6: Write Back

The write-back stage is responsible for saving the execution result to the destination.

For example:

ADD R3, R1, R2

After execution completes, the result needs to be written to R3.

If it is a load instruction:

LOAD R1, [ADDR]

then the data read must be written to R1.

If it is a store instruction, the process of writing data into the cache or storage system is handled by the corresponding memory access mechanism, and does not necessarily go through the register write-back stage in the traditional sense.

8.1 Instruction Execution Result

flowchart TD
    A["Execution unit completes computation"] --> B["Produces result"]
    B --> C{"What is the destination?"}
    C -->|Register| D["Write to target register"]
    C -->|Memory| E["Enter Store handling flow"]
    C -->|Status flags| F["Update relevant status"]
    D --> G["Complete handling of the corresponding result"]
    E --> G
    F --> G

    classDef execute fill:#E8F1FF,stroke:#3973C6,color:#17365D
    classDef decision fill:#FFF4E5,stroke:#D99A35,color:#704510
    classDef result fill:#EAF7F0,stroke:#39966B,color:#174D35

    class A,B execute
    class C decision
    class D,E,F,G result

IX. Why Can't Modern CPUs Simply Execute One Instruction to Completion Before Executing the Next?

The preceding sections described the traditional instruction execution process.

If the CPU had to work strictly in the following way:

decode only after fetch is complete, execute only after decode is complete, and fetch the next instruction only after execution is complete, then a large amount of hardware resources might sit idle.

For example, when executing a data load instruction that must wait for memory, the CPU's other execution units may not be fully utilized.

To improve processor throughput, modern CPUs introduce techniques such as pipelining, out-of-order execution, and branch prediction.

9.1 Instruction Pipeline

The core idea of a pipeline is:

Split the instruction execution process into multiple stages so that multiple instructions can be in different stages at the same time.

It can be understood as a production line:

  • The first instruction is executing.
  • The second instruction is decoding.
  • The third instruction is being fetched.

Different instructions can occupy different pipeline stages at the same time.

flowchart TB
    subgraph C1["Cycle 1"]
        A1["Instruction 1: Fetch"]
    end

    subgraph C2["Cycle 2"]
        A2["Instruction 1: Decode"]
        B2["Instruction 2: Fetch"]
    end

    subgraph C3["Cycle 3"]
        A3["Instruction 1: Execute"]
        B3["Instruction 2: Decode"]
        C3I["Instruction 3: Fetch"]
    end

    subgraph C4["Cycle 4"]
        A4["Instruction 1: Memory access"]
        B4["Instruction 2: Execute"]
        C4I["Instruction 3: Decode"]
        D4["Instruction 4: Fetch"]
    end

    subgraph C5["Cycle 5"]
        A5["Instruction 1: Write back"]
        B5["Instruction 2: Memory access"]
        C5I["Instruction 3: Execute"]
        D5["Instruction 4: Decode"]
    end

    classDef stage fill:#E8F1FF,stroke:#3973C6,color:#17365D

    class A1,A2,B2,A3,B3,C3I,A4,B4,C4I,D4,A5,B5,C5I,D5 stage

The figure above shows an idealized five-stage pipeline.

Actual processors may use more pipeline stages and may also handle multiple instructions, micro-operations, and execution threads at the same time.

A pipeline mainly improves instruction throughput; it does not mean that the execution latency of a single instruction is necessarily reduced.

9.2 Pipeline Hazards

A pipeline cannot always run smoothly.

There are three common types of hazards:

Type English Meaning
Structural hazard Structural Hazard Multiple instructions compete for the same hardware resource
Data hazard Data Hazard There is a data dependency between instructions
Control hazard Control Hazard A branch instruction makes the subsequent execution path uncertain

Data hazards

For example:

ADD R1, R2, R3
SUB R4, R1, R5

The second instruction needs to use R1.

If the first instruction has not yet produced its result, the second instruction cannot use that value directly.

This situation is a data dependency.