Back to series
· 16 min read

How Does a CPU Work? Understanding Processors Through Instruction Execution

#Computer Fundamentals and Programming Basics

I. What Is a CPU?

CPU (Central Processing Unit) is the core computing and control component of a computer. It is responsible for executing program instructions, processing data, and coordinating the various hardware components of the computer to get work done.

At its essence, a CPU is a digital circuit system that, following the rules defined by the instruction set architecture (ISA), continuously fetches instructions, decodes them, executes operations, and updates its processing state.

Whether running an operating system, compiling a program, or executing a database query or AI inference, everything ultimately must be converted into machine instructions that the CPU can recognize and execute, or be completed cooperatively by other specialized processors.

1.1 Core Responsibilities of the CPU

The CPU mainly handles the following tasks:

Responsibility Description
Instruction control Fetch, decode, and control instruction execution
Arithmetic operations Perform arithmetic operations such as addition, subtraction, multiplication, and division
Logical operations Perform logical operations such as comparison, AND, OR, and NOT
Data processing Read, convert, and write data
Flow control Perform jumps, branches, and other operations based on conditions
System coordination Work in concert with memory, I/O devices, and other hardware

1.2 Where the CPU Sits in a Computer

The CPU does not work on its own; it must form a complete computer system together with memory, storage devices, and input/output devices.

flowchart TB
    CPU["CPU<br/>Instruction execution and control"]
    MEM["Memory<br/>Programs and runtime data"]
    IO["Input/output devices<br/>Keyboard, monitor, network card"]
    DISK["Storage devices<br/>SSD, HDD"]

    CPU <--> MEM
    CPU <--> IO
    MEM <--> DISK

    classDef cpu fill:#DCEFE3,stroke:#6DAD86,color:#285C40,stroke-width:1.5px
    classDef mem fill:#DFEAF7,stroke:#82A8D1,color:#315579,stroke-width:1.5px
    classDef io fill:#F8EEDB,stroke:#D4B77C,color:#735B2D,stroke-width:1.5px
    classDef disk fill:#EAE5F4,stroke:#A99BC8,color:#574A78,stroke-width:1.5px

    class CPU cpu
    class MEM mem
    class IO io
    class DISK disk

The CPU is responsible for executing instructions, memory provides the data and instructions needed for programs to run, storage devices are responsible for long-term data retention, and input/output devices are responsible for interacting with the external environment.

Note: in modern computers, the CPU, memory, and I/O devices are not simply connected directly to one another; they work together through components such as memory controllers, buses, and I/O controllers.

II. Internal Composition of the CPU

Internally, the CPU is made up of multiple functional units. Each unit performs its own duties, and together they complete the fetching, decoding, execution, and result commit of instructions.

From a functional perspective, the core components of the CPU can be divided into:

  • Control Unit (CU)
  • Arithmetic Logic Unit (ALU)
  • Register file (Registers)
  • Cache
  • Clock and related control logic

2.1 CPU Core Structure

flowchart TB
    subgraph CPU["CPU Processor"]
        direction TB

        CU["Control Unit CU<br/>Instruction decoding and control"]
        REG["Register file<br/>Temporarily holds instructions and data"]
        ALU["Arithmetic Logic Unit ALU<br/>Arithmetic and logical operations"]
        CACHE["Cache<br/>Reduces memory access latency"]
        CTRL["Clock and control logic<br/>Coordinates the execution process"]

        CU <--> REG
        REG <--> ALU
        CU <--> ALU
        CACHE <--> REG
        CTRL --> CU
        CTRL --> ALU
    end

    MEM["Main memory RAM"]

    CACHE <--> MEM

    classDef cu fill:#DCEFE3,stroke:#6DAD86,color:#285C40
    classDef reg fill:#DFEAF7,stroke:#82A8D1,color:#315579
    classDef alu fill:#F8EEDB,stroke:#D4B77C,color:#735B2D
    classDef cache fill:#EAE5F4,stroke:#A99BC8,color:#574A78
    classDef ctrl fill:#E5ECE9,stroke:#91AAA0,color:#3C5C50
    classDef mem fill:#DFEAF7,stroke:#82A8D1,color:#315579

    class CU cu
    class REG reg
    class ALU alu
    class CACHE cache
    class CTRL ctrl
    class MEM mem

The above is an abstract structure at the functional level and does not represent the actual internal connections of every CPU. Modern processors also contain complex structures such as instruction fetch units, execution ports, load/store units, and reorder buffers.

2.2 Control Unit (CU)

The Control Unit is responsible for coordinating the work of the various components inside the CPU.

Main responsibilities:

  • Fetch instructions from the instruction stream.
  • Decode machine instructions.
  • Identify the operands and execution type required by the instruction.
  • Generate the corresponding control signals.
  • Coordinate the flow of data among registers, execution units, and memory.

For example, when executing an addition instruction, the control unit needs to identify that this is an arithmetic operation and coordinate reading the operands, performing the addition, and saving the result.

The control unit itself is not responsible for performing the actual arithmetic operations.

2.3 Arithmetic Logic Unit (ALU)

The ALU (Arithmetic Logic Unit) is the core component of the CPU that performs arithmetic and logical operations.

Main operations include:

Operation Type Example
Arithmetic operations Addition, subtraction
Logical operations AND, OR, XOR, NOT
Comparison operations Greater than, less than, equal to
Bitwise operations Left shift, right shift

For example:

ADD R1, R2, R3

This means adding the values in registers R2 and R3 and writing the result to R1.

R1 = R2 + R3

Here, the ALU is responsible for performing the addition operation, while the control logic is responsible for coordinating the entire execution process.

Note that multiplication, division, and floating-point operations are usually performed by dedicated execution units in modern CPUs and are not necessarily all handled by the ordinary integer ALU.

2.4 Registers

Registers are extremely fast, very small storage units inside the CPU, used to temporarily hold instructions, data, addresses, and execution status.

Common registers:

Register English Main function
Program Counter PC Holds the address of the next instruction to be fetched
Instruction Register IR Holds the current instruction (conceptual model)
General-Purpose Register GPR Stores operands and results of operations
Stack Pointer SP Points to the current stack position
Status Register Flags Holds condition codes and status information

Among these, the program counter (PC) is especially important.

The CPU uses the PC to determine the address of the next instruction to fetch. When execution proceeds normally, the PC is updated to the address of the next instruction; when a jump instruction is encountered, the PC may be changed to the target address.

Modern CPUs may track multiple outstanding instructions at the same time, so the program counters and related state actually used internally are more complex than in the simple model.

2.5 Cache

The CPU's processing speed is far higher than the access speed of main memory.

If the CPU had to access main memory every time it reads data, it might wait a long time for the data to return, preventing computing resources from being fully utilized.

Cache is a high-speed buffer located between the CPU and main memory, used to store data and instructions that have been used recently or may be used again.

Common three-level cache:

Type Location Characteristics
L1 Cache Inside the CPU core Small capacity, fastest speed, usually split into instruction cache and data cache
L2 Cache Inside or near the CPU core Larger capacity, slower access than L1
L3 Cache More commonly shared among multiple cores Even larger capacity, access speed usually lower than L2

Typical data access hierarchy:

flowchart TB
    CPU["CPU execution unit"]
    L1["L1 Cache<br/>Fastest, smallest capacity"]
    L2["L2 Cache<br/>Faster"]
    L3["L3 Cache<br/>Larger capacity"]
    RAM["Main memory RAM<br/>Large capacity, high latency"]
    SSD["SSD / HDD<br/>Long-term storage"]

    CPU --> L1
    L1 --> L2
    L2 --> L3
    L3 --> RAM
    RAM --> SSD

    classDef cpu fill:#DCEFE3,stroke:#6DAD86,color:#285C40
    classDef cache fill:#DFEAF7,stroke:#82A8D1,color:#315579
    classDef memory fill:#F8EEDB,stroke:#D4B77C,color:#735B2D
    classDef storage fill:#EAE5F4,stroke:#A99BC8,color:#574A78

    class CPU cpu
    class L1,L2,L3 cache
    class RAM memory
    class SSD storage

When the CPU needs to read data, it queries the nearer cache levels first.

  • Cache Hit: The data is already present in the cache and can be read directly.
  • Cache Miss: The target data is not present in the cache and must be fetched from a lower level.

The higher the cache hit rate, the less time the CPU typically spends waiting for memory data.

The design of cache mainly relies on two important characteristics:

Temporal Locality

Data that has been accessed recently may be accessed again in the future.

For example, variables in a loop may be used repeatedly within a short period of time.

Spatial Locality

After a memory address is accessed, nearby addresses may be accessed soon afterward.

For example, sequential traversal of an array.

This is also why sequentially accessing an array usually gives better performance than randomly accessing memory.

III. How Does a CPU Execute an Instruction?

The most important thing for understanding how a CPU works is understanding the instruction execution cycle.

From the perspective of classical computer organization, the execution of an instruction can be abstracted into four stages:

  1. Fetch
  2. Decode
  3. Execute
  4. Write Back

These four stages form the classic instruction execution model.

3.1 Fetch

First, the CPU needs to know where the next instruction is.

The program counter (PC) holds the address of the next instruction to be fetched.

The fetch process:

  1. The CPU obtains the instruction address from the PC.
  2. It requests the data at the corresponding address from the instruction cache or memory.
  3. It obtains the machine instruction.
  4. It sends the instruction to the subsequent processing stages.
  5. It updates the PC so that it points to the address of the subsequent instruction (the specific update method depends on the instruction and the processor implementation).

Illustration:

PC = 0x1000

Read the instruction at memory address 0x1000

Obtained instruction:
ADD R1, R2, R3

PC updated to the address of the next instruction

Here, the address 0x1000 is only an example, indicating the instruction's location in the memory address space.

3.2 Decode

After obtaining the machine instruction, the CPU needs to understand the meaning of this instruction.

The decode stage mainly identifies:

  • Opcode: what operation needs to be performed.
  • Operand: the data involved in the operation.
  • Addressing mode: how to obtain the operands.
  • Execution type: which execution unit needs to handle it.

For example:

ADD R1, R2, R3

After decoding, the CPU can determine:

Operation type: addition
Source operands: R2, R3
Destination: R1
Execution unit: integer arithmetic unit

Note: the above is an assembly representation for ease of understanding. Actual machine instructions are composed of binary encodings, and the encoding formats differ across ISAs.

3.3 Execute

After decoding is complete, the CPU hands the instruction to the corresponding execution unit.

For example:

R2 = 10
R3 = 20

ADD R1, R2, R3

The ALU performs:

10 + 20 = 30

The calculation result is temporarily stored in the data path or related registers during execution, waiting for subsequent write back.

Different types of instructions invoke different execution units:

Instruction Type Execution Unit
Integer addition/subtraction Integer ALU
Integer multiplication/division Integer multiplication/division unit
Floating-point operations Floating-point execution unit
Data read Load/store unit
Conditional branch Branch execution unit

3.4 Write Back

After the operation is complete, the result needs to be written to the destination.

For example:

R2 = 10
R3 = 20

ADD R1, R2, R3

Final result:

R1 = 30

At this point, the execution of this instruction is complete.

However, not all instructions need to write back to a register. For example, some store instructions write data to memory, while branch instructions may update the program execution location.

3.5 Complete Instruction Execution Flow

flowchart TB
    A["Fetch<br/>Get the instruction according to the PC"]
    B["Decode<br/>Parse the opcode and operands"]
    C["Execute<br/>Invoke the corresponding execution unit"]
    D["Write Back<br/>Save the execution result"]
    E["Next instruction"]

    A --> B
    B --> C
    C --> D
    D --> E
    E --> A

    classDef fetch fill:#DCEFE3,stroke:#6DAD86,color:#285C40
    classDef decode fill:#DFEAF7,stroke:#82A8D1,color:#315579
    classDef execute fill:#F8EEDB,stroke:#D4B77C,color:#735B2D
    classDef write fill:#EAE5F4,stroke:#A99BC8,color:#574A78

    class A fetch
    class B decode
    class C execute
    class D write
    class E fetch

This is the classic instruction execution cycle.

But it must be made clear: this is a simplified model for understanding how a CPU works. Modern high-performance CPUs do not strictly wait until one instruction has finished executing before starting the next; instead, through techniques such as pipelining and out-of-order execution, they keep multiple instructions in different processing stages at the same time.

IV. How Does the CPU Improve Instruction Execution Efficiency?

If the CPU had to completely finish executing one instruction before it could begin processing the next, a large amount of hardware resources might not be fully utilized.

To improve processing efficiency, modern CPUs have introduced various optimization techniques.

4.1 Instruction Pipeline

Pipelining is an important technique for modern processors to improve instruction throughput.

Its core idea is:

Split the instruction execution process into multiple stages so that multiple instructions can be in different stages at the same time.

It can be understood as an assembly line:

  • The first instruction is being executed.
  • The second instruction is being decoded.
  • The third instruction is being fetched.

Different instructions can occupy different processing stages in parallel.

Suppose an instruction needs to go through five stages:

Stage Name Main Work
IF Instruction Fetch Fetch instruction
ID Instruction Decode Decode instruction
EX Execute Execute
MEM Memory Access Access memory
WB Write Back Write back

Pipeline execution diagram:

flowchart TB
    subgraph S1["Clock Cycle 1"]
        A1["Instruction 1: IF"]
    end
    subgraph S2["Clock Cycle 2"]
        A2["Instruction 1: ID"]
        B2["Instruction 2: IF"]
    end
    subgraph S3["Clock Cycle 3"]
        A3["Instruction 1: EX"]
        B3["Instruction 2: ID"]
        C3["Instruction 3: IF"]
    end
    subgraph S4["Clock Cycle 4"]
        A4["Instruction 1: MEM"]
        B4["Instruction 2: EX"]
        C4["Instruction 3: ID"]
        D4["Instruction 4: IF"]
    end
    subgraph S5["Clock Cycle 5"]
        A5["Instruction 1: WB"]
        B5["Instruction 2: MEM"]
        C5["Instruction 3: EX"]
        D5["Instruction 4: ID"]
        E5["Instruction 5: IF"]
    end

    classDef stage fill:#DCEFE3,stroke:#6DAD86,color:#285C40

    class A1,A2,A3,A4,A5,B2,B3,B4,B5,C3,C4,C5,D4,D5,E5 stage