I. Overall Architecture of the CPU
After understanding how the CPU executes instructions, it is also necessary to further understand how the CPU organizes its computing resources.
When buying a computer or checking server configurations, you will often see:
- 8 cores, 16 threads
- 16 cores, 32 threads
- Base frequency 3.2 GHz
- Max turbo frequency 5.0 GHz
These parameters describe different characteristics of the CPU.
Among them:
- Core: the physical processing unit inside the CPU that actually executes instructions.
- Thread: an execution unit that the operating system can schedule; a distinction must be made between software threads and hardware threads.
- Clock Frequency: the operating frequency of the CPU clock signal, reflecting the processor's operating rhythm.
Understanding these three concepts is the foundation for understanding modern CPU architecture and performance.
1.1 Basic Compositional Levels of a CPU
Modern CPUs generally use a multi-core architecture: a single processor chip integrates multiple physical cores, each with its own execution resources and possibly support for multiple hardware threads.
flowchart TB
CPU["CPU"]
CPU --> C1["Core 1"]
CPU --> C2["Core 2"]
CPU --> C3["Core 3"]
CPU --> C4["Core 4"]
C1 --> T11["Hardware Thread 1"]
C1 --> T12["Hardware Thread 2"]
C2 --> T21["Hardware Thread 3"]
C2 --> T22["Hardware Thread 4"]
C3 --> T31["Hardware Thread 5"]
C3 --> T32["Hardware Thread 6"]
C4 --> T41["Hardware Thread 7"]
C4 --> T42["Hardware Thread 8"]
classDef cpu fill:#DCEFE3,stroke:#6DAD86,color:#285C40,stroke-width:2px
classDef core fill:#DFEAF7,stroke:#82A8D1,color:#315579
classDef thread fill:#F8EEDB,stroke:#D4B77C,color:#735B2D
class CPU cpu
class C1,C2,C3,C4 core
class T11,T12,T21,T22,T31,T32,T41,T42 thread
This is a simplified diagram of a 4-core, 8-thread processor.
Each physical core supports two hardware threads, so the operating system sees 8 logical processors.
Note: hardware threads are not independent physical cores; two hardware threads usually share some execution resources of the core they belong to.
II. CPU Core (Core)
2.1 What Is a CPU Core?
A CPU core is a physical computing unit inside the processor that can independently execute an instruction stream.
A core typically includes:
- Instruction fetch and decode unit.
- Integer arithmetic unit.
- Floating-point arithmetic unit.
- Register file.
- Load and store unit.
- Branch prediction unit.
- L1 and L2 caches (the specific design varies by architecture).
Each core has relatively independent instruction execution capability and can process computing tasks from different programs or threads simultaneously.
For example:
CPU: 4 cores
Core 1: execute task A
Core 2: execute task B
Core 3: execute task C
Core 4: execute task D
The four cores can execute different instruction streams at the same time; this is true hardware parallelism.
2.2 Single-Core and Multi-Core
A single-core processor has only one physical computing core.
A multi-core processor integrates multiple physical cores into one processor chip.
flowchart TB
subgraph SINGLE["Single-Core Processor"]
direction TB
S["CPU"]
S --> SC["Core 1"]
SC --> SE["Execution Resources"]
end
subgraph MULTI["Multi-Core Processor"]
direction TB
M["CPU"]
M --> C1["Core 1"]
M --> C2["Core 2"]
M --> C3["Core 3"]
M --> C4["Core 4"]
end
classDef cpu fill:#DCEFE3,stroke:#6DAD86,color:#285C40
classDef core fill:#DFEAF7,stroke:#82A8D1,color:#315579
classDef unit fill:#F8EEDB,stroke:#D4B77C,color:#735B2D
class S,M cpu
class SC,C1,C2,C3,C4 core
class SE unit
The main advantages of multi-core processors:
- They can execute multiple independent tasks simultaneously.
- They improve the execution efficiency of multithreaded programs.
- They improve the server's concurrent processing capability.
- They increase overall computing throughput within a reasonable power consumption range.
For example, when a server processes multiple HTTP requests simultaneously, it can assign different requests to different cores for execution.
2.3 More Cores Do Not Mean Linear Performance Growth
Adding more cores does not mean program performance will necessarily improve proportionally.
Suppose a program's execution process is as follows:
- 80% of the work can be parallelized.
- 20% of the work must be serial.
Even if the CPU has an infinite number of cores, the serial part still has to be executed.
This can be described by Amdahl's Law:
Where:
- P: The proportion of the program that can be executed in parallel.
- N: The number of processing cores.
- Speedup(N): The theoretical speedup relative to a single core.
For example:
When 80% of a program can be executed in parallel:
| Number of cores | Theoretical speedup |
|---|---|
| 1 core | 1.00× |
| 2 cores | 1.67× |
| 4 cores | 2.50× |
| 8 cores | 3.33× |
| Infinite cores | 5.00× |
As can be seen, the more cores there are, the higher the theoretical speedup, but the rate of increase gradually decreases.
In practical applications, factors such as thread synchronization, task scheduling, memory bandwidth, cache contention, and communication overhead also need to be considered.
3. CPU Threads (Thread)
Threads are another easily confused concept in CPU architecture.
In computing, a thread can refer either to an operating system software thread or to a hardware thread supported by the CPU.
The two are not the same concept.
3.1 Software Threads
A software thread is the basic unit of execution scheduled by the operating system.
A process can contain one or more threads.
Each thread has its own independent execution state, including:
- Program counter.
- Register context.
- Stack space.
- Thread scheduling state.
Multiple threads within the same process usually share the process's address space and resources.
For example:
Browser process
├── Thread 1: Page rendering
├── Thread 2: Network requests
├── Thread 3: JavaScript execution
└── Thread 4: Background tasks
Software threads are a concept at the level of program design and operating system scheduling.
A software thread is ultimately scheduled by the operating system onto some available logical processor for execution.
3.2 Hardware Threads
A hardware thread is an independent hardware execution context that a CPU core can maintain.
It enables a physical core to maintain the execution states of multiple threads simultaneously.
For example:
Physical core Core 1
├── Hardware thread 1
└── Hardware thread 2
The two hardware threads can each execute a different software thread.
However, they share some of the core's computing resources, such as execution units, caches, and instruction fetch/decode resources.
Therefore, hardware threads are not equivalent to two independent physical cores.
3.3 Relationship Between Software Threads and Hardware Threads
The operating system is responsible for scheduling software threads onto the logical processors represented by hardware threads for execution.
flowchart TB
APP["Application"]
APP --> P1["Software Thread A"]
APP --> P2["Software Thread B"]
APP --> P3["Software Thread C"]
APP --> P4["Software Thread D"]
P1 --> OS["Operating System Scheduler"]
P2 --> OS
P3 --> OS
P4 --> OS
OS --> H1["Logical Processor 1"]
OS --> H2["Logical Processor 2"]
OS --> H3["Logical Processor 3"]
OS --> H4["Logical Processor 4"]
H1 --> C1["Physical Core 1"]
H2 --> C1
H3 --> C2["Physical Core 2"]
H4 --> C2
classDef app fill:#DCEFE3,stroke:#6DAD86,color:#285C40
classDef software fill:#DFEAF7,stroke:#82A8D1,color:#315579
classDef os fill:#EAE5F4,stroke:#A99BC8,color:#574A78
classDef hardware fill:#F8EEDB,stroke:#D4B77C,color:#735B2D
class APP app
class P1,P2,P3,P4 software
class OS os
class H1,H2,H3,H4,C1,C2 hardware
Note the following:
- The number of software threads can far exceed the number of CPU hardware threads.
- Through scheduling and context switching, the operating system allows multiple software threads to share limited CPU resources.
- Multiple software threads do not mean that they can execute simultaneously at the physical level.
3.4 What Is Hyper-Threading?
Hyper-Threading is a simultaneous multithreading technology introduced by Intel, built on SMT (Simultaneous Multithreading).
It allows one physical core to maintain the execution state of multiple hardware threads at the same time.
For example, a core that supports two-thread SMT:
Physical core Core 1
├── Hardware thread 1
└── Hardware thread 2
Two hardware threads can issue instructions to the core at the same time, allowing the core to use ready instructions from another thread when one thread is waiting for data or cannot fully utilize execution resources.
Its core purpose is not to increase the number of physical cores, but to improve the utilization of execution resources in existing cores.
3.5 How Does SMT Improve CPU Utilization?
Suppose a core is executing thread A.
One instruction in thread A needs to wait for memory data to return; during this time, some execution resources may be idle.
If the core supports SMT, it can try to execute instructions in thread B that are already ready.
flowchart TB
subgraph CORE["Physical core supporting SMT"]
direction TB
T1["Thread A<br/>Waiting for memory data"]
T2["Thread B<br/>Has ready instructions"]
EX["Shared execution resources<br/>ALU / FPU / Load-Store"]
T1 --> EX
T2 --> EX
end
RESULT["Improve execution resource utilization"]
EX --> RESULT
classDef thread fill:#DFEAF7,stroke:#82A8D1,color:#315579
classDef unit fill:#DCEFE3,stroke:#6DAD86,color:#285C40
classDef result fill:#F8EEDB,stroke:#D4B77C,color:#735B2D
class T1,T2 thread
class EX unit
class RESULT result
Advantages of SMT:
- Improves execution resource utilization.
- Improves throughput of multithreaded workloads.
- Reduces idle time for some execution resources.
However, SMT also has limitations:
- Multiple threads share execution resources.
- It may cause cache and memory bandwidth contention.
- Single-thread performance may not improve.
- Multithreaded performance gains depend on the specific workload.
Therefore, two hardware threads do not equal two physical cores, nor do they mean performance doubles.
3.6 Relationship Between Core Count and Thread Count
For processors that support SMT, the following formula can be used:
For example:
| Physical cores | Hardware threads per core | Number of logical processors |
|---|---|---|
| 4 | 1 | 4 |
| 4 | 2 | 8 |
| 8 | 1 | 8 |
| 8 | 2 | 16 |
| 16 | 2 | 32 |
The thread count here refers to hardware threads, not the number of software threads created by applications.
4. CPU Clock Frequency
Core count and thread count determine a CPU's parallel execution capability, while clock frequency describes the pace at which the CPU runs.
4.1 What Is a Clock?
A clock is a periodic signal inside the processor used to coordinate the operation of the various digital circuits.
Many operations inside the CPU must be synchronized to clock ticks.
For example:
- Register state updates.
- Advancement of instruction pipeline stages.
- Synchronization of data transfers and control logic.
- Commit of operation results.
The clock signal is usually provided by clock generation and distribution circuitry and distributed to different processing modules.
4.2 What Is Clock Frequency?
Clock frequency indicates the number of cycles the clock signal completes per second.
Units:
| Unit | Meaning |
|---|---|
| Hz | 1 cycle per second |
| MHz | 1 million cycles per second |
| GHz | 1 billion cycles per second |
For example:
This means there are 3.5 billion clock cycles per second.
Clock period and frequency are reciprocals of each other:
Where:
- T: clock period.
- f: clock frequency.
For example: