Back to series
· 11 min read

Blocking and Non-blocking


I. Basic Concepts of Blocking and Non-blocking

Blocking and non-blocking describe whether the caller has to wait when an operation cannot be completed immediately.

They are about how the calling thread behaves while an operation is incomplete — not about whether the task runs asynchronously, and not about whether the operation ultimately succeeds.

1. Blocking

Blocking means that after the caller initiates an operation, if the operation cannot be completed immediately, the calling thread has to wait and cannot go on executing the code that follows the current call.

For example, receiving data with a blocking socket:

import socket

sock = socket.socket()
sock.connect(("example.com", 80))

data = sock.recv(1024)
print(data)

If there is no readable data in the receive buffer, recv() may keep waiting until:

  • Data arrives.
  • The connection is closed.
  • An error occurs during the operation.
  • The timeout expires (if a timeout is set).

While blocked, the thread is usually in a waiting state and does not continuously occupy the CPU to execute user code, but the thread's resources remain occupied.

2. Non-blocking

Non-blocking means that after the caller initiates an operation, if the operation cannot be completed immediately, the interface returns control right away instead of continuing to wait.

For example:

import socket

sock = socket.socket()
sock.setblocking(False)

try:
    data = sock.recv(1024)
except BlockingIOError:
    print("当前没有可读取的数据")

If the data is not yet ready, recv() returns immediately and indicates through an exception or status code that the read cannot be completed at this time.

The caller can continue with other tasks and try again later.

Core characteristics of non-blocking calls:

  • They do not keep waiting just because the current operation is not ready.
  • They can return a not-completed status immediately.
  • The caller has to handle the case where the operation is not yet ready.
  • They usually need to work together with polling, event notification, or I/O multiplexing.

3. Key Differences

Aspect Blocking Non-blocking
When the operation is not ready Waits for the operation to complete Returns immediately
Calling thread May enter a waiting state Can continue executing
Obtaining the result Returns the result after the operation completes May first return a not-ready status
Programming complexity Relatively simple Requires managing state and retries
CPU usage Usually does not consume much CPU while waiting Depends on the polling or event-notification approach
Concurrency model Often used with threads or processes Often used with an event-driven model
Typical scenarios Blocking file reads, sockets Non-blocking sockets, event loops

Note: non-blocking does not mean the operation is guaranteed to succeed, nor that it is necessarily executed in the background. It only guarantees that when an operation cannot be completed immediately, the calling thread is not made to keep waiting.


II. Execution Mechanisms of Blocking and Non-blocking

1. Blocking Call Flow

A blocking operation typically goes through the following process:

  1. The application issues a system call.
  2. The kernel checks the conditions required for the operation.
  3. If the conditions are met, the operation is performed.
  4. If the conditions are not met, the calling thread enters a waiting state.
  5. When the relevant event occurs, the thread is woken up.
  6. The operation completes and returns its result.

Taking the reading of network data as an example:

flowchart TD
    A["Application calls recv()"] --> B["Enter kernel"]
    B --> C{"Is there data in the receive buffer"}
    C -->|Yes| D["Read data"]
    C -->|No| E["Thread enters waiting state"]
    E --> F["Wait for network data to arrive"]
    F --> G["Thread is woken up"]
    G --> D
    D --> H["Return the read result"]
    H --> I["Continue executing application code"]

    style A fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style B fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style C fill:#FFF5DE,stroke:#E8D29A,color:#334155
    style D fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style E fill:#F3EDFC,stroke:#C4B2E3,color:#334155
    style F fill:#E8EDF2,stroke:#B8C3CE,color:#334155
    style G fill:#F3EDFC,stroke:#C4B2E3,color:#334155
    style H fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style I fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    linkStyle default stroke:#A3B8AC,stroke-width:1.5px

Blocking waits are usually managed by the operating system kernel. Once a thread enters a waiting state, the scheduler can allocate the CPU to other runnable threads.

Therefore, blocking does not mean the CPU keeps spinning.

2. Non-Blocking Call Flow

A non-blocking operation returns immediately when the condition is not met.

flowchart TD
    A["Application calls recv()"] --> B["Enter the kernel"]
    B --> C{"Does the receive buffer have data?"}
    C -->|Yes| D["Read data"]
    C -->|No| E["Immediately return not-ready status"]
    E --> F["Perform other tasks"]
    F --> G["Check again later"]
    G --> B
    D --> H["Return read result"]

    style A fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style B fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style C fill:#FFF5DE,stroke:#E8D29A,color:#334155
    style D fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style E fill:#F3EDFC,stroke:#C4B2E3,color:#334155
    style F fill:#E8EDF2,stroke:#B8C3CE,color:#334155
    style G fill:#FFF5DE,stroke:#E8D29A,color:#334155
    style H fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    linkStyle default stroke:#A3B8AC,stroke-width:1.5px

Non-blocking operations require the caller to manage subsequent processing itself.

If you simply keep trying to read in a loop, it will result in busy polling, wasting CPU resources.


III. The Essence of Blocking and Non-Blocking: Waiting Behavior

1. What Is Thread Blocking?

Thread blocking means that a thread temporarily cannot continue execution and must wait for a certain condition to become true.

Common causes include:

  • Waiting for network data.
  • Waiting for disk I/O.
  • Waiting for a mutex.
  • Waiting for a condition variable.
  • Waiting for a semaphore.
  • Waiting for other threads to finish.
  • Waiting for a timer or external event.

After a thread blocks, the operating system usually moves it out of the runnable state until the waiting condition is satisfied or another wake-up event occurs.

2. Blocking Does Not Mean the CPU Is Idle

Suppose a server has 1,000 threads:

  • 900 threads are waiting for network data.
  • 50 threads are waiting for database connections.
  • 50 threads are processing requests.

Although most threads are not using the CPU, they still occupy resources such as thread stacks and kernel management structures.

If the number of threads continues to increase, it may lead to:

  • Increased memory usage.
  • Increased scheduling management costs.
  • More context switches.
  • Thread pool exhaustion.
  • Decreased system throughput.

Therefore, high-concurrency systems usually need to reduce the resource consumption caused by a large number of threads blocking and waiting for long periods.

3. Non-Blocking Does Not Mean There Is No Waiting

A non-blocking call only prevents the current calling thread from waiting continuously.

The application still needs to wait for the operation to actually complete; it can simply delegate the waiting process to another mechanism.

For example:

  • Wait for file descriptors to become ready through epoll.
  • Wait for callbacks through an event loop.
  • Wait for asynchronous results through a Future.
  • Obtain I/O results through operating system completion notifications.

Waiting does not disappear; rather, the way waiting is organized changes.


IV. Differences Between Blocking and Non-Blocking, Synchronous and Asynchronous

These are the four concepts most easily confused in concurrent programming.

Synchronous and asynchronous concern how task results are delivered and how tasks collaborate.

Blocking and non-blocking focus on whether the calling thread needs to wait for an operation to complete.

The two pairs of concepts belong to different dimensions.

1. Synchronous

Synchronous means the caller must organize subsequent execution according to the result of the completed operation.

A typical approach is to call a function and obtain its return value.

2. Asynchronous

Asynchronous means task submission and task completion are separated; the caller can continue doing other work first and later obtain the result through a notification or a result object.

3. Four Combinations

Type Caller Behavior Typical Scenario
Synchronous blocking Wait for the operation to complete and obtain the result directly Blocking Socket read
Synchronous non-blocking Return status immediately and actively check the result Non-blocking Socket polling
Asynchronous blocking Submit the task asynchronously, but the caller blocks waiting for the result Call Future.result() after submitting a thread pool task
Asynchronous non-blocking Continue execution after submitting the task, and receive a notification upon completion Event loop, callbacks, asynchronous I/O

4. Relationships Among the Four Models

flowchart TB
    A["Invocation and Waiting Model"] --> B["Synchronous"]
    A --> C["Asynchronous"]

    B --> D["Synchronous Blocking"]
    B --> E["Synchronous Non-blocking"]
    C --> F["Asynchronous Blocking"]
    C --> G["Asynchronous Non-blocking"]

    D --> H["Returns after the operation completes"]
    E --> I["Returns status immediately; caller checks actively"]
    F --> J["Task executes asynchronously; caller waits for result"]
    G --> K["Notifies via event after task completes"]

    style A fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style B fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style C fill:#F3EDFC,stroke:#C4B2E3,color:#334155
    style D fill:#FFF5DE,stroke:#E8D29A,color:#334155
    style E fill:#FFF5DE,stroke:#E8D29A,color:#334155
    style F fill:#FFF5DE,stroke:#E8D29A,color:#334155
    style G fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style H fill:#E8EDF2,stroke:#B8C3CE,color:#334155
    style I fill:#E8EDF2,stroke:#B8C3CE,color:#334155
    style J fill:#E8EDF2,stroke:#B8C3CE,color:#334155
    style K fill:#E8EDF2,stroke:#B8C3CE,color:#334155
    linkStyle default stroke:#A3B8AC,stroke-width:1.5px

For example:

future = executor.submit(task)
result = future.result()

submit() submits the task to the thread pool for execution, but result() blocks the calling thread when the task is not yet complete.

Therefore, although the task is executed asynchronously by the thread pool, the caller waits in a blocking manner.


V. Blocking I/O and Non-blocking I/O

I/O is the most typical use case for blocking and non-blocking.

1. Blocking I/O

Typical characteristics of blocking I/O:

  • Initiate a read or write operation.
  • Wait when data is not ready.
  • Return the result after the operation completes.
  • The application can use simple sequential code.

Using a Linux network socket as an example:

#include <sys/socket.h>
#include <stdio.h>

int main() {
    int fd = /* connected socket */;
    char buffer[1024];

    ssize_t n = recv(fd, buffer, sizeof(buffer), 0);

    if (n > 0) {
        printf("收到 %zd 字节\n", n);
    }

    return 0;
}

If the socket is in blocking mode, recv() usually waits when there is no readable data.

2. Non-blocking I/O

After setting the socket to non-blocking mode:

#include <sys/socket.h>
#include <fcntl.h>
#include <errno.h>
#include <stdio.h>

int main() {
    int fd = /* connected socket */;

    int flags = fcntl(fd, F_GETFL, 0);
    fcntl(fd, F_SETFL, flags | O_NONBLOCK);

    char buffer[1024];

    ssize_t n = recv(fd, buffer, sizeof(buffer), 0);

    if (n > 0) {
        printf("收到 %zd 字节\n", n);
    } else if (n < 0 && (errno == EAGAIN || errno == EWOULDBLOCK)) {
        printf("数据暂时未就绪\n");
    }

    return 0;
}

When there is no data in the receive buffer, recv() usually returns -1 immediately and sets errno to EAGAIN or EWOULDBLOCK.

Specific error handling also needs to account for connection closure and other system errors.

3. Blocking I/O vs. Non-blocking I/O

Comparison Dimension Blocking I/O Non-blocking I/O
Data not ready Wait Return immediately
Thread state May enter waiting Can continue executing
Code organization Sequential State-driven
Event handling Usually waits in the call itself Usually requires polling or event notification
Thread resources Each waiting operation may occupy a thread A small number of threads can manage multiple operations
Programming complexity Lower Higher

Note that non-blocking sockets do not mean that all system calls will not block. For example, DNS resolution, file system operations, or some library functions may still use blocking implementations.


6. Busy Polling

If non-blocking calls are not combined with a reasonable event notification mechanism, busy polling can easily occur.

1. How Busy Polling Works

Busy polling means that a program continuously checks whether a condition is met instead of entering a waiting state.

For example:

while (1) {
    ssize_t n = recv(fd, buffer, sizeof(buffer), 0);

    if (n > 0) {
        process(buffer, n);
        break;
    }

    if (errno == EAGAIN || errno == EWOULDBLOCK) {
        continue;
    }

    break;
}

If the data does not arrive for a long time, the program will keep calling recv().

2. Problems with Busy Polling

  • Continuously consumes CPU.
  • A large number of repeated system calls.
  • Increases system power consumption.
  • Competes with other tasks for processor resources.
  • May reduce overall throughput in high-concurrency scenarios.

3. Busy Polling vs. Event Waiting

flowchart TD
    A["Non-blocking read"] --> B{"Is data ready?"}
    B -->|Yes| C["Read and process"]
    B -->|No| D["Poll again immediately"]
    D --> B

    E["Register I/O event"] --> F["Wait for event notification"]
    F --> G["Receive readiness notification"]
    G --> H["Perform non-blocking read"]
    H --> I{"Is there more data?"}
    I -->|Yes| H
    I -->|No| F

    style A fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style B fill:#FFF5DE,stroke:#E8D29A,color:#334155
    style C fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style D fill:#FBE9E9,stroke:#D9AAAA,color:#334155
    style E fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style F fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style G fill:#F3EDFC,stroke:#C4B2E3,color:#334155
    style H fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style I fill:#FFF5DE,stroke:#E8D29A,color:#334155
    linkStyle default stroke:#A3B8AC,stroke-width:1.5px

Event waiting is usually more CPU-efficient than unrestrained polling.

However, in extremely low-latency scenarios, optimized busy polling may also be valuable, such as in high-performance networking and real-time trading systems. It trades CPU resources for shorter event response latency.


7. I/O Multiplexing

I/O multiplexing is a core technology in non-blocking network programming.

It allows a single thread to monitor the I/O state of multiple file descriptors at the same time without allocating a separate thread for each connection.

1. Why Is I/O Multiplexing Needed?

Suppose a server needs to handle 10,000 network connections.

The traditional blocking model may allocate one thread per connection.

This model is susceptible to:

  • Too many threads.
  • Thread stacks consuming a large amount of memory.
  • Increased scheduling overhead.
  • Many threads waiting for data for long periods.
  • Decreased system resource utilization.

I/O multiplexing hands the readiness states of multiple connections to the kernel for centralized monitoring.

2. Workflow of I/O Multiplexing

flowchart TD
    A["Create multiple sockets"] --> B["Set to non-blocking mode"]
    B --> C["Register file descriptors"]
    C --> D["Kernel monitors I/O status"]
    D --> E["Wait for ready events"]
    E --> F{"Are there ready events?"}
    F -->|No| E
    F -->|Yes| G["Return ready descriptors"]
    G --> H["Application performs reads or writes"]
    H --> I["Process business data"]
    I --> D

    style A fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style B fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style C fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style D fill:#F3EDFC,stroke:#C4B2E3,color:#334155
    style E fill:#E8EDF2,stroke:#B8C3CE,color:#334155
    style F fill:#FFF5DE,stroke:#E8D29A,color:#334155
    style G fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style H fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style I fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    linkStyle default stroke:#A3B8AC,stroke-width:1.5px

The core idea is:

  • One thread manages multiple I/O connections.
  • The kernel is responsible for monitoring file descriptors.
  • The application only handles connections that are already ready.
  • Avoid a large number of threads being blocked for long periods while waiting for I/O.

3. select

select is a traditional I/O multiplexing interface.

Main characteristics:

  • Supports multiple file descriptors.
  • Describes the descriptors to be monitored through sets.
  • Each call usually requires the sets of interest to be passed in again.
  • After it returns, you need to iterate through and check which descriptors are ready.
  • Many implementations impose a limit on the number of file descriptors.

Its interface compatibility is relatively good, but in scenarios with a large number of connections, its efficiency and ease of use are limited.

4. poll

poll uses an array to manage file descriptors.

Compared with traditional select:

  • It does not rely on a fixed-size bitmap set.
  • The way the number of file descriptors is limited is different.
  • It still requires the sets of interest to be passed in at call time.
  • After it returns, it usually requires iterating through the descriptor array.

When the number of monitored connections is very large, the traversal cost may be quite noticeable.

5. epoll

epoll is an efficient I/O event notification mechanism provided by Linux.

Core interfaces:

Interface Purpose
epoll_create1() Create an epoll instance
epoll_ctl() Add, modify, or delete monitored events
epoll_wait() Wait for and retrieve ready events

Typical flow:

int epfd = epoll_create1(0);

struct epoll_event event;
event.events = EPOLLIN;
event.data.fd = client_fd;

epoll_ctl(epfd, EPOLL_CTL_ADD, client_fd, &event);

struct epoll_event events[128];

int n = epoll_wait(epfd, events, 128, -1);

for (int i = 0; i < n; i++) {
    int fd = events[i].data.fd;
    // Handle the corresponding file descriptor
}

epoll_wait() can block while waiting for events, or the waiting time can be controlled through the timeout parameter.

Therefore, non-blocking I/O and blocking event waiting can coexist: the application uses non-blocking sockets, while the event loop thread blocks on epoll_wait().

6. Level Triggered and Edge Triggered

epoll supports two important event notification modes.

Level Triggered (LT)

As long as the file descriptor remains in a ready state, subsequent wait operations may still receive notifications.

Characteristics:

  • Relatively simple to program.
  • Notifications can continue to be received even if the data has not been read all at once.
  • Easier to handle incomplete data reads.

Edge Triggered (ET)

The application is notified only when the state of the file descriptor changes.

Characteristics:

  • Reduces repeated notifications.
  • Usually needs to be used together with non-blocking I/O.
  • The application must correctly handle the ready state.
  • A common practice is to keep reading until EAGAIN or EWOULDBLOCK is returned.