Back to series
· 14 min read

Continuous Batching: Dynamic Scheduling for Large Model Inference


1. What Is Continuous Batching?

Continuous Batching, often called continuous batch scheduling, is a dynamic batching technique used in large-model inference services.

Its core idea is: it no longer requires all requests in a batch to start and finish at the same time; instead, requests can dynamically join and exit according to their own execution progress.

Traditional batching usually uses fixed batches:

  • A batch of requests enters the model at the same time.
  • All requests perform inference together.
  • Wait until all requests in the batch are complete.
  • Only then start processing the next batch of requests.

Continuous Batching is different:

  • Completed requests immediately exit the current batch.
  • Newly arrived requests can join at appropriate scheduling boundaries.
  • Unfinished requests continue executing Decode.
  • The scheduler dynamically adjusts the batch based on GPU resources and request states.

This mechanism is especially suitable for Transformer autoregressive generation tasks, because different requests often differ significantly in input length, output length, and execution time.

1.1 Why Is Continuous Batching Needed?

Suppose there are 4 requests:

Request Input Length Output Length
A 500 Token 100 Token
B 300 Token 20 Token
C 800 Token 200 Token
D 200 Token 50 Token

If traditional fixed batching is used, the 4 requests execute together. B may finish generating very early but still has to wait for A, C, and D to complete.

This causes two problems:

  1. Completed requests occupy batch slots and cannot release execution resources in time.
  2. Newly arrived requests must wait for the current batch to end and cannot promptly use idle GPU capacity.

Continuous Batching allows B to exit immediately after it completes and allows new requests to enter the scheduling process.

1.2 Core Differences

Comparison Dimension Static Batching Continuous Batching
Batch Composition Fixed Dynamically changing
Request Admission Usually before the batch starts Can join at scheduling boundaries
Request Exit Usually waits for the entire batch to finish Exits after the individual request completes
Batch Lifecycle Uniform overall Independent per request
GPU Utilization Easily affected by long requests Usually higher
Scheduling Complexity Lower Higher
Applicable Scenarios Offline batch computing Online generation services

Note that Continuous Batching does not mean that all requests on the GPU can interrupt an executing CUDA Kernel at any time.

In practice, systems usually add requests, remove requests, and reorganize batches at scheduling iteration boundaries such as Prefill and Decode.


2. Problems with Traditional Static Batching

To understand Continuous Batching, you first need to understand why traditional batching wastes GPU resources.

2.1 Fixed-Batch Execution Mechanism

Traditional Static Batching organizes a group of requests into a fixed Batch.

For example, a Batch contains three requests: A, B, and C:

flowchart TD
    A["Receive requests A, B, C"] --> B["Form a fixed Batch"]
    B --> C["Execute Prefill uniformly"]
    C --> D["Execute Decode uniformly"]
    D --> E{"Have all requests completed?"}
    E -->|"No"| D
    E -->|"Yes"| F["Release the entire Batch"]
    F --> G["Receive the next batch of requests"]

    style A fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style B fill:#EAE8FA,stroke:#A49BD5,color:#51477D
    style C fill:#EAF3EC,stroke:#83A995,color:#315D49
    style D fill:#EAF3EC,stroke:#83A995,color:#315D49
    style E fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
    style F fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style G fill:#E0F0FE,stroke:#93BCE8,color:#1E4976

The problem is that different requests usually have different generation lengths.

Assume:

  • A needs to generate 100 tokens.
  • B needs to generate 20 tokens.
  • C needs to generate 60 tokens.

When B finishes generating, A and C still need to continue executing.

If the batch must finish as a whole, the compute slot corresponding to B cannot be used to process new requests in time.

2.2 Computational Waste Caused by Padding

To organize inputs of different lengths into a uniform tensor, traditional batching often requires padding.

For example:

Request A: Hello world
Request B: Hello
Request C: Hello world today

After padding:

Request A: [Hello, world, PAD]
Request B: [Hello, PAD, PAD]
Request C: [Hello, world, today]

The model needs to process an input tensor of uniform shape.

Padding introduces invalid positions, adding some computation and memory access overhead.

For autoregressive generation tasks, there may also be the issue of inconsistent output lengths.

2.3 Head-of-Line Blocking

Head-of-Line Blocking is a typical problem in fixed batching.

A single long request can slow down the release of resources for the entire batch.

For example:

Request A: ████████████████████
Request B: ████
Request C: ██████████

B has already finished, but if the batch needs to wait for A and C, B's resources cannot be immediately used by new requests.

Its essence is:

The batch lifecycle is determined by the longest request, not by each request's own lifecycle.

2.4 Declining GPU Utilization

In online inference services, request arrival times are irregular.

If the system must wait for enough requests to form a new batch, the following may occur:

  • The GPU waits for requests to accumulate.
  • After some requests finish, GPU compute resources are underutilized.
  • Long requests delay short requests.
  • Bursty traffic increases queueing time.

Therefore, fixed batching struggles to fully adapt to the dynamic load of online services.


III. How Continuous Batching Works at Its Core

The core of Continuous Batching is transforming the batch from a fixed execution unit into a dynamic scheduling unit.

3.1 Independent Request Management

Each request has its own independent execution state.

Common states include:

State Meaning
Waiting Waiting to enter the execution queue
Running Executing model inference
Prefill Processing input tokens
Decode Generating token by token
Finished Generation completed
Aborted Cancelled or terminated
Preempted Temporarily preempted due to insufficient resources

Actual inference engines may use different state names, but the core idea is the same.

Requests no longer depend on the lifecycle of the entire batch; instead, they advance independently.

3.2 Dynamic Joining and Exiting

Assume the current batch has three requests:

Batch = [A, B, C]

When B finishes:

Batch = [A, C]

At this point, a new request D arrives:

Batch = [A, C, D]

The entire process does not need to wait for A and C to both finish.

The dynamic change process is as follows:

flowchart TD
    A["Request enters the Waiting queue"] --> B["Scheduler selects runnable requests"]
    B --> C["Form the current Batch"]
    C --> D["Run one model inference iteration"]
    D --> E{"Is the request complete?"}
    E -->|"Yes"| F["Remove from the current Batch"]
    E -->|"No"| G["Keep request for continued execution"]
    F --> H["Release corresponding execution resources"]
    G --> I["Retain KV Cache"]
    H --> J["Re-evaluate scheduling conditions"]
    I --> J
    J --> K{"Can new requests be added?"}
    K -->|"Yes"| L["Select new request to add"]
    K -->|"No"| M["Continue current request"]
    L --> C
    M --> D

    style A fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style B fill:#EAE8FA,stroke:#A49BD5,color:#51477D
    style C fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style D fill:#EAF3EC,stroke:#83A995,color:#315D49
    style E fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
    style F fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style G fill:#EAF3EC,stroke:#83A995,color:#315D49
    style H fill:#EAF3EC,stroke:#83A995,color:#315D49
    style I fill:#EAF3EC,stroke:#83A995,color:#315D49
    style J fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style K fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
    style L fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style M fill:#EAF3EC,stroke:#83A995,color:#315D49

3.3 Iteration-Level Scheduling

Continuous Batching usually uses the model inference iteration as the scheduling unit.

In the Decode phase, one iteration usually generates one Token for each currently active request.

For example:

Iteration 1: A, B, C
Iteration 2: A, B, C
Iteration 3: A, C, D
Iteration 4: A, C, D
Iteration 5: C, D, E

Here:

  • B completes after the 2nd iteration.
  • D joins in the 3rd iteration.
  • A completes after the 4th iteration.
  • E joins in the 5th iteration.

Each iteration can reorganize the batch according to request status.

This is the meaning of Continuous in Continuous Batching.

It does not build a fixed Batch once; instead, it lets the batch keep changing throughout the request lifecycle.


IV. Relationship Between Continuous Batching and Prefill/Decode

Continuous Batching is closely related to the two stages of Transformer: Prefill and Decode.

4.1 Prefill Phase

Prefill processes all Prompt Tokens from the user input.

For example:

User input:
Please explain what Transformer is and its core structure.

Tokenizer:
[T1, T2, T3, ..., T100]

The model feeds these Tokens into the Transformer all at once, computes Attention and FFN at each layer, and generates the corresponding KV Cache.

Characteristics of Prefill:

  • The number of input Tokens may be large.
  • Computational intensity is relatively high.
  • Matrix parallelism can be used.
  • Prompt lengths can vary greatly between requests.

4.2 Decode Phase

Decode generates output Token by Token.

For example:

Pass 1: Transformer
Pass 2: is a
Pass 3: based on
Pass 4: attention mechanism

Each Decode usually inputs only the newly generated Token and reads the previously saved KV Cache.

Characteristics of Decode:

  • Usually, each request generates one Token per round.
  • It needs continuous access to historical KV Cache.
  • It is relatively sensitive to VRAM bandwidth and memory access efficiency.
  • Generation lengths differ across requests.

4.3 Scheduling Differences Between the Two Phases

Dimension Prefill Decode
Input The entire prompt The current token
Output Builds the initial KV cache Newly generated token
Compute characteristics Leans compute-intensive Usually memory-bandwidth-bound
Sequence length Processes multiple input tokens at once Processes a small number of tokens per round
Scheduling challenges Long prompts occupy a large amount of compute resources Many requests continuously compete for KV cache
Common optimizations Chunked Prefill Continuous Batching

The most typical use case for Continuous Batching is the Decode phase, but modern inference engines usually bring Prefill into unified scheduling as well.

4.4 Mixed scheduling of Prefill and Decode

In an online service, the following may exist at the same time:

  • Requests that have already entered Decode.
  • Requests that have just arrived and are waiting for Prefill.
  • Requests currently performing a long-prompt Prefill.

If Prefill occupies too much GPU compute at once, it may slow down token generation for Decode requests.

Therefore, the inference engine needs to balance these two workloads.

Typical approaches include:

  1. Limit the number of tokens prefilled in a single round.
  2. Split long prompts into multiple chunks.
  3. Control the scheduling priority of Decode requests.
  4. Dynamically select tasks based on the remaining GPU compute budget.

Five, Core components of Continuous Batching

A complete Continuous Batching system usually contains the following components.

5.1 Request Queue

Responsible for receiving requests that have not yet been executed.

Requests are usually ordered by arrival time, priority, service tier, or scheduling policy.

Common queues:

  • Waiting Queue: requests waiting to be scheduled.
  • Running Queue: requests currently executing.
  • Preempted Queue: requests waiting to resume after being preempted.

The request queue manages the request lifecycle, but does not directly execute model computation.

5.2 Scheduler

The Scheduler is the core of Continuous Batching.

Its main responsibilities include:

  • Selecting which requests enter the current iteration.
  • Determining whether there are enough KV Cache Blocks.
  • Controlling resource allocation for Prefill and Decode.
  • Handling request completion and cancellation.
  • Handling preemption when memory is insufficient.
  • Controlling the maximum number of concurrent sequences and the batch token count.

The scheduler must consider not only GPU compute capacity, but also the available capacity of the KV Cache.

5.3 Model Runner

Responsible for executing model computation.

Typical responsibilities:

  • Preparing input tokens.
  • Constructing batch tensors.
  • Preparing positional encoding and Attention metadata.
  • Executing Transformer layers.
  • Calling Attention Kernels.
  • Computing Logits.
  • Performing token sampling.

5.4 KV Cache Manager

Responsible for managing the historical Key and Value state of each request.

In Continuous Batching, requests join and exit frequently, so the KV Cache must support dynamic allocation and release.

Main responsibilities:

  • Allocating KV Cache for new requests.
  • Extending the cache for Decode requests.
  • Releasing the cache for completed requests.
  • Managing shared prefix cache.
  • Reclaiming resources when memory is insufficient.

5.5 Sampler

Responsible for selecting the next token based on the Logits output by the model.

Common strategies:

  • Greedy Sampling.
  • Temperature Sampling.
  • Top-K Sampling.
  • Top-P Sampling.
  • Repetition Penalty.

After the Sampler generates a new token, the inference engine updates the request state and decides whether to continue execution.

5.6 Detokenizer

Responsible for converting the tokens generated by the model into user-readable text.

In a streaming service, the Detokenizer continuously processes new tokens and sends the generated results to the client through the API.


Six, Continuous Batching and KV Cache management

Whether Continuous Batching can run efficiently depends to a large extent on the KV Cache management capability.

6.1 Why is KV Cache the key?

During autoregressive generation, the model needs to retain the Key and Value of historical tokens.

For a request, its KV Cache size can be approximated as:

M_{KV} = 2 \times L \times S \times H_{kv} \times D_h \times B

Where:

  • L: Number of Transformer layers.
  • S: Number of tokens currently cached.
  • H_{kv}: Number of KV heads.
  • D_h: Dimension of each head.
  • B: Number of bytes per element.
  • 2: Represents Key and Value, respectively.

As Decode continues, the sequence length increases, and the KV Cache also keeps growing.

Therefore, the number of requests in a Batch is not the only resource limit.

Even if a Batch has only a few requests, it may still consume a large amount of GPU memory due to long contexts.

6.2 Dynamically Allocating KV Cache

Traditional approaches may preallocate a contiguous block of GPU memory for each request.

This creates two problems:

  • Maximum sequence length must be estimated in advance.
  • Actual request lengths vary, which easily leads to wasted GPU memory.

Modern inference engines usually use paging mechanisms such as PagedAttention to manage the KV Cache.

Divide the KV Cache into fixed-size Blocks:

Request A: Block 1 → Block 4 → Block 7
Request B: Block 2 → Block 5
Request C: Block 3 → Block 6 → Block 8

Each request uses a Block Table to record the mapping between logical positions and physical Blocks.

When a request needs more cache, new Blocks are allocated.

When a request completes, the Blocks it occupies are released.

6.3 Releasing Resources After a Request Completes

For example, the current Batch:

A: KV Block 1, 4
B: KV Block 2, 5
C: KV Block 3, 6

After B completes:

A: KV Block 1, 4
C: KV Block 3, 6

Release:
KV Block 2, 5

Released Blocks can be used by new requests.

This enables continuous batching to dynamically manage not only computational tasks but also GPU memory resources.

6.4 Relationship Between KV Cache and Batch Size

Traditional Batch Size mainly describes the number of requests in a batch.

But in large-model inference, what really matters is the resource usage of the entire Batch.

For example:

Batch Requests Average context length KV Cache pressure
A 16 512 tokens Lower
B 16 8192 tokens Higher
C 32 2048 tokens Medium-high
D 8 32768 tokens Very high

Therefore, the scheduler cannot determine whether new requests can still be added based only on the number of requests.

It must also check:

  • Currently remaining KV Cache Blocks.
  • Prompt length of the new request.
  • Expected generation length.
  • Maximum context length.
  • Cache growth rate of current requests.

7. Scheduling Strategies for Continuous Batching

Continuous Batching does more than dynamically add and remove requests; it also needs to reasonably decide which requests to execute in each round.

7.1 FCFS: First Come, First Served

FCFS (First Come First Served) schedules requests in the order they arrive.

Advantages:

  • Simple to implement.
  • Low scheduling overhead.
  • Relatively easy to ensure fairness in request order.

Disadvantages:

  • Long prompts may block short prompts.
  • Long requests may affect the response time of short requests.
  • Not easy to accommodate services with different priorities.

7.2 Priority Scheduling

Execution order is determined by request priority.

For example:

Request Priority Scheduling preference
A High Prioritized
B Normal Processed normally
C Low Processed when resources permit

Suitable for scenarios that require differentiated service levels, such as:

  • Enterprise paid users.
  • Regular users.
  • Background batch jobs.
  • Internal system calls.

But if high-priority requests continue to arrive in large numbers over a long period, low-priority requests may starve.

Therefore, it is usually necessary to combine fairness and maximum waiting time mechanisms.

7.3 Token Budget: Token Budget Scheduling

Limiting only the number of requests is not enough.

The inference engine can also limit the total number of tokens allowed to be processed in the current iteration.

For example:

max_num_batched_tokens = 4096

This means the scheduler needs to arrange Prefill and Decode work within the token budget of the current batch.

The exact meaning of the parameter and how it is implemented depend on the inference engine and version; it cannot simply be understood as the maximum number of tokens for an entire request.

Token Budget can prevent a single long Prompt from occupying too many compute resources.

7.4 Chunked Prefill: Chunked Prefill

For extremely long Prompts, Prefill can be split into multiple Chunks and executed step by step.

For example:

原始 Prompt:8192 Token

Chunk 1:2048 Token
Chunk 2:2048 Token
Chunk 3:2048 Token
Chunk 4:2048 Token

After chunking, the scheduler can arrange Decode work between different Prefill Chunks.

The goal is to reduce the interference of long Prefill with online generation requests.

7.5 Preemption and Recovery

When GPU memory is insufficient, the inference engine may need to temporarily pause some requests.

Common approaches:

  • Preemption: preempt requests.
  • Recompute: free the cache and recompute part of the historical state when recovering.
  • Swap: migrate part of the cache to CPU memory or another storage tier.

Preemption allows the system to continue accepting and scheduling tasks when resources are insufficient, but it introduces additional overhead.

For example, Recompute increases redundant computation, and Swap increases data transfer latency.

Therefore, preemption usually requires decisions that take into account request priority, cache occupancy, and service objectives.


Eight、Complete Execution Flow of Continuous Batching

The typical execution flow of an online inference service is as follows.

8.1 Requests Enter the System

The client submits a request:

{
  "model": "example-model",
  "messages": [
    {
      "role": "user",
      "content": "什么是 Continuous Batching?"
    }
  ],
  "max_tokens": 512,
  "stream": true
}

After the API Server receives the request, it performs:

  1. Parameter validation.
  2. Tokenizer encoding.
  3. Request ID assignment.
  4. Generation parameter initialization.
  5. The request enters the waiting queue.

8.2 Scheduler Selects Requests

The Scheduler checks:

  • The current number of running requests.
  • Available GPU memory.
  • Available Blocks in the KV Cache.
  • The current Token budget.
  • Request priority.
  • The resource usage of Prefill and Decode.

It selects the requests suitable for entering this round of execution.

8.3 Execute Prefill

New requests enter the Prefill stage.

The model reads the Prompt Tokens, performs the Transformer forward computation, and builds the initial KV Cache.

If the Prompt is long, it can be processed in chunks via Chunked Prefill.

8.4 Execute Decode

Requests enter the Decode stage.

Each iteration performs:

  1. Prepare the input Tokens for the current request.
  2. Read the historical KV Cache.
  3. Execute the Transformer.
  4. Compute Logits.
  5. Select the next Token via Sampling.
  6. Update the request state and KV Cache.

8.5 Dynamically Update the Batch

After each iteration ends:

  • Completed requests exit.
  • Cancelled requests release resources.
  • Incomplete requests remain.
  • New requests join according to budget and resource conditions.
  • Preempted requests wait for recovery.

8.6 Output Results

The generated Tokens are converted into text by the Detokenizer.

For streaming requests, the system continuously returns the generated content to the client.

When a request ends, the related KV Cache and scheduling resources are released.

```mermaid flowchart TD A["Client submits request"] --> B["API Server receives request"] B --> C["Tokenize and parameter validation"] C --> D["Enter Waiting Queue"] D --> E["Scheduler resource check"] E --> F{"Are scheduling conditions met?"} F -->|"No"| G["Keep waiting or adjust scheduling"] G --> E F -->|"Yes"| H["Allocate KV Cache Block"] H --> I["Prefill processes the Prompt"] I --> J["Enter Decode"] J --> K["Run Transformer forward computation"] K --> L["Logits and Sampling"] L --> M["Output new Token"] M --> N{"Is the request finished?"} N -->|"No"| O["Update KV Cache and request state"] O --> P["Reorganize the Batch"] P --> J N -->|"Yes"| Q["Release KV Cache"] Q --> R["Return final completed state"] style A fill:#E0F0FE,stroke:#93BCE8,color:#1E4976 style B fill:#EAF3EC,stroke:#83A995,color:#315D49 style C fill:#EAF3EC,stroke:#83A995,color:#315D49 style D fill:#EAE8FA,stroke:#A49BD5,color:#51477D style E fill:#EAF3EC,stroke:#83A995,color:#315D49 style F fill:#F7F4E8,stroke:#C7B77A,color:#675A2D style G fill:#F7F4E8,stroke:#C7B77A,color:#675A2D style H fill:#DCEFE5,stroke:#4F8B70,color:#24563F style I fill:#EAF3EC,stroke:#83A995,color:#315D49 style J fill:#DCEFE5,stroke:#4F8B70,color:#24563F style K fill:#EAF3EC,stroke:#83A995,color:#315D49 style L fill:#EAE8FA,stroke:#A49BD5,color:#51477D style M fill:#E0F0FE,stroke:#93BCE8,color:#1E4976 style N fill:#F7F4E8,stroke:#C7B77A,color:#675A2D style O fill:#EAF3EC,stroke:#83A995,color:#315D49 style P fill:#DCEFE5,stroke:#4F8B70,color:#24563F style Q fill:#EAF3EC,stroke:#83A995,color:#315D49 style R fill:#DCEFE5,stroke:#4F8B70,color:#24563F ```

9. Exception Handling Mechanisms in Continuous Batching

Continuous batching must handle more than normal generation: it also has to deal with request cancellation, insufficient memory, execution failures, and anomalies during resource release.

9.1 Common Exception Scenarios

Exception Possible Cause Handling Approach
Insufficient KV Cache Long context or too high concurrency Limit new requests, preempt, or reclaim
GPU OOM Weights, cache, or workspace exceed capacity Lower concurrency, limit length, adjust memory budget
Request cancellation User terminates proactively Mark the request and release resources
Model execution failure CUDA Kernel or operator exception Terminate the request, log the error
Tokenizer failure Input format or model configuration mismatch Return a request error
Request timeout Queued too long or generation too slow Terminate on timeout, release resources
Worker abnormal exit GPU fault or process crash Worker recovery or service restart

9.2 Exception Branches and Resource Reclamation

Particular care is needed here: a request cancellation does not mean the memory currently in use by a GPU Kernel can simply be released at any moment.

A real implementation must coordinate execution state, asynchronous operations, and cache reference relationships to ensure resources are not released prematurely.

flowchart TD
    A["Scheduling request"] --> B["Run inference iteration"]
    B --> C{"Did an exception occur?"}
    C -->|"No"| D{"Is the request complete?"}
    D -->|"No"| E["Retain request and KV Cache"]
    E --> B
    D -->|"Yes"| F["Wait for related executions to finish"]
    C -->|"Yes"| G{"Exception type"}
    G -->|"Insufficient KV Cache"| H["Check cache and concurrency budget"]
    G -->|"Request canceled"| I["Mark request as terminated"]
    G -->|"Model execution failed"| J["Terminate current request and log error"]
    G -->|"Timeout"| K["Perform timeout cleanup"]
    H --> L{"Is recovery possible?"}
    L -->|"Yes"| M["Adjust scheduling or perform preemption"]
    L -->|"No"| N["Return insufficient-resource error"]
    M --> B
    I --> F
    J --> F
    K --> F
    F --> O["Confirm related cache is no longer in use"]
    O --> P["Release KV Cache and request resources"]
    P --> Q["Update scheduling state"]
    N --> Q

    style A fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style B fill:#EAF3EC,stroke:#83A995,color:#315D49
    style C fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
    style D fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
    style E fill:#EAF3EC,stroke:#83A995,color:#315D49
    style F fill:#EAF3EC,stroke:#83A995,color:#315D49
    style G fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
    style H fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style I fill:#F7EAE7,stroke:#C98D7C,color:#754438
    style J fill:#F7EAE7,stroke:#C98D7C,color:#754438
    style K fill:#F7EAE7,stroke:#C98D7C,color:#754438
    style L fill:#F7F4E8,stroke:#C7B77A,color:#675A2D
    style M fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style N fill:#F7EAE7,stroke:#C98D7C,color:#754438
    style O fill:#EAF3EC,stroke:#83A995,color:#315D49
    style P fill:#DCEFE5,stroke:#4F8B70,color:#24563F
    style Q fill:#E0F0FE,stroke:#93BCE8,color:#1E4976