Back to series
· 9 min read

JIT (Just-In-Time Compilation)


I. What Is JIT Just-in-Time Compilation?

JIT (Just-In-Time Compilation) is a technique that performs compilation while a program is running. It converts bytecode or another intermediate representation into native machine code for the target platform, allowing the CPU to execute the compiled code directly.

Traditional compilation usually occurs before the program runs, whereas JIT defers part of the compilation work to runtime.

For example, a Java program typically goes through the following process:

flowchart TB
    A["Java source code"] --> B["javac compiler"]
    B --> C["JVM bytecode"]
    C --> D["JVM loading and execution"]
    D --> E["Interpreted execution"]
    D --> F["JIT compilation"]
    F --> G["Native machine code"]
    E --> H["Program execution"]
    G --> H

    style A fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style B fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style C fill:#F3EDFC,stroke:#C4B2E3,color:#334155
    style D fill:#FFF5DE,stroke:#E8D29A,color:#334155
    style E fill:#E8EDF2,stroke:#B8C3CE,color:#334155
    style F fill:#F3EDFC,stroke:#C4B2E3,color:#334155
    style G fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style H fill:#E0F0FE,stroke:#93BCE8,color:#1E4976

    linkStyle default stroke:#A3B8AC,stroke-width:1.5px

Note that JIT is not a mechanism that every language runtime must use. Some runtimes mainly use interpreted execution, some use AOT compilation, and others combine an interpreter, multiple JIT compilation tiers, and AOT compilation.

1. Core Goals of JIT

The core goal of JIT is not simply to translate bytecode into machine code, but to:

  • Reduce interpretation overhead: reduce the cost of interpreting bytecode instruction by instruction.
  • Leverage runtime information: perform targeted optimizations based on actual execution behavior.
  • Improve hot code performance: prioritize compiling frequently executed methods and loops.
  • Dynamically adjust execution strategy: recompile or deoptimize based on runtime feedback.
  • Balance startup cost and execution efficiency: avoid costly optimization of all code from the start.

2. Differences Between JIT and Traditional Compilation

Comparison dimension JIT Just-in-Time Compilation AOT Ahead-of-Time Compilation
Compilation time During program execution Before program execution
Compilation basis Source code, bytecode, or IR, combined with runtime information Information available at compile time
Target code Usually generates native machine code Usually generates native machine code
Runtime overhead Has compilation and warm-up costs Usually no runtime JIT compilation required
Dynamic optimization Can use runtime feedback Usually lacks actual runtime feedback
Startup performance May be affected by warm-up Usually better for fast startup
Use cases Long-running programs with obvious hot spots Startup-sensitive programs or programs with constrained deployment environments

AOT does not mean runtime optimization is completely impossible. For example, AOT programs can still improve performance through runtime profiling, dynamic dispatch, and other optimization mechanisms; their code generation mainly occurs before runtime.

II. Why Is JIT Needed?

To understand JIT, you first need to understand the limitations of interpreted execution and traditional compilation.

1. Performance Problems of Interpreted Execution

An interpreter usually needs to read bytecode, parse opcodes, fetch operands, and then perform the corresponding operation.

For example:

iload_0
iload_1
iadd
istore_2

The interpreter needs to process these instructions in sequence.

In large loops and repeated method calls, instruction fetching, dispatch, and execution scheduling incur additional overhead.

flowchart TB
    A["Read bytecode"] --> B["Parse opcodes"]
    B --> C["Fetch operands"]
    C --> D["Execute instruction"]
    D --> E{"More instructions?"}
    E -->|"Yes"| A
    E -->|"No"| F["Execution ends"]

    style A fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style B fill:#E8EDF2,stroke:#B8C3CE,color:#334155
    style C fill:#FFF5DE,stroke:#E8D29A,color:#334155
    style D fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style E fill:#F3EDFC,stroke:#C4B2E3,color:#334155
    style F fill:#E5F4EC,stroke:#A3CBB5,color:#334155

    linkStyle default stroke:#A3B8AC,stroke-width:1.5px

An interpreter can apply various optimization techniques to reduce this overhead, but for hot code that runs continuously, directly executing optimized native machine code is usually more advantageous.

2. Limitations of Traditional AOT Compilation

AOT compilation happens before the program runs.

A compiler can perform static optimizations such as control flow analysis, data flow analysis, and constant propagation, but it cannot fully know the program's actual future runtime behavior.

For example:

public int calculate(Object value) {
    if (value instanceof Integer) {
        return ((Integer) value) + 10;
    }
    return 0;
}

During static compilation, the compiler must consider the possibility of different types.

But if at runtime it discovers that this method almost always receives an Integer in actual execution, JIT can use this information to generate more targeted machine code.

This is the important advantage of JIT: it can optimize a program based on real runtime behavior, rather than relying only on static information available before compilation.

III. The Complete JIT Workflow

JIT usually does not compile all code at program startup; instead, it identifies code worth optimizing through runtime monitoring.

A typical workflow includes:

  1. The program starts and loads bytecode or an intermediate representation.
  2. The program executes through an interpreter or low-optimization-level code.
  3. It collects method call counts, loop execution counts, and type information.
  4. It determines whether the code has reached the hot-code threshold.
  5. It submits the hot code to the JIT compiler.
  6. It generates and installs native machine code.
  7. Subsequent execution preferentially uses the compiled code.
  8. If an optimization assumption becomes invalid, deoptimization may be triggered.
flowchart TB
    A["Program starts"] --> B["Load bytecode"]
    B --> C["Interpret or baseline compile"]
    C --> D["Collect runtime information"]
    D --> E{"Compilation condition met?"}
    E -->|"No"| C
    E -->|"Yes"| F["Submit to JIT compiler"]
    F --> G["Optimize intermediate representation"]
    G --> H["Generate native machine code"]
    H --> I["Install compiled code"]
    I --> J["Execute optimized code"]
    J --> K{"Optimization assumption holds?"}
    K -->|"Yes"| J
    K -->|"No"| L["Deoptimize"]
    L --> C

    style A fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style B fill:#F3EDFC,stroke:#C4B2E3,color:#334155
    style C fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style D fill:#E8EDF2,stroke:#B8C3CE,color:#334155
    style E fill:#FFF5DE,stroke:#E8D29A,color:#334155
    style F fill:#F3EDFC,stroke:#C4B2E3,color:#334155
    style G fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style H fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style I fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style J fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style K fill:#FFF5DE,stroke:#E8D29A,color:#334155
    style L fill:#F3EDFC,stroke:#C4B2E3,color:#334155

    linkStyle default stroke:#A3B8AC,stroke-width:1.5px

The above is a typical JIT execution model; it does not mean that all runtimes strictly follow this order.

IV. HotSpot Detection

One of the most important mechanisms of JIT is hot spot detection.

Hot Code refers to code that is executed frequently while the program runs, and usually includes:

  • Frequently called methods.
  • Frequently executed loops.
  • Code paths that are executed often.
  • Code snippets with a high optimization payoff.

JIT does not compile all code indiscriminately, because compilation itself also consumes CPU time and memory.

If a method is executed only once, then performing costly optimizations on it may not be worth the expense.

1. Method Call Counting

The runtime can record how many times a method has been called.

For example:

public void process() {
    // Business processing logic
}

If process() is called in large numbers within a short period of time, the runtime may identify it as a hot method.

2. Loop Backedge Counting

For loop structures, JIT can also monitor the number of loop iterations.

For example:

public long sum(int n) {
    long result = 0;

    for (int i = 0; i < n; i++) {
        result += i;
    }

    return result;
}

A loop may execute millions of times through just a single method call.

If only method call counts are tracked, this kind of hot spot could be missed. Therefore, the runtime also needs to pay attention to how frequently loop backedges are executed.

3. Hot Spot Thresholds

Hot spot detection usually requires setting certain trigger conditions.

For example:

Monitored Metric Purpose
Method call count Identify frequently called methods
Loop backedge count Identify high-frequency loops
Execution time Determine the actual runtime cost of the code
Type distribution Analyze object types and call targets
Compilation payoff Assess whether optimization is worthwhile

Note that hot spot thresholds are not necessarily fixed constants. At runtime, they may be adjusted dynamically based on code execution frequency, the compilation queue, code complexity, and runtime policy.

4. Why Is Hot Spot Detection Important?

Hot spot detection determines how JIT allocates compilation resources.

flowchart TB
    A["Running code"] --> B["Collect execution data"]
    B --> C["Analyze call and loop frequency"]
    C --> D{"Has hot spot characteristics?"}
    D -->|"No"| E["Continue interpreting or low-tier execution"]
    D -->|"Yes"| F["Enter compilation queue"]
    F --> G["Perform optimizing compilation"]

    style A fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style B fill:#E8EDF2,stroke:#B8C3CE,color:#334155
    style C fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style D fill:#FFF5DE,stroke:#E8D29A,color:#334155
    style E fill:#E8EDF2,stroke:#B8C3CE,color:#334155
    style F fill:#F3EDFC,stroke:#C4B2E3,color:#334155
    style G fill:#E5F4EC,stroke:#A3CBB5,color:#334155

    linkStyle default stroke:#A3B8AC,stroke-width:1.5px

The core value of hot spot detection is this: directing limited compilation resources toward the code most likely to yield performance gains.

V. JIT Compilation Tiers

A modern JIT does not necessarily have only one compiler. To balance compilation speed and execution performance, some runtimes adopt tiered compilation.

1. The Basic Idea of Tiered Compilation

Different optimization levels have different costs:

  • Low optimization level: fast compilation, but the generated machine code is less optimized.
  • High optimization level: higher compilation cost, but may generate machine code with better execution efficiency.

Tiered compilation allows the runtime to gradually raise the optimization level of code.

flowchart TB
    A["Program Code"] --> B["Interpreted Execution"]
    B --> C{"Needs Compilation?"}
    C -->|"No"| B
    C -->|"Yes"| D["Baseline Compilation"]
    D --> E["Collect More Runtime Information"]
    E --> F{"Worth Deep Optimization?"}
    F -->|"No"| D
    F -->|"Yes"| G["Highly Optimized Compilation"]
    G --> H["Optimized Machine Code"]

    style A fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
    style B fill:#E5F4EC,stroke:#A3CBB5,color:#334155
    style C fill:#FFF5DE,stroke:#E8D29A,color:#334155
    style D fill:#F3EDFC,stroke:#C4B2E3,color:#334155
    style E fill:#E8EDF2,stroke:#B8C3CE,color:#334155
    style F fill:#FFF5DE,stroke:#E8D29A,color:#334155
    style G fill:#F3EDFC,stroke:#C4B2E3,color:#334155
    style H fill:#E5F4EC,stroke:#A3CBB5,color:#334155

    linkStyle default stroke:#A3B8AC,stroke-width:1.5px

2. Advantages of Tiered Compilation

Advantage Description
Reduce Compilation Wait Use a faster execution mode first
Reduce Ineffective Optimization Avoid costly compilation for all code
Improve Long-Term Performance Perform deep optimization on hot code
Accumulate Information Gradually Use actual execution data to guide subsequent compilation

3. Tiered Compilation in Java HotSpot

The HotSpot JVM has a mature tiered compilation mechanism.

Typical execution tiers include:

Tier Execution Mode Main Characteristics
Tier 0 Interpreted execution Collect basic runtime information
Tier 1 C1 compilation, simple compilation Quickly generate machine code
Tier 2 C1 compilation, limited profiling Collect more profiling information
Tier 3 C1 compilation, full profiling Accumulate data for further optimization
Tier 4 C2 compilation Perform deeper optimization

Among them:

  • C1 is a compiler oriented toward fast compilation.
  • C2 is a compiler oriented toward high-performance optimization.

The actual execution tier is affected by factors such as the JVM version, startup parameters, and runtime policies. Not every method must go through all tiers.

VI. Core Optimization Techniques of JIT

The performance advantages of JIT largely come from runtime optimization.

1. Method Inlining

Method inlining is one of the most important JIT optimization techniques.

It embeds the code of a called method directly at the call site, reducing the extra overhead in the method invocation process.

For example:

public int square(int x) {
    return x * x;
}

public int calculate(int value) {
    return square(value) + 1;
}

When not inlined, the execution process involves a method call.

After inlining, the logic may become:

public int calculate(int value) {
    return value * value + 1;
}

The value of method inlining is not only reducing call overhead; more importantly, it creates conditions for subsequent optimizations.

For example:

  • Constant propagation.
  • Dead code elimination.
  • Common subexpression elimination.
  • Loop optimization.
  • Escape analysis.

Note that inlining is not simple text substitution. The compiler needs to consider factors such as call targets, method size, call frequency, recursion relationships, and code bloat.

2. Constant Folding

If the result of an expression can be determined at compile time, the compiler can calculate it in advance.

For example:

int result = 10 * 20 + 5;

It may be optimized to:

int result = 205;

This avoids repeatedly performing the relevant calculation at runtime.

3. Constant Propagation

If the value of a variable can be determined on an execution path, the compiler can replace the variable with the corresponding constant.

For example:

int x = 10;
int y = x + 5;

It can be optimized to:

int y = 15;

Constant propagation and constant folding are often used together.

4. Dead Code Elimination

Eliminate computations and instructions that do not affect the program's observable results.

For example:

int a = 10;
int b = 20;
int result = a + b;

return 100;

If result does not affect any observable behavior, the related computations may be removed.

Note that code that may produce side effects cannot be removed arbitrarily, such as file writes, network requests, state modifications, and exception throwing.

5. Escape Analysis

Escape analysis is used to determine whether a reference to an object escapes a certain analysis scope.

For example:

public int calculate() {
    Point point = new Point(10, 20);
    return point.x + point.y;
}

If the compiler can prove that point does not escape the current method and that the way the object is used satisfies the optimization conditions, object allocation may be eliminated.

Related optimizations include:

  • Scalar replacement.
  • Lock elimination.
  • Partial allocation optimization.

There is an important distinction here:

Escape analysis itself is not a memory allocation strategy; it is an analysis technique that helps the compiler determine whether objects can undergo further optimization.

An object not escaping does not necessarily mean it will be allocated on the stack. The actual optimization may be to completely eliminate object allocation or to decompose the object into independent fields.

6. Lock Elimination

If the JIT can prove that an object will not be accessed by other threads, synchronization operations around that object may be redundant.

For example:

public void process() {
    Object lock = new Object();

    synchronized (lock) {
        // Perform operation
    }
}

If lock does not escape, the JIT may eliminate the related synchronization operations.

Lock elimination relies on information such as escape analysis, and it does not mean that all synchronization operations in the program can be removed.

7. Loop Optimization

Loops are usually hot spots in a program.

The JIT can perform various optimizations on loops:

Optimization Technique Effect
Loop unrolling Reduce loop control and branch overhead
Loop-invariant code hoisting Move loop-invariant computations outside the loop
Strength reduction Replace expensive operations with lower-cost operations
Loop vectorization Use SIMD instructions to process multiple data in parallel
Range check elimination Reduce array bounds checks when safety is proven

For example:

for (int i = 0; i < array.length; i++) {
    result += array[i] * factor;
}

If factor does not change during the loop, the compiler may optimize the related computations or access checks.