I. What Is JIT Just-in-Time Compilation?
JIT (Just-In-Time Compilation) is a technique that performs compilation while a program is running. It converts bytecode or another intermediate representation into native machine code for the target platform, allowing the CPU to execute the compiled code directly.
Traditional compilation usually occurs before the program runs, whereas JIT defers part of the compilation work to runtime.
For example, a Java program typically goes through the following process:
flowchart TB
A["Java source code"] --> B["javac compiler"]
B --> C["JVM bytecode"]
C --> D["JVM loading and execution"]
D --> E["Interpreted execution"]
D --> F["JIT compilation"]
F --> G["Native machine code"]
E --> H["Program execution"]
G --> H
style A fill:#E5F4EC,stroke:#A3CBB5,color:#334155
style B fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style C fill:#F3EDFC,stroke:#C4B2E3,color:#334155
style D fill:#FFF5DE,stroke:#E8D29A,color:#334155
style E fill:#E8EDF2,stroke:#B8C3CE,color:#334155
style F fill:#F3EDFC,stroke:#C4B2E3,color:#334155
style G fill:#E5F4EC,stroke:#A3CBB5,color:#334155
style H fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
linkStyle default stroke:#A3B8AC,stroke-width:1.5px
Note that JIT is not a mechanism that every language runtime must use. Some runtimes mainly use interpreted execution, some use AOT compilation, and others combine an interpreter, multiple JIT compilation tiers, and AOT compilation.
1. Core Goals of JIT
The core goal of JIT is not simply to translate bytecode into machine code, but to:
- Reduce interpretation overhead: reduce the cost of interpreting bytecode instruction by instruction.
- Leverage runtime information: perform targeted optimizations based on actual execution behavior.
- Improve hot code performance: prioritize compiling frequently executed methods and loops.
- Dynamically adjust execution strategy: recompile or deoptimize based on runtime feedback.
- Balance startup cost and execution efficiency: avoid costly optimization of all code from the start.
2. Differences Between JIT and Traditional Compilation
| Comparison dimension | JIT Just-in-Time Compilation | AOT Ahead-of-Time Compilation |
|---|---|---|
| Compilation time | During program execution | Before program execution |
| Compilation basis | Source code, bytecode, or IR, combined with runtime information | Information available at compile time |
| Target code | Usually generates native machine code | Usually generates native machine code |
| Runtime overhead | Has compilation and warm-up costs | Usually no runtime JIT compilation required |
| Dynamic optimization | Can use runtime feedback | Usually lacks actual runtime feedback |
| Startup performance | May be affected by warm-up | Usually better for fast startup |
| Use cases | Long-running programs with obvious hot spots | Startup-sensitive programs or programs with constrained deployment environments |
AOT does not mean runtime optimization is completely impossible. For example, AOT programs can still improve performance through runtime profiling, dynamic dispatch, and other optimization mechanisms; their code generation mainly occurs before runtime.
II. Why Is JIT Needed?
To understand JIT, you first need to understand the limitations of interpreted execution and traditional compilation.
1. Performance Problems of Interpreted Execution
An interpreter usually needs to read bytecode, parse opcodes, fetch operands, and then perform the corresponding operation.
For example:
iload_0
iload_1
iadd
istore_2
The interpreter needs to process these instructions in sequence.
In large loops and repeated method calls, instruction fetching, dispatch, and execution scheduling incur additional overhead.
flowchart TB
A["Read bytecode"] --> B["Parse opcodes"]
B --> C["Fetch operands"]
C --> D["Execute instruction"]
D --> E{"More instructions?"}
E -->|"Yes"| A
E -->|"No"| F["Execution ends"]
style A fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style B fill:#E8EDF2,stroke:#B8C3CE,color:#334155
style C fill:#FFF5DE,stroke:#E8D29A,color:#334155
style D fill:#E5F4EC,stroke:#A3CBB5,color:#334155
style E fill:#F3EDFC,stroke:#C4B2E3,color:#334155
style F fill:#E5F4EC,stroke:#A3CBB5,color:#334155
linkStyle default stroke:#A3B8AC,stroke-width:1.5px
An interpreter can apply various optimization techniques to reduce this overhead, but for hot code that runs continuously, directly executing optimized native machine code is usually more advantageous.
2. Limitations of Traditional AOT Compilation
AOT compilation happens before the program runs.
A compiler can perform static optimizations such as control flow analysis, data flow analysis, and constant propagation, but it cannot fully know the program's actual future runtime behavior.
For example:
public int calculate(Object value) {
if (value instanceof Integer) {
return ((Integer) value) + 10;
}
return 0;
}
During static compilation, the compiler must consider the possibility of different types.
But if at runtime it discovers that this method almost always receives an Integer in actual execution, JIT can use this information to generate more targeted machine code.
This is the important advantage of JIT: it can optimize a program based on real runtime behavior, rather than relying only on static information available before compilation.
III. The Complete JIT Workflow
JIT usually does not compile all code at program startup; instead, it identifies code worth optimizing through runtime monitoring.
A typical workflow includes:
- The program starts and loads bytecode or an intermediate representation.
- The program executes through an interpreter or low-optimization-level code.
- It collects method call counts, loop execution counts, and type information.
- It determines whether the code has reached the hot-code threshold.
- It submits the hot code to the JIT compiler.
- It generates and installs native machine code.
- Subsequent execution preferentially uses the compiled code.
- If an optimization assumption becomes invalid, deoptimization may be triggered.
flowchart TB
A["Program starts"] --> B["Load bytecode"]
B --> C["Interpret or baseline compile"]
C --> D["Collect runtime information"]
D --> E{"Compilation condition met?"}
E -->|"No"| C
E -->|"Yes"| F["Submit to JIT compiler"]
F --> G["Optimize intermediate representation"]
G --> H["Generate native machine code"]
H --> I["Install compiled code"]
I --> J["Execute optimized code"]
J --> K{"Optimization assumption holds?"}
K -->|"Yes"| J
K -->|"No"| L["Deoptimize"]
L --> C
style A fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style B fill:#F3EDFC,stroke:#C4B2E3,color:#334155
style C fill:#E5F4EC,stroke:#A3CBB5,color:#334155
style D fill:#E8EDF2,stroke:#B8C3CE,color:#334155
style E fill:#FFF5DE,stroke:#E8D29A,color:#334155
style F fill:#F3EDFC,stroke:#C4B2E3,color:#334155
style G fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style H fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style I fill:#E5F4EC,stroke:#A3CBB5,color:#334155
style J fill:#E5F4EC,stroke:#A3CBB5,color:#334155
style K fill:#FFF5DE,stroke:#E8D29A,color:#334155
style L fill:#F3EDFC,stroke:#C4B2E3,color:#334155
linkStyle default stroke:#A3B8AC,stroke-width:1.5px
The above is a typical JIT execution model; it does not mean that all runtimes strictly follow this order.
IV. HotSpot Detection
One of the most important mechanisms of JIT is hot spot detection.
Hot Code refers to code that is executed frequently while the program runs, and usually includes:
- Frequently called methods.
- Frequently executed loops.
- Code paths that are executed often.
- Code snippets with a high optimization payoff.
JIT does not compile all code indiscriminately, because compilation itself also consumes CPU time and memory.
If a method is executed only once, then performing costly optimizations on it may not be worth the expense.
1. Method Call Counting
The runtime can record how many times a method has been called.
For example:
public void process() {
// Business processing logic
}
If process() is called in large numbers within a short period of time, the runtime may identify it as a hot method.
2. Loop Backedge Counting
For loop structures, JIT can also monitor the number of loop iterations.
For example:
public long sum(int n) {
long result = 0;
for (int i = 0; i < n; i++) {
result += i;
}
return result;
}
A loop may execute millions of times through just a single method call.
If only method call counts are tracked, this kind of hot spot could be missed. Therefore, the runtime also needs to pay attention to how frequently loop backedges are executed.
3. Hot Spot Thresholds
Hot spot detection usually requires setting certain trigger conditions.
For example:
| Monitored Metric | Purpose |
|---|---|
| Method call count | Identify frequently called methods |
| Loop backedge count | Identify high-frequency loops |
| Execution time | Determine the actual runtime cost of the code |
| Type distribution | Analyze object types and call targets |
| Compilation payoff | Assess whether optimization is worthwhile |
Note that hot spot thresholds are not necessarily fixed constants. At runtime, they may be adjusted dynamically based on code execution frequency, the compilation queue, code complexity, and runtime policy.
4. Why Is Hot Spot Detection Important?
Hot spot detection determines how JIT allocates compilation resources.
flowchart TB
A["Running code"] --> B["Collect execution data"]
B --> C["Analyze call and loop frequency"]
C --> D{"Has hot spot characteristics?"}
D -->|"No"| E["Continue interpreting or low-tier execution"]
D -->|"Yes"| F["Enter compilation queue"]
F --> G["Perform optimizing compilation"]
style A fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style B fill:#E8EDF2,stroke:#B8C3CE,color:#334155
style C fill:#E5F4EC,stroke:#A3CBB5,color:#334155
style D fill:#FFF5DE,stroke:#E8D29A,color:#334155
style E fill:#E8EDF2,stroke:#B8C3CE,color:#334155
style F fill:#F3EDFC,stroke:#C4B2E3,color:#334155
style G fill:#E5F4EC,stroke:#A3CBB5,color:#334155
linkStyle default stroke:#A3B8AC,stroke-width:1.5px
The core value of hot spot detection is this: directing limited compilation resources toward the code most likely to yield performance gains.
V. JIT Compilation Tiers
A modern JIT does not necessarily have only one compiler. To balance compilation speed and execution performance, some runtimes adopt tiered compilation.
1. The Basic Idea of Tiered Compilation
Different optimization levels have different costs:
- Low optimization level: fast compilation, but the generated machine code is less optimized.
- High optimization level: higher compilation cost, but may generate machine code with better execution efficiency.
Tiered compilation allows the runtime to gradually raise the optimization level of code.
flowchart TB
A["Program Code"] --> B["Interpreted Execution"]
B --> C{"Needs Compilation?"}
C -->|"No"| B
C -->|"Yes"| D["Baseline Compilation"]
D --> E["Collect More Runtime Information"]
E --> F{"Worth Deep Optimization?"}
F -->|"No"| D
F -->|"Yes"| G["Highly Optimized Compilation"]
G --> H["Optimized Machine Code"]
style A fill:#E0F0FE,stroke:#93BCE8,color:#1E4976
style B fill:#E5F4EC,stroke:#A3CBB5,color:#334155
style C fill:#FFF5DE,stroke:#E8D29A,color:#334155
style D fill:#F3EDFC,stroke:#C4B2E3,color:#334155
style E fill:#E8EDF2,stroke:#B8C3CE,color:#334155
style F fill:#FFF5DE,stroke:#E8D29A,color:#334155
style G fill:#F3EDFC,stroke:#C4B2E3,color:#334155
style H fill:#E5F4EC,stroke:#A3CBB5,color:#334155
linkStyle default stroke:#A3B8AC,stroke-width:1.5px
2. Advantages of Tiered Compilation
| Advantage | Description |
|---|---|
| Reduce Compilation Wait | Use a faster execution mode first |
| Reduce Ineffective Optimization | Avoid costly compilation for all code |
| Improve Long-Term Performance | Perform deep optimization on hot code |
| Accumulate Information Gradually | Use actual execution data to guide subsequent compilation |
3. Tiered Compilation in Java HotSpot
The HotSpot JVM has a mature tiered compilation mechanism.
Typical execution tiers include:
| Tier | Execution Mode | Main Characteristics |
|---|---|---|
| Tier 0 | Interpreted execution | Collect basic runtime information |
| Tier 1 | C1 compilation, simple compilation | Quickly generate machine code |
| Tier 2 | C1 compilation, limited profiling | Collect more profiling information |
| Tier 3 | C1 compilation, full profiling | Accumulate data for further optimization |
| Tier 4 | C2 compilation | Perform deeper optimization |
Among them:
- C1 is a compiler oriented toward fast compilation.
- C2 is a compiler oriented toward high-performance optimization.
The actual execution tier is affected by factors such as the JVM version, startup parameters, and runtime policies. Not every method must go through all tiers.
VI. Core Optimization Techniques of JIT
The performance advantages of JIT largely come from runtime optimization.
1. Method Inlining
Method inlining is one of the most important JIT optimization techniques.
It embeds the code of a called method directly at the call site, reducing the extra overhead in the method invocation process.
For example:
public int square(int x) {
return x * x;
}
public int calculate(int value) {
return square(value) + 1;
}
When not inlined, the execution process involves a method call.
After inlining, the logic may become:
public int calculate(int value) {
return value * value + 1;
}
The value of method inlining is not only reducing call overhead; more importantly, it creates conditions for subsequent optimizations.
For example:
- Constant propagation.
- Dead code elimination.
- Common subexpression elimination.
- Loop optimization.
- Escape analysis.
Note that inlining is not simple text substitution. The compiler needs to consider factors such as call targets, method size, call frequency, recursion relationships, and code bloat.
2. Constant Folding
If the result of an expression can be determined at compile time, the compiler can calculate it in advance.
For example:
int result = 10 * 20 + 5;
It may be optimized to:
int result = 205;
This avoids repeatedly performing the relevant calculation at runtime.
3. Constant Propagation
If the value of a variable can be determined on an execution path, the compiler can replace the variable with the corresponding constant.
For example:
int x = 10;
int y = x + 5;
It can be optimized to:
int y = 15;
Constant propagation and constant folding are often used together.
4. Dead Code Elimination
Eliminate computations and instructions that do not affect the program's observable results.
For example:
int a = 10;
int b = 20;
int result = a + b;
return 100;
If result does not affect any observable behavior, the related computations may be removed.
Note that code that may produce side effects cannot be removed arbitrarily, such as file writes, network requests, state modifications, and exception throwing.
5. Escape Analysis
Escape analysis is used to determine whether a reference to an object escapes a certain analysis scope.
For example:
public int calculate() {
Point point = new Point(10, 20);
return point.x + point.y;
}
If the compiler can prove that point does not escape the current method and that the way the object is used satisfies the optimization conditions, object allocation may be eliminated.
Related optimizations include:
- Scalar replacement.
- Lock elimination.
- Partial allocation optimization.
There is an important distinction here:
Escape analysis itself is not a memory allocation strategy; it is an analysis technique that helps the compiler determine whether objects can undergo further optimization.
An object not escaping does not necessarily mean it will be allocated on the stack. The actual optimization may be to completely eliminate object allocation or to decompose the object into independent fields.
6. Lock Elimination
If the JIT can prove that an object will not be accessed by other threads, synchronization operations around that object may be redundant.
For example:
public void process() {
Object lock = new Object();
synchronized (lock) {
// Perform operation
}
}
If lock does not escape, the JIT may eliminate the related synchronization operations.
Lock elimination relies on information such as escape analysis, and it does not mean that all synchronization operations in the program can be removed.
7. Loop Optimization
Loops are usually hot spots in a program.
The JIT can perform various optimizations on loops:
| Optimization Technique | Effect |
|---|---|
| Loop unrolling | Reduce loop control and branch overhead |
| Loop-invariant code hoisting | Move loop-invariant computations outside the loop |
| Strength reduction | Replace expensive operations with lower-cost operations |
| Loop vectorization | Use SIMD instructions to process multiple data in parallel |
| Range check elimination | Reduce array bounds checks when safety is proven |
For example:
for (int i = 0; i < array.length; i++) {
result += array[i] * factor;
}
If factor does not change during the loop, the compiler may optimize the related computations or access checks.