Site icon yCrash

The Architect’s Checklist: Detecting Memory Leaks in Project Loom & Virtual Threads

ARBT Nova

Virtual Threads were introduced in Java 21 by Project Loom. They allow a single JVM to handle a large number of tasks and manage millions of concurrent operations, making them especially suitable for I/O bound processes. Because the memory model of virtual threads differs from that of traditional threads, performance issues can be hard to detect.

Although virtual threads consume less memory than traditional threads, the real challenge is that Project Loom’s memory management introduces a variation in how thread count, object retention, and heap usage are connected. This change in memory management while handling a large number of threads may cause inefficiencies, leading to Memory Leaks, heap exhaustion and signaling an OutOfMemoryError.

This article provides a checklist to assist SRE teams and Java architects in detecting, diagnosing, and resolving Virtual Thread memory leaks before they reach production. If you are new to Project Loom, Java Project Loom: Unlocking the Power of Java Virtual Threads is a good starting point for building a strong foundation.

Why Virtual Threads Create a New Memory Leak Surface

Platform threads use OS-level stack memory, approximately 1 MB in size. There is a limitation on the number of threads, which results in fewer ThreadLocal variables and less request data. In contrast, Virtual threads are JVM-managed lightweight threads created and stored on the Java heap, which grows on demand. This allows millions of virtual threads to coexist. This architectural shift has a direct impact: all objects used by Virtual threads remain in the heap, leading to unexpected memory usage and creating a memory leak at scale.

For more insights on the difference between Virtual Threads and Platform threads, Virtual Threads vs Platform Threads in Java 23 is a good place to start. When a memory leak occurs, tools like HeapHero can analyze a heap dump and highlight problem areas.

Four vital memory leak patterns are observed at the Virtual Thread scale:

Fig: Virtual Thread Memory leakage patterns Vs Platform thread boundary including Detection Checklist. Every single pattern indicates root cause and suggested fix.

The Four Leak Patterns: Checklist

Let’s discuss the four vital Leak Patterns that emerge specifically for Virtual Threads.

1.ThreadLocal Accumulation 

This is the general Virtual Thread leak pattern. ThreadLocal was intended for architectures with restricted threads. Consider a system with 200 platform threads as a maximum bound; in such a system, it is possible to have a maximum of 200 ThreadLocal entries. The Virtual Thread architecture is different. Every incoming request can be assigned its own thread and, in turn, its own ThreadLocal entry. Under heavy load, processing millions of ThreadLocal entries in memory occurs. A serious memory leak occurs when these entries are not cleared automatically. Such entries remain in the heap until the thread becomes inactive.

// DANGEROUS with Virtual Threads — unbounded growth
private static final ThreadLocal<UserContext> CTX = new ThreadLocal<>();
 
// SAFE — use ScopedValue (JEP 446, Java 21 preview / Java 23 final)
private static final ScopedValue<UserContext> CTX = ScopedValue.newInstance();
ScopedValue.where(CTX, userContext).run(() -> {
	// CTX is automatically released when the scope exits
});

How to detect: 

For a clear view of memory usage, take a heap dump under load and count the number of entries for ThreadLocal$ThreadLocalMap. The count should be near zero at Virtual Thread scale. A count in the thousands can be a warning sign of an increasing Virtual Thread memory leak via ThreadLocal.

An automated heap dump analyzer like HeapHero — Free Heap Dump Analyser is capable of detecting ThreadLocal$ThreadLocalMap growth and listing the top-ranking object chains without manual intervention.

2. Carrier Thread Pinning: the Performance Killer

This issue typically arises with limited platform threads (e.g., 200 Platform Threads). The JVM pins the Virtual Thread to its carrier thread the moment a VirtualThread performs a blocking I/O operation within a synchronized block. A carrier thread is an ordinary OS thread that handles Virtual Thread processing. During pinning, the carrier thread can’t be assigned to another task. Under peak load, this pinning behavior causes extended delays and latency spikes, leading to high performance issues similar to those seen in earlier Java releases.

// CAUSES PINNING — prevents carrier thread reuse during I/O
synchronized (lock) {
	result = database.query(sql); // blocks here
}
 
// FIX — use ReentrantLock, which Virtual Threads can park without pinning
private final ReentrantLock lock = new ReentrantLock();
lock.lock();
try {
	result = database.query(sql);
} finally {
	lock.unlock();
}

How to detect: 

The JVM flag -Djdk.tracePinnedThreads=full enables the JVM to log details about pinned threads. These logs clearly highlight the pinned Virtual Threads and the exact synchronized code that is holding the carrier thread, when analyzed using fastThread.io. It is better to run this analysis during load testing, not in production.

3. StructuredTaskScope Leak: Unclosed Scopes Hold References

Java 21’s StructuredTaskScope is the recommended API for concurrent processing with Virtual Threads. Each scope creates child Virtual Threads that hold references to their subtask objects. If StructuredTaskScope is not closed appropriately—either through try-with-resources or by explicitly calling close()—the subtask objects, their results, and any objects they reference remain on the heap indefinitely.

// LEAKS — scope never closed if exception thrown before close()
var scope = new StructuredTaskScope.ShutdownOnFailure();
scope.fork(() -> fetchUserData(userId));
scope.join(); // scope.close() never called if join() throws
 
// CORRECT — try-with-resources guarantees close()
try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
	scope.fork(() -> fetchUserData(userId));
	scope.join().throwIfFailed();
} // scope.close() called automatically

How to detect: 

In a heap dump (HeapHero), look for StructuredTaskScope instances still reachable from GC roots after request completion. In GC logs (-Xlog:gc*), a steadily rising old-generation occupancy that doesn’t reclaim between requests is the key signal. In code, flag every new StructuredTaskScope() not wrapped in try-with-resources as a leak risk. 

For a more detailed understanding of how structured concurrency works under the hood, Structured Concurrency in Java 23 covers the lifecycle model and explains why try-with-resources is the only safe pattern for production code.

4. Object Pool Mismatch: Pools Sized for Platform Threads

Connection pools, object pools, and ExecutorService configurations sized for platform-thread concurrency (e.g. maximumPoolSize=200) become a bottleneck under Virtual Thread workloads, where concurrent requests may scale up to tens of thousands. Virtual Threads queue behind the pool limit, holding their heap-allocated state — request objects, response buffers, context maps — while waiting. This typically floods the live heap with objects that the garbage collector cannot reclaim, because they are still accessible from the waiting Virtual Threads.

How to detect: 

Monitor jvm_memory_used_bytes{area=”heap”} via Prometheus during peak Virtual Thread concurrency. A heap that scales linearly with request rate, despite low per-request allocation, implies pool starvation causing heap retention. Reevaluate pool sizing based on observed Virtual Thread concurrency instead of historical platform-thread limits. 

To visualize heap and GC trends in real time, Visualising JVM Metrics with Prometheus and Grafana provides a step-by-step setup guide. You can also use GCeasy – GC Log Analyser to analyze GC log files and identify old-generation growth patterns that indicate pool starvation. For Spring Boot teams in particular, How to Configure Virtual Threads in Spring Boot provides recommended executor configurations for Virtual Thread workloads.

Diagnostic Toolchain for Project Loom Memory Management

Standard JVM diagnostic tools work with Virtual Threads but need configuration adjustments to capture Virtual Thread-specific information:

Tip:  Virtual Threads do not appear in traditional thread dumps by default. Use -Djdk.virtualThreadScheduler.parallelism=N to control carrier thread count, and always use JSON format thread dumps (jcmd Thread.dump_to_file -format=json) to capture the full Virtual Thread inventory. 

For teams that need a complete picture in one place, yCrash correlates heap dumps, GC logs, and thread dumps into a single automated root-cause report, eliminating the need to correlate across three separate tools. To go deeper into GC log analysis, the GC Log Analysis Using Deterministic AI webinar is a useful walkthrough of real production log patterns.

The Architect’s Checklist: Summary

Conclusion

Project Loom’s memory model is not a fault; it is a design shift that requires architects to revisit designs that were benign at the platform-thread scale but become risky at the Virtual Thread scale. All four of the leak patterns described above are avoidable before production with the diagnostic actions explained above. Teams moving to Java 21 Virtual Threads should assess these designs during staging, and configure their JVMs for GC and heap visibility, then replace synchronized blocks with ReentrantLock. Teams that proactively identify blocking operations can leverage Project Loom’s scalability advantages by avoiding these memory-related issues, which otherwise frequently worsen performance in production.

 To make end-to-end JVM incident diagnosis a fully automated process, yCrash can correlate heap, GC, and thread data into a single root-cause report. Deeper JVM knowledge can be built across a team through a JVM Performance Masterclass, whose curriculum includes GC tuning and troubleshooting, thread dump analysis, OutOfMemoryError troubleshooting, deadlocks, HTTP errors, and poor response time.

Exit mobile version