Virtual threads, also known as project Loom, are a new feature available in Java 21 that aims to provide lightweight, efficient, and scalable concurrency. They are designed to improve the performance and efficiency of concurrent applications by reducing the overhead associated with traditional thread-based concurrency. At the same time, they allow write asynchronous code in a much simpler way than reactive programming.
Virtual Threads vs Native Threads
Before Virtual Threads, every instance of the Thread Class class was a Java Native Thread, a wrapper around an OS thread. Therefore, each time you create a Java Thread you also create an OS Thread.
For example, if you execute the “top -H <PID>” command against a Java process, you can see each Java Thread has its own PID:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 59268 frances+ 20 0 1854736 440448 28748 S 0.0 0.7 0:00.05 GC Thread#0 59269 frances+ 20 0 1854736 440448 28748 S 0.0 0.7 0:00.00 G1 Main Marker 59270 frances+ 20 0 1854736 440448 28748 S 0.0 0.7 0:00.14 G1 Conc#0 59271 frances+ 20 0 1854736 440448 28748 S 0.0 0.7 0:00.00 G1 Refine#0 59272 frances+ 20 0 1854736 440448 28748 S 0.0 0.7 0:00.02 G1 Service 59273 frances+ 20 0 1854736 440448 28748 S 0.0 0.7 0:00.02 VM Thread . . . . .
Creating a Java Native thread, however, creates an OS thread, and blocking a Native thread blocks an OS thread.
What are Virtual Threads? In a nutshell, they are lightweight, JVM-managed threads. They extend the Thread class but are not bound to one specific OS thread. Virtual Threads are implemented on top of platform threads, and managed by a dedicated ForkJoinPool maintained by the JVM.
When you create a Virtual Thread, the JVM schedules its execution on a platform thread. The stack chunk for the virtual thread is temporarily copied from the heap to the stack of the platform thread. This platform thread becomes the Carrier thread for the virtual thread.
The first time a virtual thread reaches a blocking operation, such as waiting for I/O, the carrier thread is released.
That’s the crucial part. The stack chunk of the virtual thread is then copied back to the heap. This allows the carrier thread to execute other eligible virtual threads while the blocked virtual thread is waiting.
Once the Virtual Thread completes the blocking operation, the scheduler schedules it again for execution. The execution of the virtual thread can continue on the same carrier thread or a different one, depending on the availability of platform threads in the pool.
Benefits of Java Virtual Threads
The use of virtual threads provides several benefits.
- Firstly, they are lightweight, meaning that the overhead of creating and managing virtual threads is significantly lower compared to traditional threads. This allows applications to create a large number of virtual threads without incurring excessive memory or CPU usage.
- Secondly, virtual threads are highly scalable. The JVM dynamically adjusts the number of platform threads in the pool based on the workload, ensuring optimal utilization of system resources.
- Lastly, virtual threads simplify the programming model for concurrent applications. It’s fairly easier to develop using Virtual Threads than developing Reactive application. As a matter of fact, you can plug Virtual Threads into existing Java concurrency APIs, such as CompletableFuture and ExecutorService, without requiring significant code changes.
An example of Virtual Threads
As for Java Platform Threads, there is no single way to create a Virtual Threads. You can use one of these approaches:
- Use the Thread and Thread.Builder APIs to create virtual threads.
- Use the java.util.concurrent.Executors class to create an ExecutorService that starts a new virtual thread for each task.
For example, the following code shows how to use the Thread.Builder API to create a Virtual Thread:
Thread.Builder builder = Thread.ofVirtual().name("Virtual Thread");
Runnable thread = () -> {
System.out.println("I'm running a Virtual thread");
};
Thread t = builder.start(thread);
t.join();
On the other hand, Executors let you to separate thread management and creation from the rest of your application.
This is a complete example that creates an ExecutorService with the Executors.newVirtualThreadPerTaskExecutor() method.
public class DemoVirtualThread {
public static void main(String[] args) {
Runnable runnable = () -> {
System.out.println("Hello, world!");
try {
Socket socket = new Socket();
socket.connect(new InetSocketAddress("nonexistinghost", 80), 1000); // Timeout set to 1 second
} catch (Exception exc) {
}
};
try (ExecutorService executorService = Executors.newVirtualThreadPerTaskExecutor()) {
for (int i = 0; i < 5000; i++) {
executorService.submit(runnable);
}
}
}
}
In this example, we are creating 5000 Virtual Threads through the Executors.newVirtualThreadPerTaskExecutor() . The implementation of the Virtual Thread is inside the Runnable Lambda Expression. We are basically opening a Socket to a non-existing Host so that the Operation will block for 1 Seconds. (It will eventually timeout).
Virtual Threads vs Platform Thread performance
We will now run the above code and monitor the JVM with a JVisualVM. Please note that you need JVisual 2.1.7 or higher to have support for Java 21: https://visualvm.github.io/
Here is the Overview of the Thread, CPU and Memory of the above Java Virtual Thread example:
As you can see, a Total number of just 29 Threads were started. The execution had a minimal CPU usage and the JVM Used memory reached about 52 MB .
Compare with the following example which uses a standard Threading model, to start 5000 Java Native Threads:
ExecutorService executorService = Executors.newFixedThreadPool(5000);
for (int i = 0; i < 5000; i++) {
executorService.execute(() -> {
System.out.println("Hello from thread: " + Thread.currentThread().getId());
try {
Socket socket = new Socket();
socket.connect(new InetSocketAddress("nonexistinghost", 80), 1000); // Timeout set to 1 second
} catch (Exception exc) {
System.out.println(exc.getMessage());
}
});
}
executorService.shutdown();
Here is the outcome of running the above example with the standard Platform Threads:
There is a clear difference in terms of number of Threads. Also, the difference in terms of memory is significant. This is due to the following reasons:
- Thread Metadata: Every Java Native thread has some metadata associated with it, which includes information like the thread’s ID, program counter, stack pointer, and various control flags. The amount of memory required for this metadata is in the range of a few kilobytes. That, however, can become a bottleneck when you are using an huge number of Native Threads
- Thread Stack: The primary memory allocation for a Java Native thread is its stack. The Java Virtual Machine (JVM) allocates a fixed amount of memory for each thread’s stack. The size of the stack is configurable using JVM parameters, such as
-Xssor-XX:ThreadStackSize. The default stack size varies between JVM implementations and versions, but it’s typically in the range of 512KB to 2MB.
On the other hand, Virtual Threads make better usage of memory as there is no 1:1 relationship with OS Threads and multiple Threads can be multiplex on the same carrier Thread.
Introspection of Virtual Threads
A Virtual Thread can produce a Stack Trace just like a Native Thread, therefore you will be able to capture the Thread stack trace with the common e.printStackTrace() statement.
On the other hand, if you try to run the Virtual Thread Example in this tutorial you will see that the Thread name (Thread.currentThread().getName()) returns an empty String.
To work around this issue, you can build a new Executor from the ThreadFactory interface as follows:
final ThreadFactory factory = Thread.ofVirtual().name("virtual-thread-", 0).factory();
try (ExecutorService executorService = Executors.newThreadPerTaskExecutor(factory)) {
for (int i = 0; i < 5000; i++) {
executorService.submit(runnable);
}
}
By using this code, you can create a ThreadFactory that generates virtual threads with names following the pattern “virtual-thread-“. The generated thread names can help you identify and track the execution of different virtual threads in your application.
For example, here is the combined output of the Thread.currentThread().getName() and e.printStackTrace() :
virtual-thread-4999 nonexistinghost java.net.UnknownHostException: nonexistinghost at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:567) at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327) at java.base/java.net.Socket.connect(Socket.java:751) at DemoVirtualThread2.lambda$main$0(DemoVirtualThread2.java:13) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) at java.base/java.lang.VirtualThread.run(VirtualThread.java:311)
Pitfalls of using Virtual Threads
Although Virtual Threads are a huge addition to simplify the design and execution of reactive applications, there are a few aspects to consider before adding them blindly in your core.
- Firstly, consider that you need to use a Virtual Thread in an asynchronous context. On the other hand, if your Virtual Thread executes a synchronized method (or block), it will retain control over the underlying OS thread. Consequently, it will be pinned to its carrier, so if you perform a block operation there, you are also blocking the carrier.
- Then, consider that with a large context switch scenario, such as thousands of Threads, the virtual thread can be assigned to a different Carrier before it completes its job. That can eventually cause Thread cache misses as Thread local structures needs to be recreated each time the Carrier changes.
- Finally, consider that virtual threads are a good option for IO bound applications but not for CPU bound applications. The reason is that Virtual Threads are not pre-emptive by design. They will perform their task until it’s complete. The ideal scenario would be to wait for some incoming IO. On the other hand, if your Virtual Thread will just perform CPU intensive operations, they cannot be discontinued. Therefore, you will eventually lose fairness in your application processes.
As final note, you can detect which Native Threads are pinned by adding the jdk.tracePinnedThreads start up parameter which can have the following verbosity:
-Djdk.tracePinnedThreads=full -Djdk.tracePinnedThreads=short
Conclusion
In conclusion, Java Virtual Threads, represent a significant advancement in Java concurrency. They offer a lightweight, efficient, and scalable solution for managing concurrency in Java applications. By reducing the overhead associated with traditional thread-based concurrency, Virtual Threads enhance performance and resource efficiency while simplifying the development of concurrent applications.
However, it’s essential to use Virtual Threads judiciously and consider their limitations. They are not suitable for CPU-bound tasks, and you should be cautious about using them in synchronous or blocking contexts, as they can lose some of their benefits in such scenarios.
Useful links and references:
https://github.com/quarkusio/quarkus/blob/main/docs/src/main/asciidoc/virtual-threads.adoc
https://www.youtube.com/watch?v=YRn10DR1sDM
Found the article helpful? if so please follow us on Socials
