This webinar by Andrii Rodionov (JUG UA leader) was delivered at Java Community Webinar #6 on February 9, 2021. In this talk, we have discussed Project Loom and 'virtual threads' usage - a new high-throughput lightweight concurrency model in the Java platform. In addition, we understood how Project Loom can help you write efficient code in a traditional blocking style without switching to reactive frameworks. More details and video: https://www.globallogic.com/ua/about/events/java-community-webinar-6/ Join our community: https://www.facebook.com/groups/GLJavaCommunity/

1. Project Loom
OR
How to Be Happy With Blocking
Code
@AndriiRodionov

2. https://blog.softwaremill.com/synchronous-or-asynchronous-and-why-wrestle-with-wrappers-2c5667eb7acf

3. https://www.baeldung.com/thread-pool-java-and-guava
BlockingQueue

4. Blocking request processing!

5. Problems

6. Problems
• Wasting thread-pool threads
• Wasting memory

7. Solutions
• Add more threads to thread pool?
• To be non-blocking?
• Project Loom?

8. Million Threads Baby
Why it fails?

9. Non-blocking request processing

10. McDonald's Kiosks

11. What means to be non-blocking
• An “event” should trigger a code execution -
callback
• How we can describe or register this callback?
– Callbacks
– Wrappers
– Coroutines

12. https://blog.softwaremill.com/synchronous-or-asynchronous-and-why-wrestle-with-wrappers-2c5667eb7acf

13. Future -> CompletableFuture
Java 5:
• Future<T>.get() - blocking
Java 8:
• CompletableFuture<T>.thenApply(...).... - non-blocking
• ~70 CompletableFuture methods to be non-blocking

14. Code readability
• Statements is now encoded as method calls
HttpClient.newBuilder()
.build()
.sendAsync(request,
HttpResponse.BodyHandlers.ofString())
.thenAccept(response -> . . .);
.thenApply(. . .);
.exceptionally(. . .);
• If we loved this style of programming, we would not have statements
in our programming language and code in Lisp

15. To be Reactive or not to be?
• If your Spring MVC application works fine - don’t touch!
• Imperative programming is the easiest way to write,
understand, and debug code
• Reactive and non-blocking generally do not make
applications run faster
Spring-framework-reference

16. To be Reactive or not to be?
• Reactive in some cases (WebClient to execute remote calls in
parallel) can make applications faster
• On the whole, it requires more work and can increase slightly
the required processing time

17. To be Reactive or not to be?
The key benefit of reactive and non-blocking is:
• the ability to scale with a small,
• fixed number of threads
• and less memory

18. Back to Green threads

19. Green Threads
– class Thread - @since 1.0
– M to N (M:1) mapping
– Threads scheduling - Java
– stop(), destroy(), resume(), suspend()
– When a green thread blocked, its carrier thread was also
blocked, preventing all other green threads from making
progress
– Loom is a (partial) return to the idea of green threads

20. https://www.youtube.com/watch?v=oBMZc2VMujw

21. class Thread
– At Sun’s Solaris OS with Java 1.2/1.3 - 1:1 threads mapping
– Threads scheduling - OS
– stop(), destroy(), resume(), suspend()
• @Deprecated(since="1.2", forRemoval=true)
– In 1998 the Sun Java Web Server (the precursor of Tomcat)
run each request in a separate thread, and not an OS process
– Able to serve thousands of concurrent requests this way

22. https://www.youtube.com/watch?v=oBMZc2VMujw

23. 1:1 costs
• Native thread creation time
– Native thread
• Thread stack size - 1-2Mb
• Native threads time-slice scheduling
– Context switching (Memory/Cache/Registers)

24. Problems with blocking
Imperative programming
Problems:
• Wasting thread-pool threads
• Wasting memory
Project Loom way:
• Do not use thread pool
• Make thread instantiation fast
• Do not consume much memory

25. Virtual threads
• Exists only in Java runtime
– no 1-to-1 wrappers over OS threads
• Creating a virtual thread is cheap and fast
– they should never be pooled
– no OS-level data structures for the stack - no memory wasting
• Blocking a virtual thread is cheap
• Threads scheduling - Java
– no OS context switching

26. https://www.youtube.com/watch?v=oBMZc2VMujw

27. Carrier threads
• Virtual threads must be attached to OS thread to execute
• Carrier threads - OS threads upon which a virtual thread
executes
• Over virtual thread lifetime, a single virtual thread may
run on several different carrier threads
• When a virtual thread blocks (when a blocking call such as
I/O is made), it is “parked” and another virtual thread runs
on the carrier thread

28. https://lukstei.com/2020-07-25-a-first-look-into-the-project-loom-in-java/
Virtual thread 1
…
parked/blocked
Virtual thread 2
…
Virtual thread 1
…
Carrier
Thread 1
Carrier
Thread 2
Stack is saved
Stack is restored,
execution continues
Time
Carrier threads

29. Virtual thread creation
• class Thread
Thread thread = Thread.startVirtualThread(runnable);
• unbounded executor -
ExecutorService exec =
Executors.newVirtualThreadExecutor();
exec.submit(runnable1);
exec.submit(runnable2);

30. Demo
1M threads example
Sleeping threads

31. https://blog.softwaremill.com/synchronous-or-asynchronous-and-why-wrestle-with-wrappers-2c5667eb7acf

32. Virtual threads scheduler
• The default scheduler is the work-stealing scheduler from
ForkJoinPool
• Work-stealing schedulers work well for threads involved:
– that normally process in short bursts and block often
• Long running (without blocking) virtual threads can be pinned
to its carrier
– none of the existing schedulers for virtual threads uses
time slices to preempt virtual threads

33. Demo
Pinned threads

34. STRUCTURED CONCURRENCY
• When multiple threads run without coordination, it’s spaghetti
code
• Should use different approach to coordinate them
• In the 1960s, structured programming replaced goto with branches,
loops, and functions

35. STRUCTURED CONCURRENCY
• Structured concurrency should do the same for concurrent tasks
• We should know, from reading the program text, when they all
finish
https://vorpus.org/blog/notes-on-structured-concurrency-or-
go-statement-considered-harmful

36. Demo
Structured concurrency
try(var ex = Executors.newVirtualThreadExecutor()) {
ex.submit(() -> run());
ex.submit(() -> run());
} // Blocks and waits

37. Open questions
• Thread local
• Thread dump
• Threads communication
– BlockingQueue
– Channel (Carrier)
• Structured concurrency
• Supported API
– https://wiki.openjdk.java.net/display/loom/Blocking+Operations
– except synchronized/wait/notify and local file I/O operations

38. Cheap threads
can do expensive things

39. 1M virtual threads illusion
• Loom does allow you to have many threads, even 1,000,000 threads,
• but ...
• not if those threads have deep stacks

40. Stack Trace From Hell
https://dzone.com/articles/filtering-
stack-trace-hell

41. 1M virtual threads illusion
For deep stack it increase
• total memory usage for stack entries and
• also comes at a cost of long garbage collections
https://webtide.com/do-looms-claims-stack-up-part-1/

42. Why Thread Pools?
- Why Small Thread Pools?
- We can create up to 50K native threads in a pool!
- A limited thread pool is a coarse grained limit on all
resources, not only threads
- Limiting the number of threads puts a limit on concurrent
lock contention, memory consumption and CPU usage
https://webtide.com/do-looms-claims-stack-up-part-2/

43. Conclusion
• blocking code is much easier to write
• virtual threads are very fast to start and cheap to block
• for “real world tasks” we should anyway limit the number of
virtual threads

44. 1994
“He (Bill Joy) would often go on at length about how
great Oak would be if he could only add closures and
continuations and parameterized types”
“While we all agreed these were very cool language
features, we were all kind of hoping to finish this
language in our lifetimes and get on to creating cool
applications with it”
“It is also interesting that Bill was absolutely right about
what Java needs long term. When I go look at the list of
things he wanted to add back then, I want them all. He
was right, he usually is”
Patrick Naughton,
one of the creators of the Java

45. To read
• https://inside.java/tag/loom
• https://www.javaadvent.com/2020/12/project-loom-and-
structured-concurrency.html
• https://blogs.oracle.com/javamagazine/going-inside-javas-project-
loom-and-virtual-threads
• https://blog.softwaremill.com/will-project-loom-obliterate-java-
futures-fb1a28508232
• https://webtide.com/do-looms-claims-stack-up-part-1/
• https://vorpus.org/blog/notes-on-structured-concurrency-or-go-
statement-considered-harmful/

47. Comparing APIs
API Type No value Single value Multiple values
Standard Java –
Synchronous APIs
void T Iterable<T>
Standard Java –
Asynchronous APIs
CompletableFuture
<Void>
CompletableFuture
<T>
CompletableFuture
<List<T>>
Project Reactor
implementation of
Reactive Streams
Mono<Void> Mono<T> Flux<T>
https://devblogs.microsoft.com/azure-sdk/async-programming-with-project-reactor/