Guest lecture at Informatics Institute of Technology (IIT) on 24th November 2018 Covers following topics: Concurrency Processes and Threads Concurrent vs Parallel Programming Preemptive multitasking Context Switches Java Threads Thread Dumps Java Troubleshooting, Profiling, Monitoring and Management Tools Synchronization Thread Interference Memory Consistency Errors Synchronized Methods Intrinsic Locks and Synchronization Atomic Access Liveness Deadlock Starvation Livelock Flame Graphs

1. Java Threads
Lightweight Processes
M. Isuru Tharanga Chrishantha Perera, Technical Lead at WSO2, Co-organizer of Java Colombo Meetup

2. Notice
● Most of the content in this lecture were taken from “Lesson:
Concurrency” - The Java™ Tutorials
● https://docs.oracle.com/javase/tutorial/index.html
● https://docs.oracle.com/javase/tutorial/essential/concurrency/index.html

3. Concurrency
● Computes can do many things at once.
● Concurrent software: Software that can do multiple tasks simultaneously.
● Concurrently executing multiple tasks will help to utilize the CPUs
efficiently.
● Java supports concurrent programming.
● Java platform also has high-level concurrency APIs.

4. Processes and Threads
● Two basic units of execution In concurrent programming.
● A system usually has many active processes and threads.
● Nowadays, computers usually have multiple processors or processors
with multiple cores.
● Processes and Threads share the processing time of a single core with
time slicing (Preemptive multitasking)

5. Concurrent vs Parallel Programming
● Concurrent Programming
○ Concurrently executing multiple tasks during overlapping time periods.
○ Can share a single core by multiple task, but execution of the tasks do not happen at the
same instance.
● Parallel Programming
○ Execution of multiple tasks occurs at the same physical instant.
○ Impossible on a single core processor.

6. Preemptive multitasking
● Multitasking OS permits preemption of tasks.
● Preemption is the act of temporarily interrupting a task being carried out
by a computer system, without requiring its cooperation, and with the
intention of resuming the task at a later time. (Wikipedia)
● Preemption causes Context Switches.
● The period of time allowed to run in the system is called as “Time slice”.
● Current versions of Linux, Windows and MacOS support preemptive
multitasking.

7. Context Switches
● Context switch is the process of the storing the system state for a given
task, so that it can be paused and resume another task. (Multitasking)
● Context switch can also occur due to an interrupt (a signal to the
processor indicating an event that needs immediate attention). For
example, Disk I/O, Network I/O

8. Processes
● A process has a self-contained execution environment.
● A process generally has a complete, private set of basic run-time
resources.
● A process has its own memory space.
● JVM runs as a single process.

9. Threads
● Threads are sometimes called lightweight processes.
● Both processes and threads provide an execution environment.
● Threads require fewer resources than processes.
● Threads exist within a process — every process has at least one. Threads
share the process's resources, including memory and open files.
● A application has at least one thread.
● A thread has a priority and a state.
● A thread can be marked as a daemon.

10. Daemon Threads
● Daemon threads run in background.,
● The Java Virtual Machine continues to execute threads until either of the
following occurs:
○ The exit method of class Runtime has been called and the security manager has
permitted the exit operation to take place.
○ All threads that are not daemon threads have died, either by returning from the call to
the run method or by throwing an exception that propagates beyond the run method.

11. Java Thread Objects
● In Java, each thread is associated with an instance of the class Thread
(java.lang.Thread).
● There are two ways to use Threads in a concurrent application.
○ Instantiate Thread class directly to initiate an asynchronous task. Here, we need to
control thread creation and management.
○ Use Java Executor to execute tasks and manage threads.

12. Defining a Thread
● There are two ways to define a Thread in Java.
● Provide a Runnable object to Thread constructor.
● Create a subclass of Thread and provide an implementation of run
method.
● Invoke Thread.start in order to start the new thread.

13. ● A class can only have one super class. Extending the Thread class will not
allow to extend other base classes.
● When using Runnable interface, the class can extend another base class.
Thread class vs Runnable interface

14. Java Thread States
● A thread can be in one of the following states:
○ NEW: A thread that has not yet started is in this state.
○ RUNNABLE: A thread executing in the Java virtual machine is in this state.
○ BLOCKED: A thread that is blocked waiting for a monitor lock is in this state.
○ WAITING: A thread that is waiting indefinitely for another thread to perform a particular
action is in this state.
○ TIMED_WAITING: A thread that is waiting for another thread to perform an action for up
to a specified waiting time is in this state.
○ TERMINATED: A thread that has exited is in this state.

15. Java Thread States
and Life Cycle
● A thread can be in only
one state at a given
point in time.
● These states are virtual
machine states which
do not reflect any
operating system
thread states.
https://www.uml-diagrams.org/java-thread-uml-state-machine-diagr
am-example.html?context=stm-examples

16. Thread Priorities
● Each Java thread has a priority that helps the operating system
determine the order in which threads are scheduled.
● Threads with higher priority should be given more processor time.
● However, thread priorities will not guarantee the order of threads
execution.

17. Thread Group
● A thread group represents a set of threads.
● In addition, a thread group can also include other thread groups.
● The thread groups form a tree in which every thread group except the
initial thread group has a parent.
● A thread is allowed to access information about its own thread group, but
not to access information about its thread group's parent thread group or
any other thread groups.

18. JDK Tools and Utilities
● Basic Tools (java, javac, jar)
● Security Tools (jarsigner, keytool)
● Java Web Services Tools (wsimport, wsgen)
● Java Troubleshooting, Profiling, Monitoring and Management Tools
(jcmd, jconsole, jmc, jvisualvm)

19. Java Troubleshooting,Profiling,Monitoring and
Management Tools
● jcmd - JVM Diagnostic Commands tool
● jconsole - A JMX-compliant graphical tool for monitoring a Java
application
● jvisualvm – Provides detailed information about the Java application. It
provides CPU & Memory profiling, heap dump analysis, memory leak
detection etc.
● jmc – Tools to monitor and manage Java applications without introducing
performance overhead

20. Java Experimental Tools
● Monitoring Tools
○ jps – JVM Process Status Tool
○ jstat – JVM Statistics Monitoring Tool
● Troubleshooting Tools
○ jmap - Memory Map for Java
○ jhat - Heap Dump Browser
○ jstack – Stack Trace for Java

21. Thread Dumps
● “A thread dump is a snapshot of the state of all threads that are part of
the process.”
● The state of the thread is represented with a stack trace.
● A thread can be in only one state at a given point in time.

22. How to take a Thread Dump
● jstack <pid> > thread-dump.txt
● jcmd <pid> Thread.print > thread-dump.txt
● kill -SIGQUIT <pid> #Same as kill -3
● Use jps/jcmd/ps -ef | grep java to find <pid>

23. Analyzing Thread Dumps
● Use a text editor. Thread dump format is very easy to understand.
● ThreadLogic
● IBM Thread and Monitor Dump Analyzer for Java
● Generate Flame Graphs with multiple thread dumps

24. “main”thread
● A simple Hello World Java program will have the main thread and system
threads.
● jcmd HelloWorld Thread.print
import java.util.Scanner;
public class HelloWorld {
public static void main(String[] args) {
System.out.println("Hello World!");
System.out.println("Press "ENTER" to continue...");
Scanner scanner = new Scanner(System.in);
scanner.nextLine();
}
}

25. Thread Sleep
● Thread.sleep can be used to suspend the execution for a specific period.
● Makes the processor time available to the other threads.
● Sleep period can be terminated by interrupts.
● Thread.sleep throws InterruptedException.
● Should not assume that Thread.sleep will suspend the thread for the
exact time period given.

26. Interrupts
● An interrupt is an indication to a thread that it should stop its current
work and do something else.
● Programmer decides how the thread should respond to an interrupt.
Usually threads are terminated by an interruption.
● Some methods like Thread.sleep throw InterruptedException to cancel
their current operation and return as soon as an interrupt is received.
● Use Thread.interrupted to see whether an interrupt has been received.

27. Thread Join
● Thread.join method allows one thread to wait for the completion of
another.
● For example, t.join() will cause the current thread to pause execution
until t’s thread terminates.
● There are join() overloads to specify a waiting period.
● Join also responds to an interrupt by throwing InterruptedException.

28. The SimpleThreads Example
● https://docs.oracle.com/javase/tutorial/essential/concurrency/simple.html

29. Synchronization
● Threads can communicate by sharing access to fields and the objects.
● However, there can be two kinds of errors.
○ Thread interference
○ Memory consistency errors
● These errors can be prevented by “Synchronization”

30. Thread Interference
● Interference happens when two operations, running in different threads,
but acting on the same data, interleave.
● The two operations consist of multiple steps, and the sequences of steps
overlap.
● One statement can be translated to multiple steps. For eg: c++ expression
decomposes into following steps.
○ Retrieve the current value of c.
○ Increment the retrieved value by 1.
○ Store the incremented value back in c.

31. Memory Consistency Errors
● Memory consistency errors occur when different threads have
inconsistent views of what should be the same data.
● Can be avoided, if you understand the happens-before relationship.
● This relationship is simply a guarantee that memory writes by one specific
statement are visible to another specific statement.
● Synchronization creates a happens-before relationship.

32. Synchronized Methods
● Add the synchronized keyword to method declaration.
● It is not possible for two invocations of synchronized methods on the
same object to interleave.
● When a synchronized method exits, it automatically establishes a
happens-before relationship with any subsequent invocation of a
synchronized method for the same object.
● Synchronized methods enable a simple strategy for preventing thread
interference and memory consistency errors.

33. Intrinsic Locks and Synchronization
● Synchronization is built around an internal entity known as the intrinsic
lock or monitor lock.
● With intrinsic locks, you can
○ Enforce exclusive access to an object's state
○ Establish happens-before relationships
● Every object has an intrinsic lock associated with it. A thread should
acquire an object’s intrinsic lock to get exclusive and consistent access to
an object’s fields.
● When a thread owns the lock, no other thread can acquire the same lock.

34. Locks In Synchronized Methods
● When a synchronized method is invoked by a thread, the thread
automatically acquired the intrinsic lock for that method’s object.
● The intrinsic lock is released when the method returns.
● For static synchronized methods, the thread can acquire the intrinsic
lock for the Class object associated with the class.

35. Synchronized Statements
● Another way to create synchronized code.
● Synchronized statements must specify the object that provides the
intrinsic lock.

36. Reentrant Synchronization
● Remember that a thread cannot acquire a lock owned by another thread.
But, a thread can acquire a lock that it already owns.
● Allowing a thread to acquire the same lock more than once enables
reentrant synchronization.

37. Atomic Access
● An atomic action is one that effectively happens all at once.
● An atomic action,
○ Cannot stop in the middle
○ Either happens completely or it doesn’t happen at all.
● Atomic actions cannot be interleaved. Therefore, there is no thread
interference. However, memory consistency errors are still possible. Use
volatile keyword to reduce the risk of memory consistency errors.
● Any write to a volatile variable establishes a happens-before relationship
with subsequent reads of that same variable.

38. Liveness
● A concurrent application's ability to execute in a timely manner is known
as its liveness.
● Liveness problems:
○ Deadlock
○ Starvation
○ Livelock

39. Deadlock
● Deadlock describes a situation where two or more threads are blocked
forever, waiting for each other.

40. Starvation
● Starvation describes a situation where a thread is unable to gain regular
access to shared resources and is unable to make progress.
● This happens when shared resources are made unavailable for long
periods by "greedy" threads.

41. Livelock
● A thread often acts in response to the action of another thread. If the
other thread's action is also a response to the action of another
thread, then livelock may result.
● As with deadlock, livelocked threads are unable to make further progress.
However, the threads are not blocked — they are simply too busy
responding to each other to resume work.

42. Guarded Blocks
● Threads often have to coordinate their actions.
● The most common coordination idiom is the guarded block.
● Such a block begins by polling a condition that must be true before the
block can proceed.
● Looping until the condition is satisfied will work, but it wastes processor
time!
● Invoke Object.wait to suspend the current thread and wait until another
thread has issued a notification.

43. Immutable Objects
● An object is considered immutable if its state cannot change after it is
constructed.
● Maximum reliance on immutable objects is widely accepted as a sound
strategy for creating simple, reliable code.
● Immutable objects are particularly useful in concurrent applications.
Since they cannot change state, they cannot be corrupted by thread
interference or observed in an inconsistent state.

44. High Level Concurrency Objects
● Lock objects support locking idioms that simplify many concurrent
applications.
● Executors define a high-level API for launching and managing threads.
Executor implementations provided by java.util.concurrent provide thread
pool management suitable for large-scale applications.
● Concurrent collections make it easier to manage large collections of data,
and can greatly reduce the need for synchronization.
● Atomic variables have features that minimize synchronization and help
avoid memory consistency errors.
● ThreadLocalRandom (in JDK 7) provides efficient generation of
pseudorandom numbers from multiple threads.

45. Executors
● Thread Pools are the most common kind of executor implementation.

46. Troubleshooting Issues with Thread Dumps
● High CPU usage
○ Find out the the thread(s) using high CPU using (ps or top commands)
○ Convert Thread ID (Light Weight Process) to Hexadecimal
○ Take thread dump and find the thread with the specific nid (Native ID)

47. Troubleshooting Issues with Thread Dumps
● Application slowdown
○ Find “BLOCKED” or “WAITING” threads in thread dump
● Application hangs
○ Look for deadlocks

48. Some Tips for thread dump analysis
● Make sure all threads have meaningful names. Use a custom
ThreadFactory for thread pools
● Threads “WAITING (parking)” for a task in a thread pool is not a major
issue.

49. Flame Graphs
● Flame graphs are a visualization of profiled software, allowing the most
frequent code-paths to be identified quickly and accurately.
● Flame Graphs can be generated using
https://github.com/brendangregg/FlameGraph
○ This creates an interactive SVG
http://www.brendangregg.com/flamegraphs.html

50. Types of Flame Graphs
● CPU
● Memory
● Off-CPU
● Hot/Cold
● Differential

51. Flame Graph: Definition
● The x-axis shows the stack profile population, sorted alphabetically
● The y-axis shows stack depth
○ The top edge shows what is on-CPU, and beneath it is its ancestry
● Each rectangle represents a stack frame.
● Box width is proportional to the total time a function was profiled directly
or its children were profiled

52. FlameGraph from Thread Dumps
i=0; while (( i++ < 200 )); do jstack PID >> out.jstacks;
sleep 10; done
cat out.jstacks | ./stackcollapse-jstack.pl >
out.stacks-folded