Overview of low-level concurrency guarantees of the Java Memory Model -- with some surprising examples of legal optimizations and their consequences.

1. CONCURRENCY
CONCEPTS IN JAVA
DOUGLAS Q. HAWKINS
LEAD JIT DEVELOPER
AZUL SYSTEMS
@dougqh
dougqh@gmail.com

2. TOPICS
NOT ABOUT
PARALLEL FORK JOIN
AKKA
EXECUTORS & FUTURES
…

3. CORE MEMORY MODEL
ATOMICITY
ORDERING
VISIBILITY
HISTORY & FUTURE OF THE MEMORY MODEL
TOPICS

4. WHO’S READ THIS?

5. YOU SHOULD READ THE FINE PRINT.
TLDR: “AS IF”
AS IF THERE’S ONE THREAD,
UNLESS YOU SAY OTHERWISE.

6. The behavior of threads, particularly
when not correctly synchronized, can
be confusing and counterintuitive.
“
“
Java 8 Language Specification
§ 17.4 pp 631

7. Immutable objects have a very compelling
list of positive qualities. Without question,
they are among the simplest and most robust
kinds of classes you can possibly build. When
you create immutable classes, entire
categories of problems simply disappear.
“ “
www.javapractices.com

8. ATOMICITYWHAT OPERATIONS ARE INDIVISIBLE?

9. SUCCINCT != ATOMIC
sharedX = 2L;
sharedX = 2L sharedX = -1L
set_hi sharedX, 0000 0000
set_lo sharedX, 0000 0002
set_hi sharedX, ffff ffff
set_lo sharedX, ffff ffff
thread 1 thread 2
JLS8 § 17.7 pp 658

10. SUCCINCT != ATOMIC
sharedX += 1;
localX = sharedX;
localX = localX + 1;
sharedX = localX;

11. SUCCINCT != ATOMIC
Point sharedPoint = new Point(x, y);
local1 = calloc(sizeof(Point));
local1.<init>(x, y);
Object.<init>();
this.x = x;
this.y = y;
sharedPoint = local1;

12. VISIBILITYWHAT CAN OTHER THREADS SEE?

13. NO GARBAGE VALUES
NO OUT OF “THIN AIR” VALUES
WILL SEE A ZERO-ISH VALUE
OR
AN ASSIGNED VALUE

14. NO *COMPLETELY* GARBAGE VALUES
TWO SEPARATE HALVES OF LONG OR DOUBLE
BUT
BOTH HALVES ARE ZERO OR ASSIGNED

15. ONLY ASSIGNED VALUES FOR FINALS
sharedPoint = new Point(x, y);
class Point {
final int x, y;
public Point(int x, int y) {
this.x = x;
this.y = y;
}
}
*
*After constructor return
JLS8 § 17.5 pp 652

16. NOT *ALL* VALUES ARE STORED
sharedX = 20;
sharedX = 40;
print(sharedX);
Dead Store

17. DATA DEPENDENCE
sharedX = 20;
sharedX = 40;
print(sharedX);
Dead Store
Data
Dependence

18. DATA DEPENDENCE
sharedX = 20;
sharedX = 40;
print(40);
Data
Dependence
Propagate
Data
Dead Stores

19. OKAY.

20. NOT *ALL* VALUES ARE STORED
sharedSum = 0;
for ( int x : array ) {
sharedSum += x;
}
INTERMEDIATE VALUES OPTIONAL
localSum = 0;
for ( int x : array ) {
localSum += x;
}
sharedSum = localSum;

21. WHAT?

22. THREAD PUBLISHING
METHOD PUBLISHES?
Thread.sleep NO
Thread.yield NO
Thread.join YES
Thread.isAlive YES*
* If Thread.isAlive returns false
JLS8 § 17.4 pp 636

23. NOT *ALL* VALUES ARE READ
xSquared = sharedX * sharedX
Can xSquared be 30?
local1 = sharedX
local2 = sharedX
xSquared = local1 * local2

24. NOT *ALL* VALUES ARE READ
Can xSquared be 30?
local1 = sharedX
local2 = sharedX
xSquared = local1 * local2
thread 1 thread 2
sharedX = 5;
local1 = sharedX; (5)
sharedX = 6;
local2 = sharedX; (6)
xSquared =
local1 * local2;
(30)

25. NOT *ALL* VALUES ARE READ
xSquared = sharedX * sharedX
local1 = sharedX
local2 = sharedX
xSquared = local1 * local2
local1 = sharedX
xSquared = local1 * local1
Redundant Load

26. GREAT!

27. IMPORTANT OPTIMIZATION
for ( int x: array ) {
… x …
}
for ( int i = 0; i < this.array.length; ++i ) {
… this.array[i] …
}
Object[] local = this.array;
for ( int i = 0; i < local.length; ++i ) {
… local[i] …
}

28. EXCELLENT!20+% FASTER

29. MAYBE NOT…
while ( !done ); local1 = !done;
while ( local1 );
Load False &
Loop Forever

30. To some programmers, this behavior
may seem broken. However, it should
be noted that this code is improperly
synchronized.
“
“
Java 8 Language Specification
§ 17.4 pp 638

31. MEMORY COHERENCE
Coherence is a guarantee that a
thread cannot see an old value for
the same memory location after
seeing a newer value.

32. TIME PARADOX
// sharedPoint1 might equal
// sharedPoint2
int i = sharedPoint1.x;
// another thread can change x
int j = sharedPoint2.x;
// allow old value or
// cannot optimize
int k = sharedPoint3.x;
Point
x
y
100
100
125
sharedPoint1
sharedPoint2
JLS8 § 17.4-C pp 638

33. The semantics of the Java programming
language allow compilers and micro-
processors to perform optimizations that
can interact with incorrectly synchronized
code in ways that can produce behaviors
that seem paradoxical.
“ “
Java 8 Language Specification
§ 17.4 pp 637

34. ORDERINGWHAT ORDER DOES IT RUN IN?

35. “PROGRAM ORDER”
sharedX = 2
sharedY = 3
if ( sharedX > 0 ) {
print(sharedX);
}
Control
Data
Data
DATA + CONTROL DEPENDENCE

36. DATA CONTROL DEPENDENCE+
sharedX = 2
start
end
print(x)
if ( sharedX > 0 )
true false
sharedY = 3

37. https://en.wikichip.org/wiki/intel/microarchitectures/kaby_lake
FETCH
RENAME & ALLOCATE
DECODE
ALLOCATION QUEUE
SCHEDULERS
ALU
VECT ALU
VECT SHIFT
VECT ADD
VECT MUL
FMA
DIV
BRANCH
ALU
FAST LEA
VECT ALU
VECT SHIFT
VECT ADD
VECT MUL
FMA
SLOW INT
SLOW LEA
ALU
FAST LEA
VECT ALU
VECT SHUFFLE
ALU & SHIFT
BRANCH
AGU
LOAD DATA
AGU
LOAD DATA
STORE DATA AGU
INTEL KABY LAKE
LOAD
BUFFER
STORE
BUFFER

38. INTEL KABY LAKE
https://en.wikichip.org/wiki/intel/microarchitectures/kaby_lake
FETCH
RENAME & ALLOCATE
DECODE
ALLOCATION QUEUE
SCHEDULERS
ALU
VECT ALU
VECT SHIFT
VECT ADD
VECT MUL
FMA
DIV
BRANCH
ALU
FAST LEA
VECT ALU
VECT SHIFT
VECT ADD
VECT MUL
FMA
SLOW INT
SLOW LEA
ALU
FAST LEA
VECT ALU
VECT SHUFFLE
ALU & SHIFT
BRANCH
AGU
LOAD DATA
AGU
LOAD DATA
STORE DATA AGU
LOAD
BUFFER
STORE
BUFFER
add reg0, 1 add reg1, 2

39. COOL.

40. BUT COMPLICATES
EVERYTHING.

41. STORE SOONER
local1 = calloc(sizeof(Point));
local1.<init>(…);
…
sharedPoint = local1;
sharedFoo = new Foo(…);
Data
Data

42. FREE TO REORDER
local1 = calloc(sizeof(Point));
sharedPoint = local1;
local1.<init>(…);
sharedFoo = new Foo(…);
Data
Data
Reordered!

43. *BROKEN* DOUBLE CHECKED LOCKING
static Singleton instance() {
if ( sharedInstance == null ) {
synchronized ( Singleton.class ) {
if ( sharedInstance == null ) {
sharedInstance = new Singleton();
}
}
}
return sharedInstance;
}
static Singleton instance() {
if ( sharedInstance == null ) {
synchronized ( Singleton.class ) {
if ( sharedInstance == null ) {
local = calloc(sizeof(Singleton));
sharedInstance = local;
local.<init>();
}
}
}
return sharedInstance;
}
sharedInstance is non-null,
but constructor hasn’t run.

44. PRODUCER CONSUMER
...
sharedData = ...;
sharedDone = true;
...
sharedDone = true;
sharedData = ...;
while ( !sharedDone );
print(sharedData);
Reorder!

45. AGAIN BLAME
HARDWARE.

46. RISC / ARM CISC / x86
Loads After Loads YES YES
Loads After Stores YES YES
Stores After Stores YES NO
Stores After Loads YES YES

47. ...
sharedData = …;
store_store_fence();
sharedDone = true;
while ( !sharedDone ) {
load_load_fence();
}
print(sharedData);
TO STOP REORDERING, USE A *FENCE*
PRODUCER CONSUMER

48. BLAME JAVA,
TOO.

49. RISC / ARM CISC / x86 Java
Loads After Loads YES YES YES
Loads After Stores YES YES YES
Stores After Stores YES NO YES
Stores After Loads YES YES YES

50. HOW DO YOU TELL
JAVA TO NOT REORDER?

51. SYNCHRONIZATION ACTIONS
VOLATILE
SYNCHRONIZED
FINAL / FREEZE
UNSAFE
ATOMICS
VAR HANDLES

52. FINAL & FREEZE
local1 = calloc(sizeof(Point));
local1.<init>(x, y);
Object.<init>();
this.x = x;
this.y = y;
freeze();
sharedPoint = local1;
Point sharedPoint = new Point(x, y);

53. FINAL & FREEZE
local1 = calloc(sizeof(Point));
local1.<init>(x, y);
Object.<init>();
this.x = x;
this.y = y;
OtherThread.see(this);
freeze();
sharedPoint = local1;
Point sharedPoint = new Point(x, y);
Free to Reorder!

54. VOLATILE
PRE-JAVA 5 JUST ATOMICITY FOR CATEGORY 2 TYPES:
WRITE “SYNCHRONIZES WITH”
*ALL* SUBSEQUENT READS
WRITE “HAPPENS BEFORE”
*ALL* SUBSEQUENT READS
JLS8 § 17.4.4 & 17.4.5 pp 642 & 643

55. VOLATILE
...
volatileData = …;
volatileDone = true;
while ( !volatileDone ) {
}
print(volatileData);
PRODUCER CONSUMER

56. ALSO FIXES DOUBLE CHECKED LOCKING
static Singleton instance() {
if ( sharedInstance == null ) {
synchronized ( Singleton.class ) {
if ( sharedInstance == null ) {
local = calloc(sizeof(Singleton));
local.<init>();
sharedInstance = local;
}
}
}
return sharedInstance;
}

57. RIGHT WAY TO DO LAZY SINGLETON IN JAVA
static Singleton instance() {
return Holder.INSTANCE;
}
static class Holder {
static final class Singleton INSTANCE = new Singleton();
}
CLASS INITIALIZATION IS ALREADY LAZY.

58. SUCCINCT *STILL* != ATOMIC
volatileX += 1;
localX = sharedX;
localX = localX + 1;
sharedX = localX;

59. fullFence fullFence
loadLoadFence
storeStoreFence
acquireFence
releaseFence
NOT *ENTIRELY* ACADEMIC
UNSAFE VARHANDLES

60. SYNCHRONIZED
synchronized (Foo.class) {
counter += 1;
}

61. WAIT & NOTIFY
synchronized (lock) {
sharedData = …;
sharedDone = true;
lock.notify();
}
synchronized (lock) {
while ( !sharedDone ) {
lock.wait();
}
print(sharedData);
}
PRODUCER CONSUMER

62. DOESN’T PRESERVE ORDER
synchronized (lock) {
sharedData = …;
sharedDone = true;
lock.notify();
}
synchronized (lock) {
while ( !sharedDone ) {
lock.wait();
}
print(sharedData);
}
PRODUCER CONSUMER
synchronized (lock) {
sharedDone = true;
sharedData = …;
lock.notify();
}
Reorder!

63. SPURIOUS NOTIFICATIONS
synchronized (lock) {
while ( !sharedDone ) {
lock.wait();
}
print(sharedData);
}
*CORRECT* CONSUMER
synchronized (lock) {
if ( !sharedDone ) {
lock.wait();
}
print(sharedData);
}
*INCORRECT* CONSUMER

64. LOCK COARSENING
synchronized ( buffer ) {
buffer.add(x);
}
foo = bar;
synchronized ( buffer ) {
buffer.add(y);
}
https://shipilev.net/blog/2016/close-encounters-of-jmm-kind/
synchronized ( buffer ) {
buffer.add(x);
foo = bar;
buffer.add(y);
}

65. COMPARE & SWAP
AtomicInteger atomic = new AtomicInteger(0);
atomic.getAndIncrement();
boolean applied = false;
do {
int value = sharedVar.get();
applied = sharedVar.compareAndSet(
value, value + 1);
} while ( ! applied );

66. JAVA.UTIL.CONCURRENT
ConcurrentLinkedQueue
ConcurrentHashMap
A
B
C
D
LinkedTransferQueue
head tail

67. CONCURRENCY CAN SEEM...
BROKEN
COUNTERINTUITIVE
PARADOXICAL
DESPITE THE RULES

68. Immutable objects have a very compelling
list of positive qualities. Without question,
they are among the simplest and most robust
kinds of classes you can possibly build. When
you create immutable classes, entire
categories of problems simply disappear.
“ “
www.javapractices.com

69. REFERENCES
JAVA CURRENCY IN PRACTICE
Brian Goetz et al
JAVA MEMORY MODEL PRAGMATICS
Aleksey Shipilëv
http://virtualjug.com/java-memory-model-pragmatics/
CLOSE ENCOUNTERS OF THE JMM KIND
Aleksey Shipilëv
https://shipilev.net/blog/2016/close-encounters-of-jmm-kind/

70. HISTORY
JAVA’S MEMORY MODEL IS FATALLY FLAWED
Bill Pugh
http://www.cs.umd.edu/~pugh/java/broken.pdf
Bill Pugh
FIXING JAVA’S MEMORY MODEL
http://www.cs.umd.edu/~pugh/jmm.pdf

71. AND OF COURSE