Programming Language Memory Models: What do Shared Variables Mean?

1. Programming Language Memory Models: What do Shared Variables Mean? Hans-J. Boehm HP Labs 7/28/2011 1

2. The problem • Shared memory parallel programs are built on shared variables visible to multiple threads of control. • But there is a lot of confusion about what those variables mean: – Are concurrent accesses allowed? – What is a concurrent access? – When do updates become visible to other threads? – Can an update be partially visible? • Many recent efforts with serious technical issues: – Java, OpenMP 3.0, UPC, Go … 7/28/2011 2

3. Credits and Disclaimers: • Much of this work was done by others or jointly. I‟m relying particularly on: – Basic approach: Sarita Adve, Mark Hill, Ada 83 … – JSR 133: Also Jeremy Manson, Bill Pugh, Doug Lea – C++11: Lawrence Crowl, Clark Nelson, Paul McKenney, Herb Sutter, … – Improved hardware models: Peter Sewell‟s group, many Intel, AMD, ARM, IBM participants … – Conflict exception work: Ceze, Lucia, Qadeer, Strauss – Recent Java Memory Model work: Sevcik, Aspinall, Cenciarelli – Race-free optimization: Pramod Joisha, Laura Effinger-Dean, Dhruva Chakrabarti • But some of it is still controversial. – This reflects my personal views. 7/28/2011 3

4. Outline • Emerging consensus: – Interleaving semantics (Sequential Consistency) – But only for data-race-free programs • Brief discussion of consequences – Software requirements – Hardware requirements • Major remaining problem: – Java can‟t outlaw races. – We don‟t know how to give meaning to data races. – Some speculative solutions. 7/28/2011 4

5. Naive threads programming model (Sequential Consistency) • Threads behave as though their memory accesses were simply interleaved. (Sequential consistency) Thread 1 Thread 2 x = 1; y = 2; z = 3; – might be executed as x = 1; y = 2; z = 3; 7/28/2011 5

6. Locks restrict interleavings Thread 1 Thread 2 lock(l); lock(l); r1 = x; r2 = x; x = r1+1; x = r2+1; unlock(l); unlock(l); – can only be executed as lock(l); r1 = x; x = r1+1; unlock(l); lock(l); r2 = x; x = r2+1; unlock(l); or lock(l); r2 = x; x = r2+1; unlock(l); lock(l); r1 = x; x = r1+1; unlock(l); since second lock(l) must follow first unlock(l) 7/28/2011 6

7. Atomic sections / transactional memory are just like a single global lock. 7/28/2011 7

8. But this doesn‟t quite work … • Limits reordering and other hardware/compiler transformations – “Dekker‟s” example (everything initially zero) should allow r1 = r2 = 0: Thread 1 Thread 2 x = 1; y = 1; r1 = y; r2 = x; – Compilers like to move up loads. – Hardware likes to buffer stores. 7/28/2011 8

9. Sensitive to memory access granularity Thread 1 Thread 2 x = 300; x = 100; • If memory is accessed a byte at a time, this may be executed as: x_high = 0; x_high = 1; // x = 256 x_low = 44; // x = 300; x_low = 100; // x = 356; 7/28/2011 9

10. And this is at too low a level … • And taking advantage of sequential consistency involves reasoning about memory access interleaving: – Much too hard. – Want to reason about larger “atomic” code regions • which can‟t be visibly interleaved. 09/08/2010 10

11. Real threads programming model (1) • Two memory accesses conflict if they – access the same scalar object*, e.g. variable. – at least one access is a store. – E.g. x = 1; and r2 = x; conflict • Two ordinary memory accesses participate in a data race if they – conflict, and – can occur simultaneously • i.e. appear as adjacent operations (diff. threads) in interleaving. • A program is data-race-free (on a particular input) if no sequentially consistent execution results in a data race. * or contiguous sequence of bit-fields 09/08/2010 11

12. Real threads programming model (2) • Sequential consistency only for data-race- free programs! – Avoid anything else. • Data races are prevented by – locks (or atomic sections) to restrict interleaving – declaring synchronization variables • (in two slides …) 09/08/2010 12

13. Alternate data-race-free definition: happens-before • Memory access a happens-before b if – a precedes b in program order. Thread 1: – a and b are synchronization operations, and b observes the results of a, thus lock(m); enforcing ordering between them. t = x + 1; • e.g. a unlocks m, b subsequently locks m. – Or there is a c such that a happens-before x = t; c and c happens-before b. unlock(m) • Two ordinary conflicting memory operations a and b participate in a Thread 2: data race if neither lock(m); – a happens-before b, t = x + 1; – nor b happens-before a. • Set of data-race-free programs usually the x = t; same. unlock(m)

14. Synchronization variables • Java: volatile, java.util.concurrent.atomic. • C++11: atomic<int>, atomic_int • C1x: _Atomic(int), _Atomic int, atomic_int • Guarantee indivisibility of operations. • “Don‟t count” in determining whether there is a data race: – Programs with “races” on synchronization variables are still sequentially consistent. – Though there may be “escapes” (Java, C++11). • Dekker‟s algorithm “just works” with synchronization variables. 7/28/2011 14

15. As expressive as races Double-checked locking: Double-checked locking: Wrong! Correct (C++11): bool x_init; atomic<bool> x_init; if (!x_init) { if (!x_init) { l.lock(); l.lock(); if (!x_init) { if (!x_init) { initialize x; initialize x; x_init = true; x_init = true; } } l.unlock(); l.unlock(); } } use x; use x; 7/28/2011 15

16. Data Races Revisited • Are defined in terms of sequentially consistent executions. • If x and y are initially zero, this does not have a data race: Thread 1 Thread 2 if (x) if (y) y = 1; x = 1; 7/28/2011 16

17. SC for DRF programming model advantages over SC • Supports important hardware & compiler optimizations. • DRF restriction  Synchronization-free code sections appear to execute atomically, i.e. without visible interleaving. – If one didn‟t: Thread 1 (not atomic): Thread 2(observer): a = 1; if (a == 1 && b == 0) { b = 1; … } 09/08/2010 17

18. SC for DRF implementation model (1) • Very restricted reordering of lock(); memory operations around synch-free synchronization operations: code – Compiler either understands these, or region treats them as opaque, potentially updating any location. unlock(); – Synchronization operations include synch-free instructions to limit or prevent hardware reordering (“memory code fences”). region • e.g. lock acquisition, release, atomic store, might contain memory fences.

19. SC for DRF implementation model (2) • Code may be reordered lock(); between synchronization operations. synch-free – Another thread can only tell if it code accesses the same data between region reordered operations. – Such an access would be a data unlock(); race. synch-free • If data races are disallowed code (e.g. Posix, Ada, C++11, not Java), compiler may assume region that variables don‟t change asynchronously.

20. Data races Some variants are errors C++11 draft SC for DRF, X with explicit easily avoidable exceptions C1x draft Java SC for DRF, with some exceptions More details later. Ada83 SC for DRF (?) X Pthreads SC for DRF (??) X OpenMP SC for DRF (??, except atomics) X Fortran 2008 SC for DRF (??, except atomics) X C#, .Net SC for DRF when restricted to locks + Interlocked (???) 7/28/2011 20

21. The exceptions in C++11: atomic<bool> x_init; if (!x_init.load(memory_order_acquire) { l.lock(); if (!x_init.load(memory_order_relaxed) { initialize x; x_init.store(true, memory_order_release); } l.unlock(); } use x; We‟ll ignore this for the rest of the talk …

22. Data Races as Errors • In many languages, data races are errors – In the sense of out-of-bounds array stores in C. – Anything can happen: • E.g. program can crash, local variables can appear to change, etc. – There are no benign races in e.g. C++11 or Ada 83. – Compilers may, and do, assume there are no races. If that assumption is false, all bets are off. – See e.g. Boehm, How to miscompile programs with “benign” data races, HotPar „11. . 7/28/2011 22

23. How a data race may cause a crash unsigned x; • Assume switch statement compiled as If (x < 3) { branch table. • May assume x is in … // async x change range. switch(x) { • Asynchronous change to case 0: … x causes wild branch. case 1: … – Not just wrong value. case 2: … } } 23 28 July 2011

24. Note: We can usually ignore wait/notify • In most languages wait can return without notify. – “Spurious wakeup” • Except for termination issues: – wait() ≡ unlock(); lock() – notify() ≡ nop • Applies to existence of data races, partial correctness, flow analysis. 7/28/2011 24

25. Another note on data race definition • We define it in terms of scalar accesses, but • Container libraries should ensure that Container accesses don‟t race  No races on memory locations • This means – Accesses to hidden shared state (caches, allocation) must be locked by implementation. – User must lock for container-level races. • This is often the correct library thread-safety condition. 7/28/2011 25

27. Compilers must not introduce data races • Single thread compilers currently may add data races: (PLDI 05) struct {char a; char b} x; tmp = x; x.a = „z‟; tmp.a = „z‟; x = tmp; – x.a = 1 in parallel with x.b = 1 may fail to update x.b. • … and much more interesting examples. • Still broken in gcc with bit-fields. 7/28/2011 27

28. Some restrictions are a bit more annoying: • Compiler may not introduce “speculative” stores: int count; // global, possibly shared … for (p = q; p != 0; p = p -> next) if (p -> data > 0) ++count; int count; // global, possibly shared … reg = count; for (p = q; p != 0; p = p -> next) if (p -> data > 0) ++reg; count = reg; // may spuriously assign to count 28 28 July 2011

29. Introducing data races: Potentially infinite loops ? while (…) { *p = 1; } x = 1; x = 1; while (…) { *p = 1; } • x not otherwise accessed by loop. • Prohibited in Java. • Allowed by giving undefined behavior to such infinite loops in C++0x. (Controversial.) • Common text book analyses treat as equivalent? 7/28/2011 29

30. Note on program analysis for optimization • Lots of research papers on analysis of sequentially consistent parallel programs. • AFAIK: Real compilers don’t do that! – For good reason. – Use sequential optimization in sync-free regions. • SC analysis is – Generally sound, but pessimistic for C++11 (minus SC escapes) – Very much so with incomplete information – Unsound for Java (but …) 7/28/2011 30

31. Some positive optimization consequences • Rules are finally clear! – Optimization does not need to preserve full sequential consistency! • In languages that outlaw data races – Compilers already use that. – It can be leveraged more aggressively: – Assuming no other synchronization, x not motified by this thread, x is loop invariant in while (…) { … x … l.lock(); … l.unlock(); } 7/28/2011 31

32. Language spec challenge 1 • Some common language features almost unavoidably introduce data races. • Most significant example: – Detached/daemon threads, combined with – Destructors / finalizers for static variables – Detached threads call into libraries that may access static variables • Even while they‟re being cleaned up. 7/28/2011 32

33. Language spec challenge 2: • Some really awful code: Thread 2: Don’t try this at home!! Thread 1: x = 42; while (m.trylock()==SUCCESS) ? m.lock(); m.unlock(); assert (x == 42); • Disclaimer: Example requires • Can the assertion fail? tweaking to be pthreads- compliant. • Many implementations: Yes • Traditional specs: No. C++11: Yes • Trylock() can effectively fail spuriously! 09/08/2010 33

35. Byte store instructions • x.c = „a‟; may not visibly read and rewrite adjacent fields. • Byte stores must be implemented with – Byte store instruction, or – Atomic read-modify-write. • Typically expensive on multiprocessors. • Often cheaply implementable on uniprocessors. 7/28/2011 35

36. Sequential consistency must be enforceable • Programs using only synchronization variables must be sequentially consistent. • Compiler literature contains many papers on enforcing sequential consistency by adding fences. But: – Not really possible on Itanium. – Wasn‟t possible on X86 until the re-revision of the spec last year. – Took months of discussions with PowerPC architects to conclude it‟s (barely, sort of) possible there. • The core issue is “write atomicity”: 7/28/2011 36

37. Can fences enforce SC? Unclear that hardware fences can ensure sequential consistency. “IRIW” example: x, y initially zero. Fences between every instruction pair! Thread 1: Thread 2: Thread 3: Thread 4: x = 1; r1 = x; (1) y = 1; r3 = y; (1) fence; fence; r2 = y; (0) r4 = x; (0) x set first! y set first! Fully fenced, not sequentially consistent. Does hardware allow it? 7/28/2011 37

38. Why does it matter? • Nobody cares about IRIW!? • It‟s a pain to enforce on at least PowerPC. • Many people (Sarita Adve, Doug Lea, Vijay Saraswat) spent about a year trying to relax SC requirement here. • (Personal opinion) The results were incomprehensible, and broke more important code. • No viable alternatives! 7/28/2011 38

39. Acceptable hardware memory models • More challenging requirements: 1. Precise memory model specification 2. Byte stores 3. Cheap mechanism to enforce write atomicity 4. Dirt cheap mechanism to enforce data dependency ordering(?) (Java final fields) • Other than that, all standard approaches appear workable, but … 7/28/2011 39

40. Replace fences completely? Synchronization variables on X86 • atomic store: ~1 cycle dozens of cycles – store (mov); mfence; • atomic load: ~1 cycle – load (mov) • Store implicitly ensures that prior memory operations become visible before store. • Load implicitly ensures that subsequent memory operations become visible later. • Sole reason for mfence: Order atomic store followed by atomic load. 7/28/2011 40

41. Fence enforces all kinds of additional, unobservable orderings • s is a synchronization variable: x = 1; s = 2; // includes fence r1 = y; • Prevents reordering of x = 1 and r1 = y; – final load delayed until assignment to a visible. • But this ordering is invisible to non-racing threads – …and expensive to enforce? • We need a tiny fraction of mfence functionality. 7/28/2011 41

43. Data Races in Java • C++11 leaves data race semantics undefined. – “catch fire” semantics • Java supports sand-boxed code. • Don‟t know how to prevent data-races in sand-boxed, malicious code. • Java must provide some guarantees in the presence of data races. 7/28/2011 43

44. Interesting data race outcome? x, y initially null, Loads may or may not see racing stores? Thread 1: Thread 2: r1 = x; r2 = y; y = r1; x = r2; Outcome: x = y = r1 = r2 = “<your bank password here>” 7/28/2011 44

45. The Java Solution Quotation from 17.4.8, Java Language Specification, 3rd edition, omitted, to avoid possible copyright questions. It addresses the issue by explicitly outlawing the causality cycles required to get the dubious result from the last slide. The important point is that this is a rather complex mathematical specification. … 7/28/2011 45

46. Complicated, but nice properties? • Manson, Pugh, Adve: The Java Memory Model, POPL 05 Quotation from section 9.1.2 of above paper omitted, to avoid possible copyright questions. This asserts (Theorem 1) that non-conflicting operations may be reordered by a compiler. 7/28/2011 46

47. Much nicer than prior attempts, but: • Aspinall, Sevcik, “Java Memory Model Examples: Good, Bad, and Ugly”, VAMP 2007 (also ECOOP 2008 paper) Quotation from above paper omitted, to avoid possible copyright questions. This ends in the statement: “This falsifies Theorem 1 of [paper from previous slide].” Note: The underlying observation is due to Pietro Cenciarelli. 7/28/2011 47

48. Why is this hard? • Want – Constrained race semantics for essential security properties. – Unconstrained race semantics to support compiler and hardware optimizations. – Simplicity. • No known good resolution. 7/28/2011 48

50. A Different Approach • Outlaw data races. • Require violations to be detectable! – Even in malicious sand-boxed code. • Possible approaches: – Statically prevent data races. • Tried repeatedly, ongoing work … – Dynamically detect the relevant data races. 7/28/2011 50

51. Dynamic Race Detection • Need to guarantee one of: – Program is data-race free and provides SC execution (done), – Program contains a data race and raises an exception, or – Program exhibits simple semantics anyway, e.g. • Sequentially consistent • Synchronization-free regions are atomic • This is significantly cheaper than fully accurate data-race detection. – Track byte-level R/W information – Mostly in cache – As opposed to epoch number + thread id per byte 7/28/2011 51

52. For more information: • Boehm, “Threads Basics”, HPL TR 2009-259. • Boehm, Adve, “Foundations of the C++ Concurrency Memory Model”, PLDI 08. • Sevcık and Aspinall, “On Validity of Program Transformations in the Java Memory Model”, ECOOP 08. • Sewell et al, “x86-TSO: A Rigorous and Usable Programmer‟s Model for x86”, CACM, July 2010. • S. V. Adve, Boehm, “Memory Models: A Case for Rethinking Parallel Languages and Hardware”, CACM, August 2010. • Lucia, Strauss, Ceze, Qadeer, Boehm, "Conflict Exceptions: Providing Simple Parallel Language Semantics with Precise Hardware Exceptions, ISCA 2010. Also Marino et al, PLDI 10. • Effinger-Dean, Boehm, Chakrabarti, Joisha, “Extended Sequential Reasoning for Data-Race-Free Programs”, MSPC 11. 7/28/2011 52

53. Questions? 7/28/2011 53

54. Backup slides 7/28/2011 54

55. Introducing Races: Register Promotion 2 r = g; for(...) { [g is global] if(mt) { for(...) { g = r; lock(); r = g; } if(mt) lock(); use/update r instead of g; use/update g; if(mt) { if(mt) unlock(); g = r; unlock(); r = g; } } } g = r; 7/28/2011 55

56. Trylock: Critical section reordering? • Reordering of memory operations with respect to critical sections: Expected (& Java): Naïve pthreads: Optimized pthreads lock() lock() lock() unlock() unlock() unlock() 7/28/2011 56

57. Some open source pthread lock implementations (2006): lock() lock() lock() lock() unlock() unlock() unlock() unlock() [technically incorrect] [Correct, slow] [Correct] [Incorrect] NPTL NPTL NPTL FreeBSD {Alpha, PowerPC} Itanium (&X86) { Itanium, X86 } Itanium {mutex, spin} mutex spin spin 7/28/2011 57

Programming Language Memory Models: What do Shared Variables Mean?

Esté a ler uma amostra.

Confira estes a seguir

Programming Language Memory Models: What do Shared Variables Mean?

Mais Conteúdo rRelacionado

Diapositivos para si (19)

Semelhante a Programming Language Memory Models: What do Shared Variables Mean? (20)

Mais de greenwop (9)

Mais recentes (20)