This paper presents an algorithm and a data structure for scalable dynamic synchronization in fine-grained parallelism. The algorithm supports the full generality of phasers with dynamic, two-phase, and point-to-point synchronization. It retains the scalability of classical tree barriers, but provides unbounded dynamicity by employing a tailor-made insertion tree data structure. It is the first completely documented implementation strategy for a scalable phaser synchronization construct. Our evaluation shows that it can be used as a drop-in replacement for classic barriers without harming performance, despite its additional complexity and potential for performance optimizations. Furthermore, our approach overcomes performance and scalability limitations which have been present in other phaser proposals.

1. Insertion Tree Phasers Efficient and Scalable Barrier Synchronization for Fine-grained Parallelism Stefan Marr S. Verhaegen, B. De Fraine, T. D’Hondt, W. De Meuter Software Languages Lab VrijeUniversiteitBrussel

2. Agenda Introduction Barriers, Phasers Insertion Tree Phasers Insertion Tree Phaser Algorithm Evaluation Summary 9/26/10 2

3. Barriers Synchronizing parallel activities High productivity: easy to get right Mostly for scientific computing Many-core evolution Synchronizing dynamic and irregular problems Requires low-overhead dynamic hierarchical barriers 9/26/10 3 Introduction t1p1 t2p1 t3p1 t1p2 t2p2 t3p2 t1p3 t2p3 t3p3

4. t1p1 Phasers 9/26/10 4 Introduction Extension of X10 clocks Clocks: dynamic two-phase barrier for fork/join parallelism Registration modes for barrier Enables expression of producer/consumer relation Single statements Executed only by single thread, avoids duplicated barrier operations t1p2 t2p2 t3p2 t2p2 t3p2 t2p3 t3p3

5. Hierarchical Phasers 9/26/10 5 Introduction Shirako & Sarkar in Proc. of IEEE IPDPS 2010 [1] Array access List access First scalable implementation strategy Predefined tree structure Degree, i.e., tree arity Max. number of tiers, i.e., height Composed from phasers Problematic None dynamic structure Two-phase support incomplete Leaves design decisions open Phaser Tier 0 sub sub Tier 1 sub sub sub sub Tier 2 (leafs) sig sig sig sig sig sig sig sig A1 A2 A3 A4 A5 A6 A7 A8

6. Open Questions withHierarchical Phasers Dynamic tree construction, or on initialization? Tradeoffs for atomic operations, overhead of joining/leaving phaser How are operations synchronized? Tradeoffs for overheads and restrictions on parallelism Garbage collection problem for dropped participants Keeps list of synchronization objects incl. dropped participants After reaching max. #participants Is the tree rebalanced? (Hint at it for dropped nodes) Two-phase barrier support does not hide latency for original phasers 9/26/10 6 Introduction

7. Insertion Tree Phasers 9/26/10 7

8. Design Goal Support for full generality of Phaser properties Two-phase support Signal-only/wait-only for producers/consumers Single statement Full dynamicity: fine-grained hierarchical fork/join Adaptation of existing, scalable approaches Dissemination barrier not adaptable Remaining are tree-based approaches 9/26/10 8 Insertion TreePhaserAlgorithm

9. Insertion Tree Goals Stable, i.e., minimized tree modifications Avoid inconsistent synchronization information Maximizing parallel operations Solution: Insertion Tree Inverted tree No removal Complete smallest subtree first 9/26/10 9 Insertion TreePhaserAlgorithm 1/2

10. Insertion Tree 9/26/10 10 Insertion TreePhaserAlgorithm 2/2

11. Insertion Tree 9/26/10 11 Insertion TreePhaserAlgorithm 2/2 1

12. Insertion Tree 9/26/10 12 Insertion TreePhaserAlgorithm 2/2 h1 1 2

13. Insertion Tree 9/26/10 13 Insertion TreePhaserAlgorithm 2/2 h2 h1 1 2 3

14. Insertion Tree 9/26/10 14 Insertion TreePhaserAlgorithm 2/2 h2 h1 h3 1 2 3 4

15. Insertion Tree 9/26/10 15 Insertion TreePhaserAlgorithm 2/2 h4 h2 h1 h3 1 2 3 4 5

16. Insertion Tree 9/26/10 16 Insertion TreePhaserAlgorithm 2/2 h4 h2 h6 h1 h3 h5 h7 1 2 3 4 5 6 7 8

17. Determining the Insertion Point defgetNextInsertNode(tree): result = tree.lastNode i = tree.numLeaves whileimod 2 == 0: result = result.parent i = i/2 return result # this is for 2-ary trees # is adaptable for n-ary trees, too 9/26/10 17 Insertion TreePhaserAlgorithm

18. Synchronization Tree* 9/26/10 18 Insertion TreePhaserAlgorithm Phaser phase: 0 0 0 Phase counter 0 0 0 0 wo Helper nodes Wait-only flag Phase counter 0 0 0 0 rsmd Participant nodes Resume flag *) is simplified, leaves out registration modes A1 A2 A3 A4

19. Announcing Synchronization 9/26/10 19 Insertion Tree Phaser Algorithm Phaser phase: 0 0 0 0 0 0 0 0 0 0 0 A1 A2 A3 A4

20. Announcing Synchronization 9/26/10 20 Insertion Tree Phaser Algorithm Phaser phase: 0 0 0 0 1 1 0 0 0 1 rsmd 1 rsmd A1 A2 A3 A4

21. Announcing Synchronization 9/26/10 21 Insertion Tree Phaser Algorithm Phaser phase: 0 0 0 1 1 1 1 1 rsmd 1 rsmd 1 rsmd 1 rsmd A1 A2 A3 A4

22. Announcing Synchronization 9/26/10 22 Insertion Tree Phaser Algorithm Phaser phase: 0 0 1 1 1 1 1 1 rsmd 1 rsmd 1 rsmd 1 rsmd A1 A2 A3 A4

23. Announcing Synchronization 9/26/10 23 Insertion Tree Phaser Algorithm Phaser phase: 0 1 1 1 1 1 1 1 rsmd 1 rsmd 1 rsmd 1 rsmd A1 A2 A3 A4

24. Announcing Synchronization 9/26/10 24 Insertion Tree Phaser Algorithm Synchronization reached. Continue to next phase. Phaser phase: 1 1 1 1 1 1 1 1 rsmd 1 rsmd 1 rsmd 1 rsmd A1 A2 A3 A4

25. Dropping Participants 9/26/10 25 Insertion TreePhaserAlgorithm Phaser phase: 0 0 1 0 0 1 1 0 0 1 rsmd 1 rsmd A1 A2 A3 A4

26. Dropping Participants 9/26/10 26 Insertion TreePhaserAlgorithm Phaser phase: 0 0 1 0 wo 1 1 0 1 rsmd 1 rsmd A1 A2 A3 A4

27. h1:R Dropping Participants 9/26/10 27 Insertion TreePhaserAlgorithm Phaser phase: 0 0 1 wo wo 1 1 1 rsmd 1 rsmd A1 A2 A3 A4

28. h1:R Dropping Participants 9/26/10 28 Insertion TreePhaserAlgorithm Phaser phase: 0 wo 1 wo wo 1 1 1 rsmd 1 rsmd A1 A2 A3 A4

29. Dropping Participants 9/26/10 29 Insertion TreePhaserAlgorithm Synchronization reached. Continue to next phase. Phaser phase: 1 h1:R wo 1 wo wo 1 1 1 rsmd 1 rsmd A1 A2 A3 A4

30. h1:R Dropping Participants 9/26/10 30 Insertion TreePhaserAlgorithm Phaser phase: 1 wo 1 h1:L wo wo 1 1 1 rsmd 1 rsmd A1 A2 A3 A4

31. Adding New Participants 9/26/10 31 Insertion TreePhaserAlgorithm Phaser phase: 8 9 8 8 9 9 rsmd 8 9 rsmd 8 A1 A2 A3 A4

32. Adding New Participants 9/26/10 32 Insertion TreePhaserAlgorithm Phaser phase: 8 9 8 8 8 8 9 9 rsmd 8 9 rsmd 8 A1 A2 A3 A4

33. Adding New Participants 9/26/10 33 Insertion TreePhaserAlgorithm Phaser phase: 8 -1 8 8 +1 9 8 8 9 9 rsmd 8 9 rsmd 8 A1 A2 A3 A4

34. Adding New Participants 9/26/10 34 Insertion TreePhaserAlgorithm Phaser phase: 8 8 8 propagate phase count 9 8 8 9 9 rsmd 8 9 rsmd 8 A1 A2 A3 A4

35. Evaluation 9/26/10 35

36. Two-Phaser Barrier Operation 9/26/10 36 Evaluation

37. Overhead: Two-Phase vs. Classic 9/26/10 37 Evaluation

38. Use as Drop-In Replacement for SPLASH-2Speedup compared to TmcSpinBarrier 9/26/10 38 Evaluation

39. Summary Scalable and efficient approach to Phasers Documents implementation Based on fully dynamic insertion tree Overcomes limitations of existing approaches Usable as drop-in replacement Future work Scalability beyond 59 cores Optimization for other memory architectures 9/26/10 39 Stefan Marr, IEEE HPCC 2010, Insertion TreePhasers

40. 9/26/10 40 Stefan Marr, IEEE HPCC 2010, Insertion TreePhasers Questions? Phaser phase: 1 h1:R wo 1 h1:L wo wo 1 1 Implementation http://barriers.googlecode.com/ MIT license 1 rsmd 1 rsmd A1 A2 A3 A4

41. References [1] Shirako, Jun & Sarkar, Vivek: Hierarchical Phasers for Scalable Synchronization and Reductions in Dynamic Parallelism In: Proc. of IEEE IPDPS (2010). 9/26/10 41