1. CHAPTER 12 Parallel and Distributed Algorithms

2. Algorithm 12.1.1 Maximum This algorithm returns the maximum value in array a [0], ... , a [ n - 1]. The value of a . length is the number of elements in the array. Input Parameters: a Output Parameters: None maximum ( a ) { n = a . length current_max = a [0] for i = 1 to n - 1 if ( a [ i ] > current_max ) current_max = a [ i ] return current_max }

3. Algorithm 12.2.2 Parallel Search Given an array a of size n and a number x , this algorithm searches for x in a and returns true if there is an i such that a [ i ] = x and false otherwise. Input Parameters: a , x Output Parameters: None parallel_search ( a , x ) { n = a . length found = false for i = 0 to n - 1 in parallel if ( a [ i ] == x ) found = true return found }

4. Algorithm 12.2.4 Locate Given an array a and a value x , the algorithm returns an index i such that a [ i ] = x if there is one, and -1 otherwise. Input Parameters: a , x Output Parameters: None locate ( a , x ) { n = a . length location = -1 found = false for i = 0 to n - 1 in parallel if ( a [ i ] == x ) location = i return location }

5. Cache Coherence Problem What value does a [ i ] contain, after performing this code? n = a . length x = 0 for i = 0 to n in parallel if ( i == n ) x = 1 else a [ i ] = x

6. Algorithm 12.2.5 ER Broadcast This algorithm takes as input a value x and an array a and assigns x to all elements of the array. Input Parameters: a , x Output Parameters: None er_broadcast ( a , x ) { n = a . length a [0] = x for i = 0 to  lg n  - 1 for j = 0 to 2 i - 1 in parallel if (2 i + j < n ) a [2 i + j ] = a [ j ] }

7. Algorithm 12.2.11 Sequential  -Sum This algorithm computes given an array a as input. Input Parameters: a , x Output Parameters: None sequential _  _ sum ( a ) { n = a . length current_sum = a [0] for i = 1 to n - 1 current_sum = current_sum  a [ i ] return current_sum }

8. Algorithm 12.2.12 Parallel  -Sum This EREW algorithm computes given an array a as input. Input Parameters: a , x Output Parameters: None parallel _  _ sum ( a ) { n = a . length for i = 0 to  lg n  - 1 for j = 0 to  n /2 i+1  - 1 in parallel if ( j * 2 i+1 + 2 i < n ) a [ j * 2 i+1 + 2 i ] = a [ j * 2 i+1 + 2 i ]  a [ j * 2 i+1 + 2 i ] return a [0] }

9. Algorithm 12.2.16 Optimal  -Sum This algorithm computes given an array a as input.

10. Input Parameters: a Output Parameters: None optimal _  _ sum ( a ) { n = a . length blocksize = 2  lg lg n  // first, compute  sum of all blocks of // size blocksize ≈ lg n for i = 0 to  n / blocksize  in parallel { base = i * blocksize for j = 1 to blocksize - 1 if ( base + j < n ) a [ base ] = a [ base ]  a [ base + j ] } // second, compute the  sum of the remaining //  n / blocksize  elements for i = lg blocksize to  lg n  - 1 for j = 0 to  n /2 i+1  - 1 in parallel if ( j * 2 i+1 + 2 i < n ) a [ j * 2 i+1 + 2 i ] = a [ j * 2 i+1 + 2 i ]  a [ j * 2 i+1 + 2 i ] return a [0] }

11. Algorithm 12.2.20 Optimal ER Broadcast This algorithm takes as input a value x and an array a and assigns x to all elements of the array.

12. Input Parameters: a , x Output Parameters: None optimal_er_broadcast ( a , x ) { n = a . length n =  n /lg n  a [0] = x // Use er_broadcast to fill the first n ’ cells of a with x for i = 0 to  lg n ’  - 1 for j = 0 to 2 i - 1 in parallel if (2 i + j < n ’) a [2 i + j ] = a [ j ] // Spread the contents using n processors // each performing the sequential algorithm for i = 1 to  lg n  for j = 0 to n ’ - 1 in parallel if ( i × n ’ + j < n ) a [ i × n ’ + j ] = a [ j ] }

13. Algorithm 12.2.25 Sequential Prefix This algorithm computes the prefixes for all 0 ≤ k < n given an n -element array a as input. Input Parameters: a Output Parameters: None sequential _ prefix ( a ) { n = a . length sum [0] = a [0] for i = 0 to n - 1 sum [ i ] = sum [ i – 1]  a [ i ] }

14. Algorithm 12.2.26 Parallel Prefix This algorithm computes the prefixes for all 0 ≤ k < n given an n -element array a as input. Input Parameters: a Output Parameters: None parallel _ prefix ( a ) { n = a . length for i = 0 to n - 1 sum [ j ] = a [ j ] for i = 0 to  lg n  for j = 0 to n - 1 in parallel if ( j + 2 i < n ) sum [ j + 2 i ] = sum [ j ]  sum [ j + 2 i ] }

15. Algorithm 12.2.31 Sequential List Ranking This algorithm takes as an input the first node u of a linked list. Each node has fields next , the next node of the list or null for the last node in the list, and rank , which is assigned the distance of the node from the end of the list. Input Parameters: u Output Parameters: None sequential_list_ranking ( u ) { n = 0 // counter for total number of nodes node = u while ( node ! = null) { n = n + 1 node = node . next } node = u // start again at the beginning while ( node ! = null) { node . rank = n n = n - 1 node = node . next } }

16. Algorithm 12.2.32 List Ranking This algorithm takes as an input the first node u of a linked list L ( u ). Each node has fields next , the next node of the list or null for the last node in the list, and rank , which is assigned the distance of the node from the end of the list.

17. Input Parameters: u Output Parameters: None list_ranking(u) { for each node in L ( u ) in parallel node . rank = 1 done = false do { for each node in L ( u ) in parallel if ( node . next == null) done = true else { node . rank = node . rank + node . next . rank node . next = node . next . next } } while (! done ) }

18. Algorithm 12.4.11 Minimum Label This algorithm takes as input a two-dimensional array a with indexes ranging from 0 to n + 1. Every element of the array contains fields color and label . The color field of elements in rows and columns with indexes 0 or n + 1 is assumed to be zero. The algorithm computes a component labeling for a and stores it in the label field .

19. Input Parameters: a , n Output Parameters: None minimum_label ( a , n ) { // initialize the label field of // every black pixel to a unique value for i , j  {1, ... , n } in parallel if ( a [ i , j ]. color == 1) a [ i , j ]. label = i × n + j else a [ i , j ]. label = 0 // perform the second step on // every label n 2 times for k = 1 to n 2 for i , j  {1, ... , n } in parallel for x = i - 1 to i + 1 for y = j - 1 to j + 1 if ( a [ x , y ]. color == 1) a [ i , j ]. label = min( a [ i , j ]. label , a [ x , y ]. label ) }

20. Algorithm 12.5.2 Ring Broadcast This algorithm is run on a ring network. The initiator runs init_ring_broadcast , and the noninitiators run ring_broadcast . All processors terminate successfully after having received a message from the initiator. init_ring_broadcast () { send token to successor receive token from predecessor terminate } ring_broadcast () { receive token from predecessor send token to successor terminate }

21. Algorithm 12.5.4 Broadcast This algorithm works on any (fixed) connected network. The initiator runs init_broadcast and the noninitiators broadcast . All processors terminate successfully after having received a message from the initiator. In the algorithms we assume that the current machine we are running the code on is called p . init_broadcast () { N = { q | q is a neighbor of p } for each q  N send token to neighbor q terminate } broadcast () { receive token from neighbor q N = { q | q is a neighbor of p } for each q  N send token to neighbor q terminate }

22. Algorithm 12.5.5 Echo This algorithm works on any (fixed) connected network. The initiator runs init_echo and the noninitiators echo . All processors terminate successfully after having received a message from the initiator. The initiator terminates after all processors have received its message. The machine on which we are running the code is called p . init_echo () { N = { q | q is a neighbor of p } for each q  N send token to neighbor q counter = 0 while ( counter < | N |) { receive token counter ++ } terminate }

23. echo () { receive token from neighbor q parent = q N = { q | q is a neighbor of p } - { parent } for each q  N send token to neighbor q counter = 0 while ( counter < | N |) { receive token counter ++ } send token to neighbor parent terminate }

24. Algorithm 12.5.9 Leader Election This algorithm runs on the unidirectional ring. The initiators run init_election and the noninitiators run election . All processors terminate successfully, and exactly one initiator has its leader attribute set to true. In the algorithms, we assume that the current machine we are running the code on is called p . Every processor has a next neighbor, p . next , to which it can send messages.

25. init_election() { send  token , p . ID  to p . next min = p . ID receive  token , I  while ( p . ID != I ) { if ( I < min ) min = I send  token , I  to p . next receive  token , I  } if ( p . ID == min ) p . leader = true else p . leader = false terminate } election () { p . leader = false // noninitiator is never chosen as leader do { receive  token , I  send  token , I  to p . next } while (true) }