Scala eXchange 2013 presentation of the ScalaBlitz - a highly optimized data-parallel collection framework for Scala.

1. ScalaBlitz
Efficient Collections Framework

2. What’s a Blitz?

3. Blitz-chess is a style of
rapid chess play.

4. Blitz-chess is a style of
rapid chess play.

5. Knights have horses.

6. Horses run fast.

7. def mean(xs: Array[Float]): Float =
xs.par.reduce(_ + _) / xs.length

10. With Lists, operations
can only be executed
from left to right

11. 1 2 4 8

12. 1 2 4 8
Not your typical list.

13. Bon app.

18. Apparently not enough

20. No amount of
documentation is
apparently enough

22. The reduceLeft
guarantees operations are
executed from left to right

23. Parallel and sequential
collections sharing operations

24. There are several
problems here

25. How we see users

26. How users
see the docs

27. Bending the truth.

28. And sometimes we
were just slow

29. So, we have a new API now
def findDoe(names: Array[String]): Option[String] =
{
names.toPar.find(_.endsWith(“Doe”))
}

30. Wait, you renamed a
method?
def findDoe(names: Array[String]): Option[String] =
{
names.toPar.find(_.endsWith(“Doe”))
}

31. Yeah, par already exists.
But, toPar is different.
def findDoe(names: Array[String]): Option[String] =
{
names.toPar.find(_.endsWith(“Doe”))
}

32. def findDoe(names: Array[String]): Option[String] = {
names.toPar.find(_.endsWith(“Doe”))
}
implicit class ParOps[Repr](val r: Repr) extends AnyVal {
def toPar = new Par(r)
}

33. def findDoe(names: Array[String]): Option[String] = {
ParOps(names).toPar.find(_.endsWith(“Doe”))
}
implicit class ParOps[Repr](val r: Repr) extends AnyVal {
def toPar = new Par(r)
}

34. def findDoe(names: Array[String]): Option[String] = {
ParOps(names).toPar.find(_.endsWith(“Doe”))
}
implicit class ParOps[Repr](val r: Repr) extends AnyVal {
def toPar = new Par(r)
}
class Par[Repr](r: Repr)

35. def findDoe(names: Array[String]): Option[String] = {
(new Par(names)).find(_.endsWith(“Doe”))
}
implicit class ParOps[Repr](val r: Repr) extends AnyVal {
def toPar = new Par(r)
}
class Par[Repr](r: Repr)

36. def findDoe(names: Array[String]): Option[String] = {
(new Par(names)).find(_.endsWith(“Doe”))
}
class Par[Repr](r: Repr)
But, Par[Repr] does not
have the find method!

37. def findDoe(names: Array[String]): Option[String] = {
(new Par(names)).find(_.endsWith(“Doe”))
}
class Par[Repr](r: Repr)
True, but Par[Array[String]]
does have a find method.

38. def findDoe(names: Array[String]): Option[String] = {
(new Par(names)).find(_.endsWith(“Doe”))
}
class Par[Repr](r: Repr)
implicit class ParArrayOps[T](pa: Par[Array[T]]) {
...
def find(p: T => Boolean): Option[T]
...
}

39. More flexible!

40. More flexible!
● does not have to implement methods that
make no sense in parallel

41. More flexible!
● does not have to implement methods that
make no sense in parallel
● slow conversions explicit

42. No standard library collections were
hurt doing this.

43. More flexible!
● does not have to implement methods that
make no sense in parallel
● slow conversions explicit
● non-intrusive addition to standard library

44. More flexible!
● does not have to implement methods that
make no sense in parallel
● slow conversions explicit
● non-intrusive addition to standard library
● easy to add new methods and collections

45. More flexible!
● does not have to implement methods that
make no sense in parallel
● slow conversions explicit
● non-intrusive addition to standard library
● easy to add new methods and collections
● import switches between implementations

46. def findDoe(names: Seq[String]): Option[String] = {
names.toPar.find(_.endsWith(“Doe”))
}

47. def findDoe(names: Seq[String]): Option[String] = {
names.toPar.find(_.endsWith(“Doe”))
}

48. def findDoe(names: Seq[String]): Option[String] = {
names.toPar.find(_.endsWith(“Doe”))
}
But how do I write generic code?

49. def findDoe[Repr[_]](names: Par[Repr[String]]) = {
names.toPar.find(_.endsWith(“Doe”))
}

50. def findDoe[Repr[_]](names: Par[Repr[String]]) = {
names.toPar.find(_.endsWith(“Doe”))
}
Par[Repr[String]]does not
have a find

51. def findDoe[Repr[_]: Ops](names: Par[Repr[String]]) = {
names.toPar.find(_.endsWith(“Doe”))
}

52. def findDoe[Repr[_]: Ops](names: Par[Repr[String]]) = {
names.toPar.find(_.endsWith(“Doe”))
}
We don’t do this.

53. Make everything as simple as
possible, but not simpler.

54. def findDoe(names: Reducable[String])= {
names.find(_.endsWith(“Doe”))
}

55. def findDoe(names: Reducable[String])= {
names.find(_.endsWith(“Doe”))
}
findDoe(Array(1, 2, 3).toPar)

56. def findDoe(names: Reducable[String])= {
names.find(_.endsWith(“Doe”))
}
findDoe(toReducable(Array(1, 2, 3).toPar))

57. def findDoe(names: Reducable[String])= {
names.find(_.endsWith(“Doe”))
}
findDoe(toReducable(Array(1, 2, 3).toPar))
def arrayIsReducable[T]: IsReducable[T] = { … }

58. So let’s write a program!

60. import scala.collection.par._
val pixels = new Array[Int](wdt * hgt)
for (idx <- (0 until (wdt * hgt)).toPar) {
}

61. import scala.collection.par._
val pixels = new Array[Int](wdt * hgt)
for (idx <- (0 until (wdt * hgt)).toPar) {
val x = idx % wdt
val y = idx / wdt
}

62. import scala.collection.par._
val pixels = new Array[Int](wdt * hgt)
for (idx <- (0 until (wdt * hgt)).toPar) {
val x = idx % wdt
val y = idx / wdt
pixels(idx) = computeColor(x, y)
}

63. import scala.collection.par._
val pixels = new Array[Int](wdt * hgt)
for (idx <- (0 until (wdt * hgt)).toPar) {
val x = idx % wdt
val y = idx / wdt
pixels(idx) = computeColor(x, y)
}
Scheduler not found!

64. import scala.collection.par._
import Scheduler.Implicits.global
val pixels = new Array[Int](wdt * hgt)
for (idx <- (0 until (wdt * hgt)).toPar) {
val x = idx % wdt
val y = idx / wdt
pixels(idx) = computeColor(x, y)
}

65. import scala.collection.par._
import Scheduler.Implicits.global
val pixels = new Array[Int](wdt * hgt)
for (idx <- (0 until (wdt * hgt)).toPar) {
val x = idx % wdt
val y = idx / wdt
pixels(idx) = computeColor(x, y)
}

66. New parallel collections
33% faster!
Now
103 ms
Previously
148 ms

67. Workstealing tree scheduler
rocks!

68. Workstealing tree scheduler
rocks!
But, are there other interesting
workloads?

69. Fine-grained uniform
workloads are on the opposite
side of the spectrum.

70. def mean(xs: Array[Float]): Float = {
val sum = xs.toPar.fold(0)(_ + _)
sum / xs.length
}

71. def mean(xs: Array[Float]): Float = {
val sum = xs.toPar.fold(0)(_ + _)
sum / xs.length
}
Now
Previously
15 ms
565 ms

72. But how?

73. def fold[T](a: Iterable[T])(z:T)(op: (T, T) => T) = {
var it = a.iterator
var acc = z
while (it.hasNext) {
acc = op(acc, it.next)
}
acc
}

74. def fold[T](a: Iterable[T])(z:T)(op: (T, T) => T) = {
var it = a.iterator
var acc = z
while (it.hasNext) {
acc = box(op(acc, it.next))
}
acc
}

75. def fold[T](a: Iterable[T])(z:T)(op: (T, T) => T) = {
var it = a.iterator
var acc = z
while (it.hasNext) {
acc = box(op(acc, it.next))
}
acc
}
Generic methods cause boxing of primitives

76. def mean(xs: Array[Float]): Float = {
val sum = xs.toPar.fold(0)(_ + _)
sum / xs.length
}

77. def mean(xs: Array[Float]): Float = {
val sum = xs.toPar.fold(0)(_ + _)
sum / xs.length
}
Generic methods hurt performance
What can we do instead?

78. def mean(xs: Array[Float]): Float = {
val sum = xs.toPar.fold(0)(_ + _)
sum / xs.length
}
Generic methods hurt performance
What can we do instead?
Inline method body!

79. def mean(xs: Array[Float]): Float = {
val sum = {
var it = xs.iterator
var acc = 0
while (it.hasNext) {
acc = acc + it.next
}
acc
}
sum / xs.length
}

80. def mean(xs: Array[Float]): Float = {
val sum = {
var it = xs.iterator
var acc = 0
while (it.hasNext) {
acc = acc + it.next
}
acc
}
sum / xs.length
}
Specific type
No boxing!
No memory allocation!

81. def mean(xs: Array[Float]): Float = {
val sum = {
var it = xs.iterator
var acc = 0
while (it.hasNext) {
acc = acc + it.next
}
acc
}
sum / xs.length
}
Specific type
No boxing!
No memory allocation!
565 ms → 281 ms
2X speedup

82. def mean(xs: Array[Float]): Float = {
val sum = {
var it = xs.iterator
var acc = 0
while (it.hasNext) {
acc = acc + it.next
}
acc
}
sum / xs.length
}

83. def mean(xs: Array[Float]): Float = {
val sum = {
var it = xs.iterator
var acc = 0
while (it.hasNext) {
acc = acc + it.next
}
acc
}
sum / xs.length
}
Iterators? For Array?
We don’t need them!

84. def mean(xs: Array[Float]): Float = {
val sum = {
var i = 0
val until = xs.size
var acc = 0
while (i < until) {
acc = acc + a(i)
i = i + 1
}
acc
}
sum / xs.length
}
Use index-based access!

85. def mean(xs: Array[Float]): Float = {
val sum = {
var i = 0
val until = xs.size
var acc = 0
while (i < until) {
acc = acc + a(i)
i = i + 1
}
acc
}
sum / xs.length
}
Use index-based access!
281 ms → 15 ms
19x speedup

86. Are those optimizations parallel-collections specific?

87. Are those optimizations parallel-collections specific?
No

88. Are those optimizations parallel-collections specific?
No
You can use them on sequential collections

89. def mean(xs: Array[Float]): Float = {
val sum = xs.fold(0)(_ + _)
sum / xs.length
}

90. import scala.collections.optimizer._
def mean(xs: Array[Float]): Float = optimize{
val sum = xs.fold(0)(_ + _)
sum / xs.length
}

91. import scala.collections.optimizer._
def mean(xs: Array[Float]): Float = optimize{
val sum = xs.fold(0)(_ + _)
sum / xs.length
}
You get 38 times speedup!

92. Future work

93. @specialized collections
● Maps
● Sets
● Lists
● Vectors
Both faster &
consuming less
memory

94. @specialized collections
● Maps
● Sets
● Lists
● Vectors
Both faster &
consuming less
memory
Expect to get this for free inside
optimize{} block

95. jdk8-style streams(parallel views)
● Fast
● Lightweight
● Expressive API
● Optimized
Lazy data-parallel
operations made
easy

96. Future’s based asynchronous API
val sum = future{ xs.sum }
val normalized = sum.andThen(sum => sum/xs.size)
Boilerplate code, ugly

97. Future’s based asynchronous API
val sum = xs.toFuture.sum
val scaled = xs.map(_ / sum)
● Simple to use
● Lightweight
● Expressive API
● Optimized
Asynchronous data-parallel
operations
made easy

98. Current research: operation fusion
val minMaleAge = people.filter(_.isMale)
.map(_.age).min
val minFemaleAge = people.filter(_.isFemale)
.map(_.age).min

99. Current research: operation fusion
val minMaleAge = people.filter(_.isMale)
.map(_.age).min
val minFemaleAge = people.filter(_.isFemale)
.map(_.age).min
● Requires up to 3 times more memory than original collection
● Requires 6 traversals of collections

100. Current research: operation fusion
val minMaleAge = people.filter(_.isMale)
.map(_.age).min
val minFemaleAge = people.filter(_.isFemale)
.map(_.age).min
● Requires up to 3 times more memory than original collection
● Requires 6 traversals of collections
We aim to reduce this to single traversal with no
additional memory.
Without you changing your code