1. PREDICTABLY FAST
CLOJURE
ZachTellman
@ztellman

2. (nth s 2)

3. (nth s 2)
(.nth ^clojure.lang.Indexed s 2)

4. (nth s 2)
(java.lang.reflect.Array/get s 2)
(.nth ^clojure.lang.Indexed s 2)

5. (nth s 2)
(java.lang.reflect.Array/get s 2)
(.get ^java.util.RandomAccess s 2)
(.nth ^clojure.lang.Indexed s 2)

6. (nth s 2)
(java.lang.reflect.Array/get s 2)
(.get ^java.util.RandomAccess s 2)
(.nth ^clojure.lang.Indexed s 2)
(Character. (.charAt ^CharSequence s 2))

7. (nth s 2)
(java.lang.reflect.Array/get s 2)
(.get ^java.util.RandomAccess s 2)
(.nth ^clojure.lang.Indexed s 2)
(Character. (.charAt ^CharSequence s 2))
(first (next (next s)))

8. REFERENTIAL
TRANSPARENCY
(+ 1 1)
2
~

9. REFERENTIAL
TRANSPARENCY
(f x)
((memoize f) x)
~

10. REFERENTIAL
TRANSPARENCY
(map f s)
(doall (map f s))
~

11. REFERENTIAL
TRANSPARENCY
(map f s)
(pmap f s)
~

12. (map f s)
(map
(fn [x]
(Thread/sleep 1000)
(f x))
s)
?

13. REFERENTIAL
TRANSPARENCY
ASSUMES
INFINITE RESOURCES

14. REFERENTIAL
TRANSPARENCY
IS
CONTEXTUAL

15. THE UTILITY OF
ANY ABSTRACTION
IS
CONTEXTUAL

16. CORRECTNESS
IS
CONTEXTUAL

17. PREDICTABLY FAST
CLOJURE
ZachTellman
@ztellman

18. LOOKING DEEP INTO
THE ABYSS
ZachTellman
@ztellman

19. LOOKING DEEP INTO
THE ABYSS,
USING CLOJURE
ZachTellman
@ztellman

20. WHATTO DO WHEN
YOUR ASSUMPTIONS
COME CRASHING DOWN
AROUNDYOU
ZachTellman
@ztellman

23. MY RESPONSIBILITIES
• 100k requests/sec, at peak
• intake of terabytes of compressed data per day
• eight hours of uninterrupted sleep

24. MY RESPONSIBILITIES
• looking deep into the abyss, using Clojure
• waiting for my assumptions to come crashing
down around me

25. (count s)

26. (defn count
{:inline (fn [x]
`(. clojure.lang.RT (count ~x)))}
[coll]
(clojure.lang.RT/count coll))

27. public static int count(Object o) {
if(o instanceof Counted)
return ((Counted) o).count();
return countFrom(Util.ret1(o, o = null));
}

28. (let [v [1 2 3]]
(quick-bench
(count v)))
(let [v [1 2 3]]
(quick-bench
(.count ^Counted v)))
~10 ns
~5 ns

29. (defn matching-index [offset ks prefix]
(let [cnt-ks (- (count ks) offset)
cnt-prefix (Array/getLength prefix)
cnt (Math/min cnt-ks cnt-prefix)]
(loop [idx 0]
(if (== cnt idx)
(+ offset idx)
(if (= (nth ks (+ offset idx))
(aget prefix idx))
(recur (inc idx))
(+ offset idx))))))
~200 ns

30. (defn matching-index [offset ks prefix]
(let [cnt-ks (- (.count ^Counted ks) offset)
cnt-prefix (Array/getLength prefix)
cnt (Math/min cnt-ks cnt-prefix)]
(loop [idx 0]
(if (== cnt idx)
(+ offset idx)
(if (= (nth ks (+ offset idx))
(aget prefix idx))
(recur (inc idx))
(+ offset idx))))))
~100 ns

31. PERFORMANCE
IS
ALMOST NEVER
THE
SUM OF ITS PARTS

32. WHY IS IT
SLOWER THAN JAVA?
THE ETERNAL QUESTION:

33. (+ 2 1)
!
(+ (Long. 2) 1)
!
(+ (/ 3 2) (/ 3 2))
!
(+ 2 (BigInteger. 1))

34. NO.DISASSEMBLE

35. > (use 'no.disassemble)
nil
!
> (defn log [x] (Math/log x))
#’log
!
> (println (disassemble log))

36. public final class user$log extends clojure.lang.AFunction {
public static {};
...
public user$log();
...
public java.lang.Object invoke(java.lang.Object x);
...
}

37. public java.lang.Object invoke(java.lang.Object x);
0 aload_1 [x]
1 aconst_null
2 astore_1 [x]
3 checkcast java.lang.Number
6 invokestatic clojure.lang.RT.doubleCast(java.lang.Object) : double
9 invokestatic java.lang.Math.log(double) : double
12 invokestatic java.lang.Double.valueOf(double) : java.lang.Double
15 areturn
(fn [x] (Math/log x))

38. public java.lang.Object invoke(java.lang.Object x);
0 aload_1 [x]
1 aconst_null
2 astore_1 [x]
3 checkcast java.lang.Number
6 invokestatic clojure.lang.RT.doubleCast(java.lang.Object) : double
9 invokestatic java.lang.Math.log(double) : double
12 invokestatic java.lang.Double.valueOf(double) : java.lang.Double
15 areturn
(fn [x] (Math/log x))

39. public java.lang.Object invoke(java.lang.Object x);
0 aload_1 [x]
1 aconst_null
2 astore_1 [x]
3 checkcast java.lang.Number
6 invokestatic clojure.lang.RT.doubleCast(java.lang.Object) : double
9 invokestatic java.lang.Math.log(double) : double
12 invokestatic java.lang.Double.valueOf(double) : java.lang.Double
15 areturn
(fn [x] (Math/log x))

40. public java.lang.Object invoke(java.lang.Object x);
0 aload_1 [x]
1 aconst_null
2 astore_1 [x]
3 checkcast java.lang.Number
6 invokestatic clojure.lang.RT.doubleCast(java.lang.Object) : double
9 invokestatic java.lang.Math.log(double) : double
12 invokestatic java.lang.Double.valueOf(double) : java.lang.Double
15 areturn
(fn [x] (Math/log x))

41. public java.lang.Object invoke(java.lang.Object x);
0 aload_1 [x]
1 aconst_null
2 astore_1 [x]
3 checkcast java.lang.Number
6 invokestatic clojure.lang.RT.doubleCast(java.lang.Object) : double
9 invokestatic java.lang.Math.log(double) : double
12 invokestatic java.lang.Double.valueOf(double) : java.lang.Double
15 areturn
(fn [x] (Math/log x))

42. (fn ^double [^double x] (Math/log x))
public java.lang.Object invoke(java.lang.Object arg0);
0 aload_0
1 aload_1
2 checkcast java.lang.Number
5 invokestatic clojure.lang.RT.doubleCast(java.lang.Object) : double
8 invokeinterface clojure.lang.IFn$DD.invokePrim(double) : double
13 new java.lang.Double
16 dup_x2
17 dup_x2
18 pop
19 invokespecial java.lang.Double(double)
22 areturn

43. (fn ^double [^double x] (Math/log x))
public final double invokePrim(double x);
0 dload_1 [x]
1 invokestatic java.lang.Math.log(double) : double
4 dreturn

44. (fn [x y] (Math/min x y))
public java.lang.Object invoke(java.lang.Object x, java.lang.Object y);
0 ldc <String "java.lang.Math">
2 invokestatic java.lang.Class.forName(java.lang.String) : java.lang.Class
5 ldc <String "min">
7 iconst_2
8 anewarray java.lang.Object
11 dup
12 iconst_0
13 aload_1 [x]
14 aconst_null
15 astore_1 [x]
16 aastore
17 dup
18 iconst_1
19 aload_2 [y]
20 aconst_null
21 astore_2
22 aastore
23 invokestatic clojure.lang.Reflector.invokeStaticMethod …
26 areturn

45. (fn [x y] (+ x y))
public java.lang.Object invoke(java.lang.Object x, java.lang.Object y);
0 aload_1 [x]
1 aconst_null
2 astore_1 [x]
3 aload_2 [y]
4 aconst_null
5 astore_2 [y]
6 invokestatic clojure.lang.Numbers.add(java.lang.Object, java.lang.Object) …
9 areturn

46. static public Number add(Object x, Object y){
return ops(x).combine(ops(y)).add((Number)x, (Number)y);
}
class LongOps {
final public Number add(Number x, Number y){
return num(Numbers.add(x.longValue(),y.longValue()));
}
}
static public long add(long x, long y){
long ret = x + y;
if ((ret ^ x) < 0 && (ret ^ y) < 0)
return throwIntOverflow();
return ret;
}
(fn [x y] (+ x y))

47. (fn [^long x ^long y] (+ x y))
public final java.lang.Object invokePrim(long x, long arg1);
0 lload_1 [x]
1 lload_3
2 invokestatic clojure.lang.Numbers.add(long, long) : long
5 invokestatic clojure.lang.Numbers.num(long) : java.lang.Number
8 areturn

48. (fn ^long [^long x ^long x] (+ x y))
public final long invokePrim(long x, long arg1);
0 lload_1 [x]
1 lload_3
2 invokestatic clojure.lang.Numbers.add(long, long) : long
5 lreturn

49. (fn ^long [^long x ^long y] (unchecked-add x y))
public final long invokePrim(long x, long arg1);
0 lload_1 [x]
1 lload_3
2 ladd
5 lreturn

50. (fn ^long [^long x ^long y] (unchecked-add x y))
AS FAST AS JAVA!
public final long invokePrim(long x, long arg1);
0 lload_1 [x]
1 lload_3
2 ladd
5 lreturn

51. (fn [x y] (unchecked-add x y))
public java.lang.Object invoke(java.lang.Object x, java.lang.Object y);
0 aload_1 [x]
1 aconst_null
2 astore_1 [x]
3 aload_2 [y]
4 aconst_null
5 astore_2 [y]
6 invokestatic clojure.lang.Numbers.unchecked_add(java.lang.Object, …
9 areturn

52. AND SO…

53. public class Primitives {
!
…
!
public static long add(long a, long b) {
return a + b;
}
public static double add(double a, double b) {
return a + b;
}
!
…
!
}

54. > (require '[primitive-math :as p])
nil
!
> (macroexpand '(p/+ x y))
(. primitive_math.Primitives add x y)

55. (fn ^long [^long x ^long y] (p/+ x y))
public final long invokePrim(long x, long arg1);
0 lload_1 [x]
1 lload_3
2 invokestatic primitive_math.Primitives.add(long, long) : long
5 lreturn

56. > (set! *warn-on-reflection* true)
true
!
> (fn [x y] (unchecked-add x y))
#<...>
!
> (fn [x y] (p/+ x y))
Reflection warning - call to add can't be resolved.
#<...>

57. • does not supplant Clojure’s numerics
• invariants via feedback when they aren’t
satisfied
• using Java isn’t cheating

58. INTHE BEGINNING,
THERE WASTHE
INPUT STREAM

59. AND SO…

60. (quick-bench
(.getBytes
"a reasonably long string"))
(let [f (memoize identity)]
(quick-bench
(f 1)))
~60 ns
~100 ns

61. (defn memoize
[f]
(let [mem (atom {})]
(fn [& args]
(if-let [e (find @mem args)]
(val e)
(let [ret (apply f args)]
(swap! mem assoc args ret)
ret)))))

62. {} or (array-map)
(hash-map)
• up to eight calls to .equiv()
A LOOKUP
• one call to .hasheq(), approx. one call to .equiv()

63. if(obj instanceof IPersistentVector) {
Collection ma = (Collection) obj;
if (ma.size() != v.count())
return false;
for(Iterator i1 = ((List) v).iterator(),
i2 = ma.iterator();
i1.hasNext();) {
if (!Util.equiv(i1.next(), i2.next()))
return false;
}
return true;
}

64. AND SO…

65. (deftype Tuple0 [])
!
(deftype Tuple1 [a])
!
(deftype Tuple2 [a b])
!
(deftype Tuple3 [a b c])
!
…

66. (if (instance? ~name x##)
~(if (zero? cardinality)
true
`(and
~@(map
(fn [f]
`(Util/equiv ~f (. ~other ~f)))
fields)))
…)

67. (let [v [1 2 3]]
(quick-bench
(nth v 0)))
(let [v [1 2 3]]
(quick-bench
(first v)))
~5 ns
~50 ns

68. (let [t (tuple 1 2 3)]
(quick-bench
(nth t 0)))
(let [t (tuple 1 2 3)]
(quick-bench
(first t)))
~5 ns
~5 ns

69. • if you can, do the hard work once
• if you don’t control the context, assume
every bit matters

70. • if you can, do the hard work at compile-time
• the tools for library-sized macros are a bit
lacking right now

71. I WORK WITH
THIS GUY
(he is programming)

72. AND SO…

73. MUTABILITY!

74. (let-mutable [x 0]
(dotimes [_ 100]
(set! x (inc x)))
x)

75. (let [x (LongContainer. 0)]
(dotimes [_ 100]
(.set x (inc (.get x))))
(.get x))

76. (let-mutable [x 1]
(fn [] x))
(let [x (LongContainer. 1)]
(let [x (.get x)]
(fn [] x)))

77. • Clojure’s general solutions are fairly
pessimistic, and assume few invariants
• try creating a more optimistic context

78. YOU DON’T HAVETO DO
WHAT HE DID

79. AND SO…

80. SOME FINALTHOUGHTS
• be curious about the tools you use
• if you’re going for a bounded solution, describe the
boundaries fully
• if you’re going for a general solution, make sure it’s
actually general

82. QUESTIONS?