4. public interface Publisher<T> {
public void subscribe(Subscriber<? super T> s);
}
public interface Subscriber<T> {
public void onSubscribe(Subscription s);
public void onNext(T t);
public void onError(Throwable t);
public void onComplete();
}
public interface Processor<T, R> extends Subscriber<T>, Publisher<R> {
}
public interface Subscription {
public void request(long n);
public void cancel();
}
Reactive Streams
13. • We know async IO from last week
• But there are other types of async operations, that
cross over different async boundaries
• between applications
• between threads
• and over the network as we saw
48. Composition
In FP this makes us warm and fuzzy
val f: A => B
val g: B => C
val h: A => C = f andThen g
49. • Using Actors?
• An Actor is aware of who sent it messages and where it
must forward/reply them.
• No compositionality without thinking about it explicitly.
50. Data Flow
• What are streams ? Flows of data.
• Imagine a 10 stage data pipeline you want to model
• Now imagine writing that in Actors.
51.
52. • Following the flow of data in Actors requires
jumping around all over the code base
• Low level, error prone and hard to reason about
68. val src = Source(1 to 10)
val double = Flow[Int].map(_ * 2)
val negate = Flow[Int].map(_ * -1)
val print = Sink.foreach[Int](println)
val graph = src via double via negate to print
graph.run()
-2
-4
-6
-8
-10
-12
-14
-16
-18
-20
69. • Flow is immutable, thread-safe, and thus
freely shareable
70. • Are Linear flows enough ?
• No, we want to be able to describe arbitrarilly
complex steps in our pipelines
74. • We define multiple linear flows and then use the
Graph DSL to connect them.
• We can combine multiple streams - fan in
• Split a stream into substreams - fan out
81. Sink.fromGraph(GraphDSL.create(highRes, mediumRes, lowRes)((_, _, _){ implicit b =>
(highSink, mediumSink, lowSink) => {
import GraphDSL.Implicits._
val bcastInput = b.add(Broadcast[ByteString](1))
val bcastRawBytes = b.add(Broadcast[Array[Byte]](3))
val processHigh: Flow[Array[Byte], ByteString, NotUsed]
val processMedium: Flow[Array[Byte], ByteString, NotUsed]
val processLow: Flow[Array[Byte], ByteString, NotUsed]
bcastInput.out(0) ~> byteAcc ~> bcastRawBytes ~> processHigh ~> highSink
bcastRawBytes ~> processMedium ~> mediumSink
bcastRawBytes ~> processLow ~> lowSink
SinkShape(bcastInput.in)
}
})
Has one input of type ByteString
82. Takes 3 Sinks, which can be Files, DBs, etc.
Has one input of type ByteString
Sink.fromGraph(GraphDSL.create(highRes, mediumRes, lowRes)((_, _, _){ implicit b =>
(highSink, mediumSink, lowSink) => {
import GraphDSL.Implicits._
val bcastInput = b.add(Broadcast[ByteString](1))
val bcastRawBytes = b.add(Broadcast[Array[Byte]](3))
val processHigh: Flow[Array[Byte], ByteString, NotUsed]
val processMedium: Flow[Array[Byte], ByteString, NotUsed]
val processLow: Flow[Array[Byte], ByteString, NotUsed]
bcastInput.out(0) ~> byteAcc ~> bcastRawBytes ~> processHigh ~> highSink
bcastRawBytes ~> processMedium ~> mediumSink
bcastRawBytes ~> processLow ~> lowSink
SinkShape(bcastInput.in)
}
})
83. Describes 3 processing stages
That are Flows of Array[Byte] => ByteString
Sink.fromGraph(GraphDSL.create(highRes, mediumRes, lowRes)((_, _, _){ implicit b =>
(highSink, mediumSink, lowSink) => {
import GraphDSL.Implicits._
val bcastInput = b.add(Broadcast[ByteString](1))
val bcastRawBytes = b.add(Broadcast[Array[Byte]](3))
val processHigh: Flow[Array[Byte], ByteString, NotUsed]
val processMedium: Flow[Array[Byte], ByteString, NotUsed]
val processLow: Flow[Array[Byte], ByteString, NotUsed]
bcastInput.out(0) ~> byteAcc ~> bcastRawBytes ~> processHigh ~> highSink
bcastRawBytes ~> processMedium ~> mediumSink
bcastRawBytes ~> processLow ~> lowSink
SinkShape(bcastInput.in)
}
})
Has one input of type ByteString
Takes 3 Sinks, which can be Files, DBs, etc.
84. Describes 3 processing stages
That are Flows of Array[Byte] => ByteString
Sink.fromGraph(GraphDSL.create(highRes, mediumRes, lowRes)((_, _, _){ implicit b =>
(highSink, mediumSink, lowSink) => {
import GraphDSL.Implicits._
val bcastInput = b.add(Broadcast[ByteString](1))
val bcastRawBytes = b.add(Broadcast[Array[Byte]](3))
val processHigh: Flow[Array[Byte], ByteString, NotUsed]
val processMedium: Flow[Array[Byte], ByteString, NotUsed]
val processLow: Flow[Array[Byte], ByteString, NotUsed]
bcastInput.out(0) ~> byteAcc ~> bcastRawBytes ~> processHigh ~> highSink
bcastRawBytes ~> processMedium ~> mediumSink
bcastRawBytes ~> processLow ~> lowSink
SinkShape(bcastInput.in)
}
})
Has one input of type ByteString
Emits result to the 3 Sinks
Takes 3 Sinks, which can be Files, DBs, etc.
85. Has a type of:
Sink[ByteString, (Future[IOResult], Future[IOResult], Future[IOResult])]
Sink.fromGraph(GraphDSL.create(highRes, mediumRes, lowRes)((_, _, _){ implicit b =>
(highSink, mediumSink, lowSink) => {
import GraphDSL.Implicits._
val bcastInput = b.add(Broadcast[ByteString](1))
val bcastRawBytes = b.add(Broadcast[Array[Byte]](3))
val processHigh: Flow[Array[Byte], ByteString, NotUsed]
val processMedium: Flow[Array[Byte], ByteString, NotUsed]
val processLow: Flow[Array[Byte], ByteString, NotUsed]
bcastInput.out(0) ~> byteAcc ~> bcastRawBytes ~> processHigh ~> highSink
bcastRawBytes ~> processMedium ~> mediumSink
bcastRawBytes ~> processLow ~> lowSink
SinkShape(bcastInput.in)
}
})