O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Reactive Streams at "Underscore" Scala User Group Tel Aviv, 2.7.2015

1.028 visualizações

Publicada em

http://www.meetup.com/de/underscore/events/223584473/

Publicada em: Software
  • Seja o primeiro a comentar

Reactive Streams at "Underscore" Scala User Group Tel Aviv, 2.7.2015

  1. 1. Reactive Streams and Akka Streams Lutz Hühnken (@lutzhuehnken)
  2. 2. Streams
  3. 3. Reactive Streams What is a Stream? • Ephemeral flow of data • Potentially unbounded in size • Processed by describing transformation of data 4
  4. 4. Reactive Streams Common Use of Streams • Bulk Data Transfer • Real Time Data Sources • Batch Processing of large data sets • Monitoring and Analytics 5
  5. 5. Reactive Streams
  6. 6. Reactive Streams Reactive 7
  7. 7. Reactive Streams Reactive Streams Overview • How do we: • Handle potentially infinite streams of data? • Handle data in a reactive manner? • Achieve asynchronous non-blocking data flow? • Avoid out of memory errors? 8
  8. 8. Reactive Streams Reactive Streams Projects / Companies • Typesafe • Akka Streams • Netflix • rxJava / rxScala • Pivotal • Spring Reactor • Redhat • Vert.x • Oracle 9
  9. 9. Reactive Streams Supply and Demand • Data Items Flow Downstream • Demand Flows Upstream • Data Items flow only when there is demand. 10
  10. 10. Reactive Streams Dynamic Push-Pull • “Push” behavior when consumer is faster • “Pull” behavior when producer is faster • Switches automatically between these • Batching demand allows batching data 11
  11. 11. Reactive Streams Reactive Streams Specification • Interface Specification • Java interfaces for implementations • TCK • Test Harness to validate implementations • Specification Website • http://www.reactive-streams.org/ 12
  12. 12. Reactive Streams Publisher package org.reactivestreams; public interface Publisher<T> { public void subscribe<T>( Subscriber<T> subscriber); } 13
  13. 13. Reactive Streams Subscriber public interface Subscriber<T> { public void onSubscribe( Subscription subscription); public void onNext(T element); public void onComplete(); public void onError(Throwable cause); } 14
  14. 14. Reactive Streams Subscription public interface Subscription { public void cancel(); public void request(long elements); } 15
  15. 15. Reactive Streams Backpressure • Downstream consumers pushing back on the producer to prevent flooding. • In reactive-streams: • Consumers stop requesting more elements until they are ready to process more. • Producers only fire elements if there is demand. 16
  16. 16. Reactive Streams Why Backpressure? • Explicitly design for system overload • Demand is propagated throughout the WHOLE flow • Can decide WHERE to handle overload • Limit the number of in-flight messages throughout the system • Bounded memory consumption • Bounded cpu contention • Recipient is in control of incoming data rate • Data in flight is bounded by signaled demand 17
  17. 17. Reactive Streams Buffering • We can prefetch stream elements by requesting more than we really need. • We can use this technique to ensure no "sputter" in the stream. • We can also use this technique to pull faster than downstream consumer. 18
  18. 18. Akka Streams & Actors
  19. 19. Reactive Streams Basic Actor 20 case class Greeting(who: String) class GreetingActor extends Actor with ActorLogging { def receive = { case Greeting(who) => log.info("Hello " + who) } } val system = ActorSystem("MySystem") val greeter: ActorRef = system.actorOf(Props[GreetingActor]) greeter ! Greeting("London Scala User Group")
  20. 20. Reactive Streams Properties of Actors • Message Based / Event Driven • Isolated State • Sane Concurrency Model • Isolated Failure Handling (Supervision) 21
  21. 21. Reactive Streams Akka Streams – A Bridge • Converts Publisher/Subscriber API into Actor messages • Simplify creating Publisher/Subscribers using Actors • Attach Reactive streams into existing Akka applications 22
  22. 22. Reactive Streams Creating an ActorSubscriber import akka.stream._ class PrintlnActor extends Actor with ActorSubscriber { val requestStrategy = OneByOneRequestStrategy def receive: Receive = { case ActorSubscriberMessage.OnNext(element) => println(element) } } val printlnActor:ActorRef = system.actorOf(Props[PrintlnActor], "println") val subscriber = ActorSubscriber(printlnActor) 23
  23. 23. Reactive Streams Creating an ActorPublisher import akka.stream._ class IntActor extends Actor with ActorPublisher[Int] { def receive: Receive = { case ActorPublisherMessage.Request(elements) => while (totalDemand > 0) { onNext(1) } } } val intActor: ActorRef = system.actorOf(Props[IntActor], "intActor") val publisher = ActorPublisher(intActor) 24
  24. 24. Reactive Streams Why use Actors as Publishers? • Actors are smart • They can keep internal state to track demand and supply • They can buffer data to meet anticipated demand • Actors are powerful • They can spin up child actors to meet demand • With Akka clustering, can spread load across multiple machines • Actors are resilient • On exception, actor can be killed and restarted by supervisor • Actor interaction is thread-safe, and actor state is private 25
  25. 25. Switch to code
  26. 26. Flows
  27. 27. Reactive Streams Linear Transformations 28 Demand Source Sinkmap …
  28. 28. Reactive Streams Linear Transformations 29 Source Demand Sink drop map
  29. 29. Reactive Streams Linear Stream Transformations • Deterministic (like for collections) • map, filter, collect, grouped, drop, take, groupBy, ... • Time-Based • takeWithin, dropWithin, groupedWithin, ... • Rate-Detached • expand, conflate, buffer, ... • asynchronous • mapAsync, mapAsyncUnordered, ... 30
  30. 30. Reactive Streams Non-linear Stream Transformations • Fan-In • merge, concat, zip • Fan-Out • broadcast, route, unzip 31
  31. 31. Reactive Streams Fan In 32
  32. 32. Reactive Streams Fan Out 33
  33. 33. Reactive Streams Materialization • Akka Streams separate the what from the how • declarative Source/Flow/Sink DSL to create blueprint • FlowMaterializer turns this into running Actors • this allows alternative materialization strategies • optimization • verification / validation • cluster deployment • only Akka Actors for now, but more to come! 34
  34. 34. Switch to code
  35. 35. Slick
  36. 36. Reactive Streams JDBC • Synchronous I/O • Will block the thread it’s running on. • This is true not only for JDBC, but any DB lib running on top of JDBC, Slick included. We like to avoid that. 37
  37. 37. Reactive Streams Imperfect Solution: Use Future / blocking combo • Put blocking database calls in Future(blocking( ... )) • Contention for Connections (but may be limited by the ExecutionContext) • A saturated thread pool blocks everything. 38
  38. 38. Reactive Streams Slick Every Database contains an AsyncExecutor that manages the thread pool for asynchronous execution of Database I/O Actions. 39 mydb = { dataSourceClass = "org.postgresql.ds.PGSimpleDataSource" properties = { databaseName = "mydb" user = "myuser" password = "secret" } numThreads = 10 }
  39. 39. Akka HTTP
  40. 40. Image by Mathias Doenitz
  41. 41. Interoperability
  42. 42. Image by Mathias Doenitz
  43. 43. Switch to code
  44. 44. Thanks @lutzhuehnken
  45. 45. ©Typesafe 2014 – All Rights Reserved

×