Simplifying concurrency: Actors. In particular, AKKA.
â˘
0 gostouâ˘1,559 visualizaçþes
Akka, actor systems and reactive programming are new trends in software development with Java and Scala. This talk dives into details of Akka usage with Scala. We will discuss different models of concurrency handling and the actor model in particular. The particular implementation of the model we will be talking about is Akka, a system that supports writing code with the actor model in Java, Scala and .net. We will cover how actors send messages to each other via mailboxes and dispatchers, will take a look at the supervision hierarchy and error handling with the error kernel pattern. In addition to the theoretical section we will also discuss a few projects where Planet OS has used Akka and the actor model to achieve high reliability systems: - an operational dashboard for the world's second largest offshore wind park - DPipe, the highly configurable data processing pipeline that is doing the heavy lifting behind http://data.planetos.com, our universal data access api By Kristo Iila Engineering Manager at Planet OS.
Recomendados
Mais conteĂşdo relacionado
Ăltimo
Ăltimo (20)
Destaque
Destaque (20)
Simplifying concurrency: Actors. In particular, AKKA.
- 1. Simplifying concurrency: Actors In particular, AKKA KRISTO IILA, ENGINEERING MANAGER
- 2. PLANET OS - MARCH 2016 The Many Ways of Concurrency ⢠Multithreading / semaphores / locking / mutexes ⢠Lock-free data structures ⢠Software Transactional Memory ⢠Message passing ⢠Futures / Async ⢠Actors
- 3. PLANET OS - MARCH 2016 What are Actors? ⢠Lightweight object ⢠No shared state ⢠Communication via message passing ⢠Supervision hierarchies ⢠Originally implemented in Erlang ⨠(high availability telephone switches) ⢠Donât need to think in terms of threads and locks ⢠Distributable by design
- 4. PLANET OS - MARCH 2016 An Actor System Implementation - AKKA ⢠Implements actor model ⢠Makes remoting seamless ⢠Works with Java, Scala and .net ⢠Fits very well with scala ⢠Automatically scheduled to threads ⢠Can have multiple thread pools Actor A Mailbox Actor B Message
- 5. PLANET OS - MARCH 2016 Messages ⢠Communication between actors ⢠Tell: Fire and Forget ⢠Ask: Tell + Future { reply } ⢠Guaranteed ordering for one sender and receiver pair ⢠Access to a mailbox ⢠Messages can be anything // Tell example actorRef ! Notify("hi!") // Ask example val future = qaActor ? Question(âWhat?â) // Process result future.onComplete { ⌠} // Send result to actor future pipeTo actorRef2
- 6. PLANET OS - MARCH 2016 Futures ⢠Future is a way to Future { socket.read() } retrieve results ⌠of concurrent operations ⌠asynchronously or synchronously. ⢠Examples Eg code block: Or actor ask: qaActor ? Question(âWhat is the meaning of life?â) ⢠Futures are composable
- 7. PLANET OS - MARCH 2016 Actor Creation ⢠Each actor val qaActor = system.actorOf( Props[QuestionAnswerer], ⨠âOracleâ ) ... is an instance of a class with parameters ⌠has a name ⢠To create an actor:
- 8. PLANET OS - MARCH 2016 Actor References ⢠You can not talk to actors directly ⢠When you create an actor, you get an ActorRef ⢠ActorRef is a reference to an actor ⌠that can be passed around ⌠also in messages ⌠is serializable ⌠possibly remote ⌠hides local state
- 9. PLANET OS - MARCH 2016 QA Actor example
- 10. PLANET OS - MARCH 2016 But What if We Want an Already Existing ActorRef? val selection = context.actorSelection("/user/AllKnowing/oracle") ⢠ActorSelection val allAnswerers = context.actorSelection("/user/AllKnowing/*") ⢠The result can contain multiple actors: allAnswerers ! Question(âWhy are there petunias?â) ⢠But you can still send it messages: ⢠Transparently remote
- 11. PLANET OS - MARCH 2016 But how do I know the actor path? ⢠Actors are in a hierarchy (tree) ⢠Tree root is ActorSystem ⢠System guardian and user guardian ⢠All user-created actors live under ⨠/user ⢠It is a bit more complicated if the actor is in a remote machine ActorSystem /user /system /user/Allknowing /user/AllKnowing/Oracle
- 12. PLANET OS - MARCH 2016 What else can we do with the hierarchy? Supervision! ⢠Reliability ⢠Error kernel pattern ⢠Errors are escalated up the hierarchy ⢠Each level can decide how to respond ⢠Escalate error ⢠Restart child ⢠Stop child ⢠Resume child ⢠Top level guardian will shut down entire actor system /c1 /b1 /c2 /a1 /b2 /user /a2 /b3 /c4 /c5 / / system
- 13. PLANET OS - MARCH 2016 Where do messages go? Mailboxes! ⢠Each actor has access to one ⢠Possibly shared ⢠Multiple types ⢠Bounded / Unbounded ⢠Priority ⢠Can be order-preserving, but not always Actor A Mailbox Actor B Message
- 14. PLANET OS - MARCH 2016 Message Delivery Reliability ⢠Local message delivery is reliable⨠Except for vm errors, and itâs better to always code for failure cases anyway. ⢠Mostly order preserving⨠Unless you explicitly select non-order-preserving mailboxes of course ⢠Remote delivery is unreliable ⢠Should retry messages until acknowledged⨠Or be ok with losing some, or most, or all, depending on the network. ⢠If message processing is idempotent, things get a lot easier to manage
- 15. PLANET OS - MARCH 2016 Who schedules actors to threads? Dispatchers! ⢠Messages arrive to actor mailboxes ⢠Dispatcher takes a message ⢠from actor mailbox ⌠⢠and schedules actor ⌠⢠to handle it on a thread ⢠Threads are taken from execution contexts ⢠⌠aka thread pools ⢠Dispatchers are assigned to actors via rules Actor A Mailbox Actor B Message Dispatcher
- 16. Examples
- 17. Planet OS Data Intelligence
- 18. PLANET OS - MARCH 2016 Websocket handling with actors ⢠Actor based web server ⢠One actor that receives connections ⢠Creates per-connection actors ⢠Supervises / terminates connection actors ⢠Connection actors handle data updates ⢠Better performance than ⨠thread-per-connection model ⢠Easier and scales better than ⨠⌠callbacks for data send/receive⨠⌠event loops Listening⨠Socket Connection ⨠Manager Actor Connection⨠Actor Connection⨠Actor Connection⨠Actor
- 19. PLANET OS - MARCH 2016 Data update publish Kafka Connection⨠Actor Connection⨠Actor Connection⨠Actor Kafka⨠receiver actor Socket Connection Socket Connection Socket Connection
- 20. PLANET OS - MARCH 2016 SCADA SCADA SCADA WS Kafka Queue Process Turbine 1 Turbine 2 Turbine 3 Power Plant Dashboard
- 21. PLANET OS - MARCH 2016 Event processing with actors ⢠Actors represent domain entities ⢠wind turbine ⢠power plant ⢠alert ⢠Receive event updates from Kafka ⢠Produce new events sent to Kafka ⢠Some events are joins of multiple events (eg alerts) ⢠Some events go to Web Sockets
- 22. PLANET OS - MARCH 2016 Challenges ⢠Code must correctly handle idempotency ⢠Code must correctly handle crashes / restarts anywhere ⢠Data updates must not be lost ⢠Even cascaded data updates must not be lost ⢠Purely actor based model must mix consistency handling in business logic
- 23. Planet OS Data Hub
- 24. PLANET OS - MARCH 2016 DPipe - the magic behind the Data Hub ⢠Data pipeline: ⢠Acquisition ⢠Transformation ⢠Analysis ⢠Indexing ⢠For spatio-temporal data: ⢠Time series ⢠Forecasts ⢠Satellites ⢠Sensors ⢠Models
- 25. PLANET OS - MARCH 2016 DPipe - large scale data processing ⢠Multiple stages ⢠Each stage runs a cluster of actor systems ⢠Actor system has a scheduler that distributes work to worker actors ⢠Multiple pipelines also within a stage ⢠Can scale linearly with data sources ⢠Can do fork/join in data pipes with parallelisation ⢠Framework for exactly-once processing at scale
- 26. As if it wasnât already quite obvious We ARE in fact looking for new team members. There are many interesting challenges to work on. Interested? Come talk to me or ⨠e-mail kristo@planetos.com