From polling to real time: Scala, Akka, and Websockets from scratch
•
3 likes•1,692 views
Talk in Barcelona Software Craftsmanship about how we've build a chat with scala and akka
Recommended
More Related Content
What's hot
What's hot (20)
Similar to From polling to real time: Scala, Akka, and Websockets from scratch
Similar to From polling to real time: Scala, Akka, and Websockets from scratch (20)
Recently uploaded
Recently uploaded (20)
From polling to real time: Scala, Akka, and Websockets from scratch
- 1. Letgo chat From polling to real time Scala, Akka, and WebSockets from scratch @SergiGP @GVico46 @JavierCane#scbcn16 - Software Craftsmanship Barcelona 2016
- 11. App downloads Messages sent monthly growth Messages sent every day 30M 20 - 40% ~4M
- 12. Context (not Bounded) Where do we come ! Mobile first ◕ Internal REST API ! Startup with less than 2 years ◕ Externalize services (Parse, Kahuna…) ! Funding: $200M ◕ Ads in TV in USA and Turkey
- 13. 2. Legacy
- 14. Legacy ! PHP ! No test ! New Features
- 15. Legacy REST API in PHP Do I have new messages? No And now? And now? And now? And now? No No!! NO!! 😑 🔫 💣
- 16. Legacy No test ! Rebuild a system without tests => 🦄💩💣💀 ! Coupled system => Acceptance tests ◕ Learning what the system does ◕ Find existing weird behaviors
- 17. Background: Given there are test users: | user_object_id | user_name | user_token | | 19fd3160-8643-11e6-ae22-56b6b6499611 | seller | sellerToken | | 120291b2-8643-11e6-ae22-56b6b6499611 | buyer | buyerToken | And user "seller" has a product with: | id | objectId | | 120291b2-8643-11e6-ae22-56b6b6499611 | SuperProductId | Scenario: A user can get messages from another user associated to product Given user "seller" has a conversation related to product "SuperProductId" with user "buyer" When user "seller" asks for messages related to product "SuperProductId" from user "buyer" Then the response status code should be 200 And the response should be in JSON And the JSON should be valid according to the schema "messages.schema" Acceptance test with Behat
- 18. Legacy Taking advantage of backwards compatibility Leaving The Monolith thanks to #EventSourcing @ #scpna
- 19. Legacy New Features ! Product always want more features ! Negotiation: ◕ Archive conversations ◕ Mute interlocutor ◕ Stickers
- 21. Getting started ! Why and how to switch to Scala ! Scala and Akka crash course ! Takeaways
- 22. Why and how to switch to Scala
- 23. We want a WhatsApp inside Letgo
- 24. LOL I’ve payed $22 Billion for WhatsApp
- 25. Getting started Why and how to switch to Scala ! Realtime (WebSockets) ! Akka ! Scale! Why How ! Learning a lot ! External consultancy ! Akka :) ! Backwards Compatible
- 27. Getting started Scala quick start ! Case classes ! Functional ! Optionals ! Futures ! OOP
- 28. class User { private $id; private $name; public function __construct(Uuid $id, string $name) { $this!→id = $id; $this!→name = $name; } public function id() : Uuid { return $this!→id; } Case Class
- 29. { return $this!→id; } public function name() : string { return $this!→name; } public function setId(Uuid $id) : self { return new static($id, $this!→name); } public function setName(string $name) : self { return new static($this!→id, $name); } } Case Class
- 30. rafa.name = "Santi" val santi = rafa.copy(name = "Santi") println(santi.name) #$ Santi val rafa = User(UUID.randomUUID(), "Rafa") println(rafa.name) #$ Rafa case class User(id: UUID, name: String) Case classes Does not compile Usage Immutability
- 31. val users = List( User(UUID.randomUUID(), "Rafa"), User(UUID.randomUUID(), "Santi"), User(UUID.randomUUID(), "Jaime"), User(UUID.randomUUID(), "Diyan") ) Functional Mutable state val names = users.map(user %& user.name) val names = users.map(_.name) List[String] names = new ArrayList(); for (User user: users) { names.add(user.name) } Procedural
- 33. def searchUser(id: UUID): Option[User] = { #$ …search user in database (blocking) Some(rafa) } Option searchUser(userId) match { case Some(user) %& #$ do stuff case None %& #$ user not found } Usage (pattern matching)
- 34. Option usage (functional) searchUser(userId) match { case Some(user) %& #$ do stuff case None %& #$ user not found } searchUser(userId).map { userFound %& #$ do stuff }
- 35. Futures def searchUser(id: UUID): Future[Option[User]] = { Future { Thread.sleep(1000) Some(rafa) } }
- 36. Futures usage searchUser(userId).onComplete { case Success(Some(user)) %& #$ do stuff case Success(None) %& #$ user not found case Failure(exception) %& #$ future has crashed } searchUser(userId).map { case Some(user) %& #$ do stuff case None %& #$ user not found } searchUser(userId).map(_.map(_.name))
- 37. trait UserRepository { def search(id: UUID): Future[Option[User]] } trait ConsoleLogger { def warning(message: String) = { println(message) } } OOP - DIP
- 38. OOP - DIP class MysqlUserRepository extends UserRepository with ConsoleLogger { def search(id: UUID): Future[Option[User]] = { #$ implementation warning("user not found") Future(Some(rafa)) } }
- 39. OOP - Companion object object UserId { def random: UserId = { UserId(UUID.randomUUID()) } } case class UserId(id: UUID) val userId = UserId.random println(userId.id) case class User(id: UserId, name: String) Usage
- 41. Scala quick start Akka (actor model) ! Concept ! Introductory examples ! Chat actors architecture
- 42. Scala quick start Akka (actor model) - Concept ! Mailbox (1 each time) ! receive to handle incoming messages ! ActorRef ! Tell or ask methods to interact with the ActorRef ! Location transparency
- 43. final class ConnectionActor extends Actor { } object ConnectionActor { def props: Props = Props(new ConnectionActor) } Building our first actor Instantiation val connection: ActorRef = context.actorOf(ConnectionActor.props)
- 44. object ConnectionActor { def props: Props = Props(new ConnectionActor) } final class ConnectionActor extends Actor { override def receive: Receive = { case PingQuery %& } } Building our first actor Instantiation val connection: ActorRef = context.actorOf(ConnectionActor.props)
- 45. final class ConnectionActor(webSocket: ActorRef) extends Actor { override def receive: Receive = { case PingQuery %& webSocket ! PongResponse } } Tell (Fire & forget) object ConnectionActor { def props(webSocket: ActorRef): Props = Props(new ConnectionActor(webSocket)) } Building our first actor Instantiation val connection: ActorRef = context.actorOf(ConnectionActor.props(webSocket))
- 46. case class ConnectionActorState( lastRequestSentAt: Option[DateTime] ) { def requestSent: ConnectionActorState = copy(lastRequestSentAt = Some(DateTime.now)) } Dealing with state
- 47. case class ConnectionActorState( lastRequestSentAt: Option[DateTime] ) { def requestSent: ConnectionActorState = copy(lastRequestSentAt = Some(DateTime.now)) } final class ConnectionActor(webSocket: ActorRef) extends Actor { var state = ConnectionActorState(lastRequestSentAt = None) override def receive: Receive = { case PingQuery(requestId) %& state = state.requestSent webSocket.actorRef ! PongResponse } Dealing with state State model Akka: 1 message at a time (no race conditions)
- 48. final class ConnectionActor(webSocket: ActorRef) extends Actor { var state = ConnectionActorState(lastRequestSentAt = None) override def preStart(): Unit = { context.system.scheduler.schedule( initialDelay = 1.minute, interval = 1.minute, receiver = self, message = CheckWebSocketTimeout ) } override def receive: Receive = { case PingQuery(requestId) %& Lifecycle
- 49. override def preStart(): Unit = { context.system.scheduler.schedule( initialDelay = 1.minute, interval = 1.minute, receiver = self, message = CheckWebSocketTimeout ) } override def receive: Receive = { case PingQuery(requestId) %& state = state.requestSent webSocket ! PongResponse() case CheckWebSocketTimeout %& if (state.hasBeenIdleFor(5.minutes)) { self ! PoisonPill } Lifecycle
- 50. override def receive: Receive = { case PingQuery %& Future { Thread.sleep(1000) sender() ! PongResponse } } Akka and Futures - SHIT HAPPENS sender() could have changed
- 51. Be careful dealing with futures - sender() override def receive: Receive = { case PingQuery %& Future { Thread.sleep(1000) PongResponse }.pipeTo(sender()) } sender() outside Future Same happens with self
- 52. Chat actors architecture TalkerS ConversationSJ TalkerJ ConnectionS1 ConnectionJ1
- 53. Maintains consistency between 2 talkers : 1 conversation Kill connections if shit happens Chat actors architecture TalkerS ConversationSJ ConnectionS1 CS2CS3 Connection Supervisor Talker Provider Conversation Provider Maintains consistency between N connections : 1 talker “Singleton” actor Non “singleton” actor
- 54. Takeaways
- 55. Scala quick start Takeaways ! Neophyte guide for scala ! Raúl Raja ! Akka Concurrency book ! Scala for the impatient ! Lightbend webinars ! Lightbend activator
- 56. 4. Pain Points
- 57. Pain Points ! MaxScale ! Slick ! Deploy ! Dependency Injection ! Sync between chats
- 58. Chat protocol SERVER TO CLIENT Events Commands Queries ACK RESPONSE Events CLIENT TO SERVER
- 59. SERVER TO CLIENT Commands typing_started typing_stopped interlocutor_typing_started interlocutor_typing_stopped interlocutor_message_sent interlocutor_reception_confirmed interlocutor_read_confirmed Events Queries Events fetch_conversations fetch_conversation_details fetch_messages fetch_messages_newer_than_id fetch_messages_older_than_id confirm_reception confirm_read archive_conversations unarchive_conversations CLIENT TO SERVER Chat protocol authenticate create_conversation send_message
- 62. Legacy events workerN Legacy events worker… Legacy events worker2 Scaling domain events workers Legacy events worker1 Auto scaling supervisor actor SQS
- 63. Scaling domain events workers Legacy events worker1 Legacy events worker2 Legacy events worker… Legacy events workerN Auto scaling supervisor actor SQS
- 64. 5. From PHP to Scala
- 65. From PHP to Scala ! Language community ! Composer vs SBT ◕ Semantic Versioning (scalaz, play…) ! Developer eXperience ◕ Not descriptive errors ◕ Scala and IntelliJ ! Learning Curve ! Loving and hating the compiler ! Another set of problems
- 67. Credits ! Presentation base template by SlidesCarnival ! Graphics generated using draw.io