SlideShare uma empresa Scribd logo
1 de 64
Baixar para ler offline
at
“Top microservices and
fast data experts in the world.”
How Events Are Reshaping Modern Systems
Jonas Bonér

Lightbend CTO
The future is a function of the past - A. P. Robertson
The (local) present is a merge function of
multiple concurrent pasts - Jonas Bonér
val newLocalPresent = observedPasts
foldLeft(oldLocalPresent) { _ merge _ }
Stream All the Things!!
Dean Wampler

VP Fast Data Engineering

Lightbend
Hadoop classic batch architecture
Streaming, Fast Data Architecture
Latency Use case Suggested engine
Picoseconds to a few microseconds
(real-time)
Space X rocket
Custom hardware (FPGA), no JVM or
anything on top
Less than 100 microseconds Financial services Actors on the JVM
Less than 10 milliseconds Authorising credit cards
Fast data streaming like Flink, Akka
Streams, Kafka Streams
Less than 100 milliseconds Dashboards, Machine learning
We can think of micro-batches, so to
process records in bulk. Spark /
Streaming SQL
Less than 1 second to a minute ETL (Extract, Transform, Load) Mini-batches (spark streaming)
More than a few minutes
Check for fake reviews on a travel
website
Batch jobs
Latency cases to choose a streaming engine
Other choosing criteria
• Volume of data?

• Process records individually or in bulk?

• How do I want to deploy my app?
Overview of streaming engines and relation
with micro services
“Dean Wampler

StreamAllTheThings.pdf”
http://bit.ly/lightbend-fast-data
Search for:
Using Akka, Spark and Cassandra
To Scale Real-Time Auto Loan
Decisioning At Capital One
Fredrick Crable

Director Software Development

Capital One
• Capital One is the #1 bank in the US for loans

• Every loan request needs to be validated

• Legacy depending on 3rd party DSL

• Chose Akka after investigation
End result: a decision performance which runs
faster on smaller hardware.
Very happy with Akka (but I suspect they regret
being “too early” in the game)
Islands in the Stream: Integrating
Akka Streams and Akka Actors
Colin Breck

Tesla
Gather measurements from
a set of wind turbines
Attempt with actors, one actor per turbine
• Each actor sends a measure to the DB.

• Actor gets an internal counter to 1000.

• Actor gets a period flush with a scheduler.

• Actor gets some state to see if we just flushed.

• Actor gets some rate limiting mechanism.
Actor has become quite wordy and bug-prone.
Akka Streams to the rescue!
getMeasurementSource
.groupWithin(1000, 1.second)
.throttle(50, 1.second, 50, ThrottleMode.shaping)
.mapAsync(1)(seq => dbActor ? newMeasures(seq) …)
Source.queue
Source.actorRef
mapAsync + ask
Sink.actorRefWithAck
Bringing streams & actors together
• Streams can be encapsulated within actors

• Actor - Streams communication:
Simulating A LOT OF wind turbines
with actor sharding
Final demo
http://blog.colinbreck.com
Speeding up innovation at Verizon

(part of IBM’s talk)
Keyur Shah 

Associate Fellow

Verizon
• Two years ago: massive monolithic app - 2+
months to rollout new features

• Set up a small team and investigated the
possible platforms.

• They picked up the Reactive platform,
completely microservices driven.
They liked it so much that they want to have
all groups in Verizon to adpot it.
Next on the roadmap, try to use all this data
better. Looking at IBM Watson.
8 Akka Anti-Patterns you better be aware of
Manuel Bernhardt

Consultant
1. Using mutable state
• use immutable messages

• use queries like ask for state inquiries

• use pipeTo pattern -- including pipeTo self
instead…
2. Flat actor hierarchies.
If we don't have a hierarchy, we're missing the
point: the most important thing is the failure
tolerance part. No hierarchy == no failure
handling.
Behaviour of catch blocks
in some random study
3. Too many ActorSystems
1. string concatenation
2. non-async logging
3. not turning debug logging off in prod system
4. logging to files (we should stop doing that)
4. Wrong logging
a. use Akka's logging facility
b. carefully consider the logback appenders, with async variants
c. use placeholders: log.debug("{} hell", callback)
d. use a logging aggregation like logstash, don’t log to files
4. Lose touch with your hardware
We use a lot of abstractions. 

VM, then docker, then JVM, then Akka.
a. know your hardware (and configure
accordingly)

b. beware of virtualization 

c. run load tests on the same hardware of
the production system (!!!)
6. Blocking.
a. use a dedicated dispatcher if you really
must block.

b. if you need to wait, instead of
Thread.sleep, use a context scheduler.
7. Using Akka Remoting directly.
Use Akka cluster instead,
with membership service.
8. Using Java serialization.
Bad performance!
Use a proper binary format from the start (avro,
thrift, kryo), because once you are in production
it's already too late.
Reactive party
Thanks!

Mais conteúdo relacionado

Mais procurados

(APP310) Scheduling Using Apache Mesos in the Cloud | AWS re:Invent 2014
(APP310) Scheduling Using Apache Mesos in the Cloud | AWS re:Invent 2014(APP310) Scheduling Using Apache Mesos in the Cloud | AWS re:Invent 2014
(APP310) Scheduling Using Apache Mesos in the Cloud | AWS re:Invent 2014Amazon Web Services
 
Using eBPF to Measure the k8s Cluster Health
Using eBPF to Measure the k8s Cluster HealthUsing eBPF to Measure the k8s Cluster Health
Using eBPF to Measure the k8s Cluster HealthScyllaDB
 
RxJS Animations Talk - 2017
RxJS Animations Talk - 2017RxJS Animations Talk - 2017
RxJS Animations Talk - 2017Ben Lesh
 
Keeping Latency Low and Throughput High with Application-level Priority Manag...
Keeping Latency Low and Throughput High with Application-level Priority Manag...Keeping Latency Low and Throughput High with Application-level Priority Manag...
Keeping Latency Low and Throughput High with Application-level Priority Manag...ScyllaDB
 
Aleksandr Khokhlov - Do not chase the hype train. Just steal ideas
Aleksandr Khokhlov - Do not chase the hype train. Just steal ideasAleksandr Khokhlov - Do not chase the hype train. Just steal ideas
Aleksandr Khokhlov - Do not chase the hype train. Just steal ideasOdessaJS Conf
 
ILM - Pipeline in the cloud
ILM - Pipeline in the cloudILM - Pipeline in the cloud
ILM - Pipeline in the cloudAaron Carey
 
Cassandra at Glogster
Cassandra at GlogsterCassandra at Glogster
Cassandra at GlogsterRoman Komkov
 
Sync with async
Sync with  asyncSync with  async
Sync with asyncprabathsl
 
From APIs to Electrons: A JS on Hardware Journey
From APIs to Electrons: A JS on Hardware JourneyFrom APIs to Electrons: A JS on Hardware Journey
From APIs to Electrons: A JS on Hardware JourneyTechnicalMachine
 
Big data Lambda Architecture - Batch Layer Hands On
Big data Lambda Architecture - Batch Layer Hands OnBig data Lambda Architecture - Batch Layer Hands On
Big data Lambda Architecture - Batch Layer Hands Onhkbhadraa
 
Processing Big Data in Real-Time - Yanai Franchi, Tikal
Processing Big Data in Real-Time - Yanai Franchi, TikalProcessing Big Data in Real-Time - Yanai Franchi, Tikal
Processing Big Data in Real-Time - Yanai Franchi, TikalCodemotion Tel Aviv
 
Scaling Ethereum using Zero-Knowledge Proofs
Scaling Ethereum using Zero-Knowledge ProofsScaling Ethereum using Zero-Knowledge Proofs
Scaling Ethereum using Zero-Knowledge ProofsHyojun Kim
 
Veritas NetBackup versus Veeam Benchmark Infographic
Veritas NetBackup versus Veeam Benchmark InfographicVeritas NetBackup versus Veeam Benchmark Infographic
Veritas NetBackup versus Veeam Benchmark InfographicArjan van Proosdij
 
Infrastructure as code with Terraform
Infrastructure as code with TerraformInfrastructure as code with Terraform
Infrastructure as code with TerraformSam Bashton
 
Comparison nodejs frameworks using Polls API
Comparison nodejs frameworks using Polls APIComparison nodejs frameworks using Polls API
Comparison nodejs frameworks using Polls APILadislav Prskavec
 
From Gust To Tempest: Scaling Storm
From Gust To Tempest: Scaling StormFrom Gust To Tempest: Scaling Storm
From Gust To Tempest: Scaling StormDataWorks Summit
 

Mais procurados (20)

(APP310) Scheduling Using Apache Mesos in the Cloud | AWS re:Invent 2014
(APP310) Scheduling Using Apache Mesos in the Cloud | AWS re:Invent 2014(APP310) Scheduling Using Apache Mesos in the Cloud | AWS re:Invent 2014
(APP310) Scheduling Using Apache Mesos in the Cloud | AWS re:Invent 2014
 
Using eBPF to Measure the k8s Cluster Health
Using eBPF to Measure the k8s Cluster HealthUsing eBPF to Measure the k8s Cluster Health
Using eBPF to Measure the k8s Cluster Health
 
RxJS Animations Talk - 2017
RxJS Animations Talk - 2017RxJS Animations Talk - 2017
RxJS Animations Talk - 2017
 
Keeping Latency Low and Throughput High with Application-level Priority Manag...
Keeping Latency Low and Throughput High with Application-level Priority Manag...Keeping Latency Low and Throughput High with Application-level Priority Manag...
Keeping Latency Low and Throughput High with Application-level Priority Manag...
 
Aleksandr Khokhlov - Do not chase the hype train. Just steal ideas
Aleksandr Khokhlov - Do not chase the hype train. Just steal ideasAleksandr Khokhlov - Do not chase the hype train. Just steal ideas
Aleksandr Khokhlov - Do not chase the hype train. Just steal ideas
 
Software Exploitation Techniques by Amit Malik
Software Exploitation Techniques by Amit MalikSoftware Exploitation Techniques by Amit Malik
Software Exploitation Techniques by Amit Malik
 
ILM - Pipeline in the cloud
ILM - Pipeline in the cloudILM - Pipeline in the cloud
ILM - Pipeline in the cloud
 
Cassandra at Glogster
Cassandra at GlogsterCassandra at Glogster
Cassandra at Glogster
 
Sync with async
Sync with  asyncSync with  async
Sync with async
 
From APIs to Electrons: A JS on Hardware Journey
From APIs to Electrons: A JS on Hardware JourneyFrom APIs to Electrons: A JS on Hardware Journey
From APIs to Electrons: A JS on Hardware Journey
 
Big data Lambda Architecture - Batch Layer Hands On
Big data Lambda Architecture - Batch Layer Hands OnBig data Lambda Architecture - Batch Layer Hands On
Big data Lambda Architecture - Batch Layer Hands On
 
London Hug 19/5 - Terraform in Production
London Hug 19/5 - Terraform in ProductionLondon Hug 19/5 - Terraform in Production
London Hug 19/5 - Terraform in Production
 
Scalable Event Tracking
Scalable Event TrackingScalable Event Tracking
Scalable Event Tracking
 
Processing Big Data in Real-Time - Yanai Franchi, Tikal
Processing Big Data in Real-Time - Yanai Franchi, TikalProcessing Big Data in Real-Time - Yanai Franchi, Tikal
Processing Big Data in Real-Time - Yanai Franchi, Tikal
 
Scaling Ethereum using Zero-Knowledge Proofs
Scaling Ethereum using Zero-Knowledge ProofsScaling Ethereum using Zero-Knowledge Proofs
Scaling Ethereum using Zero-Knowledge Proofs
 
Apache Storm
Apache StormApache Storm
Apache Storm
 
Veritas NetBackup versus Veeam Benchmark Infographic
Veritas NetBackup versus Veeam Benchmark InfographicVeritas NetBackup versus Veeam Benchmark Infographic
Veritas NetBackup versus Veeam Benchmark Infographic
 
Infrastructure as code with Terraform
Infrastructure as code with TerraformInfrastructure as code with Terraform
Infrastructure as code with Terraform
 
Comparison nodejs frameworks using Polls API
Comparison nodejs frameworks using Polls APIComparison nodejs frameworks using Polls API
Comparison nodejs frameworks using Polls API
 
From Gust To Tempest: Scaling Storm
From Gust To Tempest: Scaling StormFrom Gust To Tempest: Scaling Storm
From Gust To Tempest: Scaling Storm
 

Semelhante a Reactive Summit 2017 Highlights!

Akka in Production - ScalaDays 2015
Akka in Production - ScalaDays 2015Akka in Production - ScalaDays 2015
Akka in Production - ScalaDays 2015Evan Chan
 
Andy Parsons Pivotal June 2011
Andy Parsons Pivotal June 2011Andy Parsons Pivotal June 2011
Andy Parsons Pivotal June 2011Andy Parsons
 
3.2 Streaming and Messaging
3.2 Streaming and Messaging3.2 Streaming and Messaging
3.2 Streaming and Messaging振东 刘
 
Springone2gx 2014 Reactive Streams and Reactor
Springone2gx 2014 Reactive Streams and ReactorSpringone2gx 2014 Reactive Streams and Reactor
Springone2gx 2014 Reactive Streams and ReactorStéphane Maldini
 
Ruby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.com
Ruby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.comRuby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.com
Ruby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.comIlya Grigorik
 
Project Tungsten Phase II: Joining a Billion Rows per Second on a Laptop
Project Tungsten Phase II: Joining a Billion Rows per Second on a LaptopProject Tungsten Phase II: Joining a Billion Rows per Second on a Laptop
Project Tungsten Phase II: Joining a Billion Rows per Second on a LaptopDatabricks
 
Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv
Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv
Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv Amazon Web Services
 
Reactive Streams - László van den Hoek
Reactive Streams - László van den HoekReactive Streams - László van den Hoek
Reactive Streams - László van den HoekRubiX BV
 
Capacity Management from Flickr
Capacity Management from FlickrCapacity Management from Flickr
Capacity Management from Flickrxlight
 
John adams talk cloudy
John adams   talk cloudyJohn adams   talk cloudy
John adams talk cloudyJohn Adams
 
Reactive Stream Processing with Akka Streams
Reactive Stream Processing with Akka StreamsReactive Stream Processing with Akka Streams
Reactive Stream Processing with Akka StreamsKonrad Malawski
 
Flink Forward Berlin 2017: Robert Metzger - Keep it going - How to reliably a...
Flink Forward Berlin 2017: Robert Metzger - Keep it going - How to reliably a...Flink Forward Berlin 2017: Robert Metzger - Keep it going - How to reliably a...
Flink Forward Berlin 2017: Robert Metzger - Keep it going - How to reliably a...Flink Forward
 
Fixing twitter
Fixing twitterFixing twitter
Fixing twitterRoger Xia
 
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...smallerror
 
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...xlight
 
Golang Performance : microbenchmarks, profilers, and a war story
Golang Performance : microbenchmarks, profilers, and a war storyGolang Performance : microbenchmarks, profilers, and a war story
Golang Performance : microbenchmarks, profilers, and a war storyAerospike
 
Performance and predictability (1)
Performance and predictability (1)Performance and predictability (1)
Performance and predictability (1)RichardWarburton
 
Performance and Predictability - Richard Warburton
Performance and Predictability - Richard WarburtonPerformance and Predictability - Richard Warburton
Performance and Predictability - Richard WarburtonJAXLondon2014
 
DotNext 2020 - When and How to Use the Actor Model and Akka.NET
DotNext 2020 - When and How to Use the Actor Model and Akka.NETDotNext 2020 - When and How to Use the Actor Model and Akka.NET
DotNext 2020 - When and How to Use the Actor Model and Akka.NETpetabridge
 

Semelhante a Reactive Summit 2017 Highlights! (20)

Akka in Production - ScalaDays 2015
Akka in Production - ScalaDays 2015Akka in Production - ScalaDays 2015
Akka in Production - ScalaDays 2015
 
Andy Parsons Pivotal June 2011
Andy Parsons Pivotal June 2011Andy Parsons Pivotal June 2011
Andy Parsons Pivotal June 2011
 
3.2 Streaming and Messaging
3.2 Streaming and Messaging3.2 Streaming and Messaging
3.2 Streaming and Messaging
 
Springone2gx 2014 Reactive Streams and Reactor
Springone2gx 2014 Reactive Streams and ReactorSpringone2gx 2014 Reactive Streams and Reactor
Springone2gx 2014 Reactive Streams and Reactor
 
Ruby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.com
Ruby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.comRuby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.com
Ruby Proxies for Scale, Performance, and Monitoring - GoGaRuCo - igvita.com
 
Project Tungsten Phase II: Joining a Billion Rows per Second on a Laptop
Project Tungsten Phase II: Joining a Billion Rows per Second on a LaptopProject Tungsten Phase II: Joining a Billion Rows per Second on a Laptop
Project Tungsten Phase II: Joining a Billion Rows per Second on a Laptop
 
Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv
Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv
Streaming data analytics (Kinesis, EMR/Spark) - Pop-up Loft Tel Aviv
 
Reactive Streams - László van den Hoek
Reactive Streams - László van den HoekReactive Streams - László van den Hoek
Reactive Streams - László van den Hoek
 
Capacity Management from Flickr
Capacity Management from FlickrCapacity Management from Flickr
Capacity Management from Flickr
 
John adams talk cloudy
John adams   talk cloudyJohn adams   talk cloudy
John adams talk cloudy
 
Reactive Stream Processing with Akka Streams
Reactive Stream Processing with Akka StreamsReactive Stream Processing with Akka Streams
Reactive Stream Processing with Akka Streams
 
Flink Forward Berlin 2017: Robert Metzger - Keep it going - How to reliably a...
Flink Forward Berlin 2017: Robert Metzger - Keep it going - How to reliably a...Flink Forward Berlin 2017: Robert Metzger - Keep it going - How to reliably a...
Flink Forward Berlin 2017: Robert Metzger - Keep it going - How to reliably a...
 
Fixing twitter
Fixing twitterFixing twitter
Fixing twitter
 
Fixing_Twitter
Fixing_TwitterFixing_Twitter
Fixing_Twitter
 
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
 
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
 
Golang Performance : microbenchmarks, profilers, and a war story
Golang Performance : microbenchmarks, profilers, and a war storyGolang Performance : microbenchmarks, profilers, and a war story
Golang Performance : microbenchmarks, profilers, and a war story
 
Performance and predictability (1)
Performance and predictability (1)Performance and predictability (1)
Performance and predictability (1)
 
Performance and Predictability - Richard Warburton
Performance and Predictability - Richard WarburtonPerformance and Predictability - Richard Warburton
Performance and Predictability - Richard Warburton
 
DotNext 2020 - When and How to Use the Actor Model and Akka.NET
DotNext 2020 - When and How to Use the Actor Model and Akka.NETDotNext 2020 - When and How to Use the Actor Model and Akka.NET
DotNext 2020 - When and How to Use the Actor Model and Akka.NET
 

Mais de Fabio Tiriticco

Intro slides - Global Reactive Meetup - Move over JDBC!
Intro slides - Global Reactive Meetup - Move over JDBC!Intro slides - Global Reactive Meetup - Move over JDBC!
Intro slides - Global Reactive Meetup - Move over JDBC!Fabio Tiriticco
 
Planespotting - From Zero To Deep Learning
Planespotting - From Zero To Deep Learning Planespotting - From Zero To Deep Learning
Planespotting - From Zero To Deep Learning Fabio Tiriticco
 
From Zero To Deep Learning With Scala
From Zero To Deep Learning With ScalaFrom Zero To Deep Learning With Scala
From Zero To Deep Learning With ScalaFabio Tiriticco
 
Reactive Amsterdam - Maxim Burgerhout - Quarkus Intro
Reactive Amsterdam - Maxim Burgerhout - Quarkus IntroReactive Amsterdam - Maxim Burgerhout - Quarkus Intro
Reactive Amsterdam - Maxim Burgerhout - Quarkus IntroFabio Tiriticco
 
Ten Frustrations From The Community Trenches (And How To Deal With Them)
Ten Frustrations From The Community Trenches (And How To Deal With Them)Ten Frustrations From The Community Trenches (And How To Deal With Them)
Ten Frustrations From The Community Trenches (And How To Deal With Them)Fabio Tiriticco
 
We all need friends and Akka just found Kubernetes
We all need friends and Akka just found KubernetesWe all need friends and Akka just found Kubernetes
We all need friends and Akka just found KubernetesFabio Tiriticco
 
Cloud native akka and kubernetes holy grail to elasticity
Cloud native akka and kubernetes   holy grail to elasticityCloud native akka and kubernetes   holy grail to elasticity
Cloud native akka and kubernetes holy grail to elasticityFabio Tiriticco
 
Reactive Programming or Reactive Systems? (spoiler: both)
Reactive Programming or Reactive Systems? (spoiler: both)Reactive Programming or Reactive Systems? (spoiler: both)
Reactive Programming or Reactive Systems? (spoiler: both)Fabio Tiriticco
 
Beyond Fault Tolerance with Actor Programming
Beyond Fault Tolerance with Actor ProgrammingBeyond Fault Tolerance with Actor Programming
Beyond Fault Tolerance with Actor ProgrammingFabio Tiriticco
 
Akka Streams at Weeronline
Akka Streams at WeeronlineAkka Streams at Weeronline
Akka Streams at WeeronlineFabio Tiriticco
 
Reactive in Android and Beyond Rx
Reactive in Android and Beyond RxReactive in Android and Beyond Rx
Reactive in Android and Beyond RxFabio Tiriticco
 
Reactive Android: RxJava and beyond
Reactive Android: RxJava and beyondReactive Android: RxJava and beyond
Reactive Android: RxJava and beyondFabio Tiriticco
 
WebSockets wiith Scala and Play! Framework
WebSockets wiith Scala and Play! FrameworkWebSockets wiith Scala and Play! Framework
WebSockets wiith Scala and Play! FrameworkFabio Tiriticco
 

Mais de Fabio Tiriticco (13)

Intro slides - Global Reactive Meetup - Move over JDBC!
Intro slides - Global Reactive Meetup - Move over JDBC!Intro slides - Global Reactive Meetup - Move over JDBC!
Intro slides - Global Reactive Meetup - Move over JDBC!
 
Planespotting - From Zero To Deep Learning
Planespotting - From Zero To Deep Learning Planespotting - From Zero To Deep Learning
Planespotting - From Zero To Deep Learning
 
From Zero To Deep Learning With Scala
From Zero To Deep Learning With ScalaFrom Zero To Deep Learning With Scala
From Zero To Deep Learning With Scala
 
Reactive Amsterdam - Maxim Burgerhout - Quarkus Intro
Reactive Amsterdam - Maxim Burgerhout - Quarkus IntroReactive Amsterdam - Maxim Burgerhout - Quarkus Intro
Reactive Amsterdam - Maxim Burgerhout - Quarkus Intro
 
Ten Frustrations From The Community Trenches (And How To Deal With Them)
Ten Frustrations From The Community Trenches (And How To Deal With Them)Ten Frustrations From The Community Trenches (And How To Deal With Them)
Ten Frustrations From The Community Trenches (And How To Deal With Them)
 
We all need friends and Akka just found Kubernetes
We all need friends and Akka just found KubernetesWe all need friends and Akka just found Kubernetes
We all need friends and Akka just found Kubernetes
 
Cloud native akka and kubernetes holy grail to elasticity
Cloud native akka and kubernetes   holy grail to elasticityCloud native akka and kubernetes   holy grail to elasticity
Cloud native akka and kubernetes holy grail to elasticity
 
Reactive Programming or Reactive Systems? (spoiler: both)
Reactive Programming or Reactive Systems? (spoiler: both)Reactive Programming or Reactive Systems? (spoiler: both)
Reactive Programming or Reactive Systems? (spoiler: both)
 
Beyond Fault Tolerance with Actor Programming
Beyond Fault Tolerance with Actor ProgrammingBeyond Fault Tolerance with Actor Programming
Beyond Fault Tolerance with Actor Programming
 
Akka Streams at Weeronline
Akka Streams at WeeronlineAkka Streams at Weeronline
Akka Streams at Weeronline
 
Reactive in Android and Beyond Rx
Reactive in Android and Beyond RxReactive in Android and Beyond Rx
Reactive in Android and Beyond Rx
 
Reactive Android: RxJava and beyond
Reactive Android: RxJava and beyondReactive Android: RxJava and beyond
Reactive Android: RxJava and beyond
 
WebSockets wiith Scala and Play! Framework
WebSockets wiith Scala and Play! FrameworkWebSockets wiith Scala and Play! Framework
WebSockets wiith Scala and Play! Framework
 

Último

Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersThousandEyes
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreternaman860154
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdfhans926745
 
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | DelhiFULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhisoniya singh
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024Results
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...shyamraj55
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Allon Mureinik
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 
Google AI Hackathon: LLM based Evaluator for RAG
Google AI Hackathon: LLM based Evaluator for RAGGoogle AI Hackathon: LLM based Evaluator for RAG
Google AI Hackathon: LLM based Evaluator for RAGSujit Pal
 
Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Paola De la Torre
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Igalia
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slidevu2urc
 

Último (20)

Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | DelhiFULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
Google AI Hackathon: LLM based Evaluator for RAG
Google AI Hackathon: LLM based Evaluator for RAGGoogle AI Hackathon: LLM based Evaluator for RAG
Google AI Hackathon: LLM based Evaluator for RAG
 
Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 

Reactive Summit 2017 Highlights!

  • 1. at
  • 2. “Top microservices and fast data experts in the world.”
  • 3. How Events Are Reshaping Modern Systems Jonas Bonér Lightbend CTO
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13.
  • 14. The future is a function of the past - A. P. Robertson The (local) present is a merge function of multiple concurrent pasts - Jonas Bonér val newLocalPresent = observedPasts foldLeft(oldLocalPresent) { _ merge _ }
  • 15.
  • 16.
  • 17.
  • 18.
  • 19.
  • 20.
  • 21. Stream All the Things!! Dean Wampler VP Fast Data Engineering Lightbend
  • 22. Hadoop classic batch architecture
  • 23. Streaming, Fast Data Architecture
  • 24.
  • 25. Latency Use case Suggested engine Picoseconds to a few microseconds (real-time) Space X rocket Custom hardware (FPGA), no JVM or anything on top Less than 100 microseconds Financial services Actors on the JVM Less than 10 milliseconds Authorising credit cards Fast data streaming like Flink, Akka Streams, Kafka Streams Less than 100 milliseconds Dashboards, Machine learning We can think of micro-batches, so to process records in bulk. Spark / Streaming SQL Less than 1 second to a minute ETL (Extract, Transform, Load) Mini-batches (spark streaming) More than a few minutes Check for fake reviews on a travel website Batch jobs Latency cases to choose a streaming engine
  • 26. Other choosing criteria • Volume of data? • Process records individually or in bulk? • How do I want to deploy my app? Overview of streaming engines and relation with micro services
  • 28.
  • 29. Using Akka, Spark and Cassandra To Scale Real-Time Auto Loan Decisioning At Capital One Fredrick Crable Director Software Development Capital One
  • 30. • Capital One is the #1 bank in the US for loans • Every loan request needs to be validated • Legacy depending on 3rd party DSL • Chose Akka after investigation
  • 31.
  • 32.
  • 33. End result: a decision performance which runs faster on smaller hardware. Very happy with Akka (but I suspect they regret being “too early” in the game)
  • 34. Islands in the Stream: Integrating Akka Streams and Akka Actors Colin Breck Tesla
  • 35. Gather measurements from a set of wind turbines
  • 36. Attempt with actors, one actor per turbine • Each actor sends a measure to the DB. • Actor gets an internal counter to 1000. • Actor gets a period flush with a scheduler. • Actor gets some state to see if we just flushed. • Actor gets some rate limiting mechanism. Actor has become quite wordy and bug-prone.
  • 37. Akka Streams to the rescue! getMeasurementSource .groupWithin(1000, 1.second) .throttle(50, 1.second, 50, ThrottleMode.shaping) .mapAsync(1)(seq => dbActor ? newMeasures(seq) …)
  • 38. Source.queue Source.actorRef mapAsync + ask Sink.actorRefWithAck Bringing streams & actors together • Streams can be encapsulated within actors • Actor - Streams communication:
  • 39. Simulating A LOT OF wind turbines with actor sharding
  • 42. Speeding up innovation at Verizon (part of IBM’s talk) Keyur Shah Associate Fellow Verizon
  • 43. • Two years ago: massive monolithic app - 2+ months to rollout new features • Set up a small team and investigated the possible platforms. • They picked up the Reactive platform, completely microservices driven.
  • 44.
  • 45.
  • 46.
  • 47. They liked it so much that they want to have all groups in Verizon to adpot it. Next on the roadmap, try to use all this data better. Looking at IBM Watson.
  • 48. 8 Akka Anti-Patterns you better be aware of Manuel Bernhardt Consultant
  • 49. 1. Using mutable state • use immutable messages • use queries like ask for state inquiries • use pipeTo pattern -- including pipeTo self instead…
  • 50. 2. Flat actor hierarchies. If we don't have a hierarchy, we're missing the point: the most important thing is the failure tolerance part. No hierarchy == no failure handling.
  • 51. Behaviour of catch blocks in some random study
  • 52. 3. Too many ActorSystems
  • 53. 1. string concatenation 2. non-async logging 3. not turning debug logging off in prod system 4. logging to files (we should stop doing that) 4. Wrong logging a. use Akka's logging facility b. carefully consider the logback appenders, with async variants c. use placeholders: log.debug("{} hell", callback) d. use a logging aggregation like logstash, don’t log to files
  • 54. 4. Lose touch with your hardware
  • 55. We use a lot of abstractions. VM, then docker, then JVM, then Akka.
  • 56. a. know your hardware (and configure accordingly) b. beware of virtualization c. run load tests on the same hardware of the production system (!!!)
  • 57. 6. Blocking. a. use a dedicated dispatcher if you really must block. b. if you need to wait, instead of Thread.sleep, use a context scheduler.
  • 58. 7. Using Akka Remoting directly.
  • 59. Use Akka cluster instead, with membership service.
  • 60. 8. Using Java serialization. Bad performance! Use a proper binary format from the start (avro, thrift, kryo), because once you are in production it's already too late.
  • 62.
  • 63.