Slides from my talk at JUG Erfurt on 22nd or march 2018. Sample code available: https://github.com/flowing/flowing-retail/tree/zeebe Recording in German available: https://youtu.be/lZIe02um5eI

1. Zeebe.io
Event-driven
Microservice Orchestration
@berndruecker

2. Berlin, Germany
bernd.ruecker@camunda.com
@berndruecker
Bernd Ruecker
Co-founder and
Developer Advocate of
Camunda

3. Camunda Open source platform for workflow and
decision automation.
Developer friendly!
> 75 people
HQ Berlin + Offices US
No VC
0
20
40
60
80
100
120
140
160
180
200
2014 2015 2016 2017 2018

4. What is
orchestration?

5. Orchestration is one use case of
workflow automation

6. What is
workflow automation?

7. <= milliseconds seconds minutes, weeks, …
Business
IT
long runningalways short running short running, but
potentially long running
Use cases for workflow automation

8. <= milliseconds seconds minutes, weeks, …
Business
IT
Business processes
automation
long runningalways short running short running, but
potentially long running
Use cases for workflow automation

9. The „hello world“ example
Human task
management
Service
orchestration
Events , e.g.
timers

10. BPMN
Business Process
Model and Notation
ISO Standard

11. More realistic examples

12. <= milliseconds seconds minutes, weeks, …
Business
IT
Business processes
automation
long runningalways short running short running, but
potentially long running
Use cases for workflow automation

13. <= milliseconds seconds minutes, weeks, …
Business
IT
Business processes
automation
long runningalways short running short running, but
potentially long running
Use cases for workflow automation

14. <= milliseconds seconds minutes, weeks, …
Business
IT
Business processes
automation
Communication in
distributed systems
long runningalways short running short running, but
potentially long running
Use cases for workflow automation

15. https://www.infoworld.com/article/3254777/application-development/
3-common-pitfalls-of-microservices-integrationand-how-to-avoid-them.html

16. Payment
Ever called a RESTAPI?
Credit
Card
REST

17. Distributed systems

18. Payment
Ever called a RESTAPI?
Credit
Card
REST

19. Payment
Stateful retry
Credit
Card
REST

20. Distributed systems introduce complexity you have to tackle!
Credit
Card
Payment
REST

21. Distributed
systems

22. It is impossible to
differentiate certain
failure scenarios.
Independant of
communication style!
Service
Provider
Client

23. Distributed systems introduce complexity you have to tackle!
Credit
Card
Payment
REST

24. Distributed systems introduce complexity you have to tackle!
Credit
Card
Payment
REST

25. <= milliseconds seconds minutes, weeks, …
Business
IT
Business processes
automation
Communication in
distributed systems
long runningalways short running short running, but
potentially long running
Use cases for workflow automation

26. <= milliseconds seconds minutes, weeks, …
Business
IT
Business processes
automation
Distributed
Transactions
Communication in
distributed systems
long runningalways short running short running, but
potentially long running
Use cases for workflow automation

27. Distributed
systems

28. Distributed transactions using compensation *
Compensation

29. Eventual consistency
Temporarily
inconsistent state
But only
temporarily!

30. <= milliseconds seconds minutes, weeks, …
Business
IT
Business processes
automation
Distributed
Transactions
Communication in
distributed systems
long runningalways short running short running, but
potentially long running
Use cases for workflow automation

31. Long running
Potentially

32. Long running = sit and wait

33. <= milliseconds seconds minutes, weeks, …
Business
IT
Business processes
automation
Distributed
Transactions
Communication in
distributed systems
long runningalways short running short running, but
potentially long running
Use cases for workflow automation

34. <= milliseconds seconds minutes, weeks, …
Business
IT
Business processes
automation
Distributed
Transactions
Orchestration
Communication in
distributed systems
long runningalways short running short running, but
potentially long running
Use cases for workflow automation

35. Simplified example:
dash button
Photo by 0xF2, available under Creative Commons BY-ND 2.0
license. https://www.flickr.com/photos/0xf2/29873149904/

36. (Micro-)services
Checkout
Payment
Inventory
Shipment

37. Three steps…

38. Peer-to-peer event chains
Checkout
Payment
Inventory
Shipment
Order
placed
Payment
received
Goods
shipped
Goods
fetched

39. Peer-to-peer event chains
Checkout
Payment
Inventory
Shipment
Order
placed
Payment
received
Goods
shipped
Goods
fetched

40. The danger is that it's very easy to make
nicely decoupled systems with event
notification, without realizing that you're
losing sight of that larger-scale flow, and
thus set yourself up for trouble in future
years.
https://martinfowler.com/articles/201701-event-driven.html

41. The danger is that it's very easy to make
nicely decoupled systems with event
notification, without realizing that you're
losing sight of that larger-scale flow, and
thus set yourself up for trouble in future
years.
https://martinfowler.com/articles/201701-event-driven.html

42. The danger is that it's very easy to make
nicely decoupled systems with event
notification, without realizing that you're
losing sight of that larger-scale flow, and
thus set yourself up for trouble in future
years.
https://martinfowler.com/articles/201701-event-driven.html

43. Peer-to-peer event chains
Checkout
Payment
Inventory
Shipment
Order
placed
Payment
received
Goods
shipped
Goods
fetched
Fetch the goods
before the
payment

44. Peer-to-peer event chains
Checkout
Payment
Inventory
Shipment
Fetch the goods
before the
payment
Goods
fetched
Order
placed
Payment
received
Goods
shipped

45. Peer-to-peer event chains
Checkout
Payment
Inventory
Shipment
Fetch the goods
before the
payment
Customers can
pay via invoice
Goods
fetched
Order
placed
Payment
received
Goods
shipped
…

46. Photo by born1945, available under Creative Commons BY 2.0 license.

47. Order
Extract the end-to-end responsibility
Checkout
Payment
Inventory
Shipment
*Commands have an intent about
what needs to happen in the future
Order
placed
Payment
received
Retrieve
payment

48. Orchestration logic

49. Workflows live inside service boundaries

50. <= milliseconds seconds minutes, weeks, …
Business
IT
Business processes
automation
Distributed
Transactions
Orchestration
Communication in
distributed systems
long runningalways short running short running, but
potentially long running
Use cases for workflow automation

51. Traditional workflow engines

52. Performance?

54. Status quo

55. In general, we may conclude that
Camunda performed better and
more stable for all metrics when
compared with WfMS A and WfMS B.
Micro-Benchmarking BPMN 2.0 Workflow Management Systems with Workflow Patterns,
June 2016, University of Stuttgart, Germany and University of Lugano, Switzerland
„

56. Concrete numbers
please!*
*using my ThinkPad T460s
consumer notebook
in a totally unrepeatable manner
while rendereing a talk recording ;-)

57. 0
50
100
150
200
250
300
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Instancesstarted/sec
Time
on average
181
instances started/sec

58. Scaling

60. At Google, we have
different ideas
about scalability.
Senior Architect at Google discussing BPMN.
„

61. Photo by Mimzy, available under Creative Commons CC0 1.0 license.

62. Telematics
night-2420297_1920.jpg
Photo by jievoyage, available under Creative Commons CC0 1.0 license.

63. IOT
Photo by Mark Bradshaw, available under Creative Commons BY 2.0 license.

64. Logistics
Photo by Sam Churchill, available under Creative Commons BY 2.0 license.

65. Financial trading
Photo by 3844328, available under Creative Commons CC0 1.0 license.

66. Can you handle
200.000 trades/second?
„

67. Workflow automation at scale!
low latency,
high-throughput
low frequency,
latency doesn‘t matter
What people think workflow
automation can do
What we currently teach
workflow automation
to be able to do
What workflow automation
can already do today

68. New requirements -> new solutions
Remove
RDMS
Distributed
system
Design for
performance
Resilience
Flexible
persistence
Load

70. Client
Zeebe
Zeebe
Broker
Your
applicationModel
defined here
and executed
here
Workers subscribe
(pub/sub)
JVM
JVM

71. public static void main(String[] args) {
ZeebeClient zeebe = ZeebeClient.create(new Properties());
zeebe.workflows().deploy("default-topic")
.addResourceFromClasspath("sample.bpmn")
.execute();
zeebe.workflows().create("default-topic")
.bpmnProcessId("sample")
.payload(" {'a':'42'} ")
.execute();
zeebe.tasks().newTaskSubscription("default-topic")
.taskType("calc")
.lockTime(Duration.ofMinutes(5))
.lockOwner("worker")
.handler(new TaskHandler() {
public void handle(TasksClient client, TaskEvent task) {
// do calculation...
client.complete(task);
}
}).open();
}

72. Live hacking

73. https://github.com/zeebe-io/spring-zeebe

74. Client
Zeebe
Zeebe
Broker
Your
application
Streaming / reactive
Binary protocll (MsgPack) + Batching
Scale elastically and
horizontally
Append only log / event sourcing
Peer-to-peer cluster
Single Writer
Consensus via Raft
Model
defined here
and executed
here
Workers subscribe

75. YAML possible
(converted to
BPMN internally)

76. BPMN
Executable
and mature Easy to
understand*
Widespread
*(for Biz and Dev)

77. Living documentation for long-running behaviour

78. Biz Dev
Leverage
state machine &
workflow engine
Living
documentation
Visibility in
testing
Operate with visibility
and context
Understand and discuss
business processes
Evaluate optimizations
in-sync with
implementation
improve
communication
improve
communication
Ops

79. Proper
Operations
Visibility + Context

80. Compare to e.g. Step Functions

81. Compare to e.g. Step Functions

82. Imagine more complex stuff

83. https://netflix.github.io/conductor/metadata/kitchensink/

84. Zeebe Uber Cadence
Netflix
Orcherstrator
AWS Step
Functions
Client language or
API
Java, Go Go Java, REST AWS SDK, REST
Persistent storage
Event log
on file system
Cassandra
Netflix Dynomite
(k-v)
-
License and
business model
OSS, backed by
vendor (EE)
OSS project OSS project Cloud
Distribution Source, Binary Source Source -
Flow language
BPMN
(XML, YAML, Java)
Proprietary
(YAML, Go)
Proprietary
(JSON)
Proprietary
(JSON)
Visibility BPMN None Proprietary Proprietary

85. Zeebe Uber Cadence
Netflix
Orcherstrator
AWS Step
Functions
Client language or
API
Java, Go Go Java, REST AWS SDK, REST
Persistent storage
Event log
on file system
Cassandra
Netflix Dynomite
(k-v)
-
License and
business model
OSS, backed by
vendor (EE)
OSS project OSS project Cloud
Distribution Source, Binary Source Source -
Flow language
BPMN
(XML, YAML, Java)
Proprietary
(YAML, Go)
Proprietary
(JSON)
Proprietary
(JSON)
Visibility BPMN None Proprietary Proprietary

86. TL;DR:
Do our stuff. Not the others‘ ;-)

87. Architecture
Client
Zeebe
Broker
Your
application

88. Architecture

89. Zeebe Client
BPMN Workflow Definition
Zeebe Broker
Zeebe Client
Application
BPMN Workflow Definition
Zeebe Broker
Application
Inventory
Service
Payment
Service
Shipment
Service
Checkout
Service
Shipment
Service
Inventory
Service
Payment
Service
Checkout
Service

90. Zeebe Client
BPMN Workflow Definition
Zeebe Broker
Zeebe Client
BPMN Workflow Definition
Zeebe Broker
Order Service Order Service
Application Application
Inventory
Service
Payment
Service
Shipment
Service
Checkout
Service
Shipment
Service
Inventory
Service
Payment
Service
Checkout
Service

91. Zeebe Client
BPMN Workflow Definition
Zeebe Broker
Kafka Client (or AMQP, REST, …) Zeebe Client
BPMN Workflow Definition
Zeebe Broker
Order Service Order Service
Application Application
Inventory
Service
Payment
Service
Shipment
Service
Checkout
Service
Shipment
Service
Inventory
Service
Payment
Service
Checkout
Service
Kafka
(or RabbitMQ, …)

92. Zeebe Client
BPMN Workflow Definition
WorkItemHandler
Retrieve payment
WorkItemHandler
Fetch goods
WorkItemHandler
Ship goods
Zeebe Broker
Kafka Client (or AMQP, REST, …) Zeebe Client
BPMN Workflow Definition
Zeebe Broker
Order Service Order Service
Application
Kafka ClientKafka ClientKafka ClientKafka Client
Inventory
Service
Payment
Service
Shipment
Service
Checkout
Service
Zeebe Client
WorkItemHandler
Shipment
Service
Zeebe Client
WorkItemHandler
Zeebe Client
WorkItemHandler
Zeebe Client
WorkItemHandler
Inventory
Service
Payment
Service
Checkout
Service
Application
Kafka
(or RabbitMQ, …)

93. Live hacking

94. Status and roadmap
• Tech preview (0.7.0)
• Working towards production-ready state
• Focus on core, not yet on completeness of features
• We love to get your feedback and discuss!

95. Performance?

96. on average
23.414
events/sec
on average
4.682
started instance/sec

97. The box said
„Requires
Windows 7
or better“
so I installed
Linux

98. on average
213.152
events/sec
on average
35.525
started instance/sec

100. Should I throw away my traditional workflow engine?
A-F G-Z

101. https://db-engines.com/en/ranking_trend

102. https://db-engines.com/en/ranking_trend

103. Use Zeebe when
# No RDMS
# Masive scale
# GoLang
Use Camunda when
# otherwise

104. Thank you!

105. bernd.ruecker@camunda.com
@berndruecker
https://bernd-ruecker.com
https://blog.bernd-ruecker.com
https://github.com/flowing
https://www.infoq.com/articles/
events-workflow-automation With thoughts from http://flowing.io
@berndruecker | @martinschimak
Contact:
Slides:
Blog:
Code:
https://www.infoworld.com/article/3254777/
application-development/
3-common-pitfalls-of-microservices-
integrationand-how-to-avoid-them.html