My talk from Kafka Summit London 2019. It's aimed at Kafka users who want to know what the serverless buzz is all about, with an explanation of what it's for, and a quick crash course in how to get started with Apache OpenWhisk. Video at https://dalelane.co.uk/blog/?p=3769

1. © 2019 IBM Corporation
Building serverless apps with Kafka
Dale Lane
An introduction to serverless with Apache OpenWhisk
IBM Event StreamsApache Kafka
@dalelane

2. © 2019 IBM Corporation
Agenda
§ What is “serverless”?
§ Implementations
§ Apache OpenWhisk
§ Demos

3. © 2019 IBM Corporation
What is “serverless”?

4. © 2019 IBM Corporation
Serverless
§An architectural pattern where your code is run:
– in response to events
– only on-demand on a per-request basis

5. © 2019 IBM Corporation
Evolution of deployment patterns
Monolith

6. © 2019 IBM Corporation
Evolution of deployment patterns
Microservice
Microservice
Microservice
Monolith

7. © 2019 IBM Corporation
Evolution of deployment patterns
Function
Function
Function
Function
Function
Function
Function
Function
Microservice
Microservice
Microservice
Monolith

8. © 2019 IBM Corporation
Evolution of deployment patterns
Bare metal Virtual
machines
Containers Orchestration Serverless

9. © 2019 IBM Corporation
Function
FaaS
Platform
PaaS
Infrastructure
IaaS
Container
CaaS
Bare metal
Evolution of deployment patterns

10. © 2019 IBM Corporation
Definition
github.com/cncf/wg-serverless

11. © 2019 IBM Corporation
Definition
github.com/cncf/wg-serverless

12. © 2019 IBM Corporation
Definition
applications that do not require server management
applications bundled as one or more functions
[developers] no longer need to spend time and resources on server
provisioning, maintenance, updates, scaling and capacity planning

13. © 2019 IBM Corporation
How is serverless different?
Monolith Microservice
Microservice
Microservice
Function
Function
Function
Function
Function
Function
Function
Function

14. © 2019 IBM Corporation
How is serverless different?

15. © 2019 IBM Corporation
How is serverless different?

16. © 2019 IBM Corporation
How is serverless different?
£ $ €
kr Fr
¥

17. © 2019 IBM Corporation
Example use cases
§ Event processing
§ Data processing
§ Scheduled tasks
§ Scalable REST APIs

18. © 2019 IBM Corporation
The Twelve Factor App

19. © 2019 IBM Corporation
Serverless is good for:
§ Infrequent or sporadic demand
§ Focused, short duration processing
§ Highly dynamic workloads
§ Stateless workloads
§ Inconsistent demand
§ Scheduled infrequent tasks
§ Flexibility

20. © 2019 IBM Corporation
Implementations

21. © 2019 IBM Corporation
Implementations – platform

22. © 2019 IBM Corporation
Implementations – platform

23. © 2019 IBM Corporation
Implementations – platform

24. © 2019 IBM Corporation
Implementations – platform

25. © 2019 IBM Corporation
Implementations – platform

26. © 2019 IBM Corporation
Implementations – hosted

27. © 2019 IBM Corporation
Implementations – private cloud

28. © 2019 IBM Corporation
Implementations – installable

29. © 2019 IBM Corporation
Implementations
s.cncf.io

30. © 2019 IBM Corporation
Serverless framework
serverless.com/framework

31. © 2019 IBM Corporation
Apache OpenWhisk

32. © 2019 IBM Corporation
OpenWhisk
openwhisk.apache.org

33. © 2019 IBM Corporation
OpenWhisk – who

34. © 2019 IBM Corporation
OpenWhisk – where

35. © 2019 IBM Corporation
OpenWhisk concepts

36. © 2019 IBM Corporation
OpenWhisk concepts
event

37. © 2019 IBM Corporation
OpenWhisk concepts
event action

38. © 2019 IBM Corporation
OpenWhisk concepts
event action
function (eventobj)
{
// do something
return result;
}

39. © 2019 IBM Corporation
OpenWhisk languages

40. © 2019 IBM Corporation
OpenWhisk languages

41. © 2019 IBM Corporation
OpenWhisk concepts
event action

42. © 2019 IBM Corporation
OpenWhisk concepts
event actiontrigger

43. © 2019 IBM Corporation
OpenWhisk concepts
actiontrigger ruleevent

44. © 2019 IBM Corporation
OpenWhisk concepts
actionrule
event
actionrule
actionrule
actionrule
trigger

45. © 2019 IBM Corporation
OpenWhisk concepts
rule
event
rule
rule
rule
trigger
trigger
trigger
trigger
action
event
event
event

46. © 2019 IBM Corporation
OpenWhisk concepts
actiontrigger ruleevent
function main(params) {
return { hello : 'world' };
} hello.js

47. © 2019 IBM Corporation
OpenWhisk concepts
actiontrigger ruleevent
function main(params) {
return { hello : 'world' };
} hello.js

48. © 2019 IBM Corporation
OpenWhisk concepts
actiontrigger ruleevent
function main(params) {
return { hello : 'world' };
}
$ wsk action create helloworld hello.js
ok: created action helloworld
hello.js

49. © 2019 IBM Corporation
OpenWhisk concepts
actiontrigger ruleevent
function main(params) {
return { hello : 'world' };
}
$ wsk action create helloworld hello.js
ok: created action helloworld
$ wsk action invoke helloworld
ok: invoked /guest/helloworld with id 7df9c9dfbb
hello.js

50. © 2019 IBM Corporation
OpenWhisk actions
blocking
non-blocking

51. © 2019 IBM Corporation
OpenWhisk actions
blocking
non-blocking
periodic

52. © 2019 IBM Corporation
OpenWhisk actions
action action action

53. © 2019 IBM Corporation
OpenWhisk concepts
actiontrigger ruleevent

54. © 2019 IBM Corporation
OpenWhisk implementation
proxy

55. © 2019 IBM Corporation
OpenWhisk implementation
proxy controller

56. © 2019 IBM Corporation
OpenWhisk implementation
proxy controller
store

57. © 2019 IBM Corporation
OpenWhisk implementation
proxy controller
store
service registry
invoker

58. © 2019 IBM Corporation
OpenWhisk implementation
proxy controller
store
service registry
invoker
invoker
invoker
invoker
invoker
invoker

59. © 2019 IBM Corporation
OpenWhisk implementation
controller
invoker
invoker
invoker
invoker
invoker
invoker

60. © 2019 IBM Corporation
OpenWhisk implementation
controller
invoker
invoker
invoker
invoker
invoker
invoker

61. © 2019 IBM Corporation
OpenWhisk implementation
controller
invoker
invoker
invoker
invoker
invoker
invoker

62. © 2019 IBM Corporation
OpenWhisk implementation
controller
invoker
invoker
invoker
invoker
invoker
invoker

63. © 2019 IBM Corporation
OpenWhisk implementation
controller
invoker
invoker
invoker
invoker
invoker
invoker

64. © 2019 IBM Corporation
OpenWhisk implementation
controller
invoker
invoker
invoker
invoker
invoker
invoker

65. © 2019 IBM Corporation
OpenWhisk implementation
controller
invoker
invoker
invoker
invoker
invoker
invoker

66. © 2019 IBM Corporation
OpenWhisk implementation
controller
invoker
invoker
invoker
invoker
invoker
invoker

67. © 2019 IBM Corporation
Container management
start X
init X X
run X X X
medium.com/openwhisk/squeezing-the-milliseconds-how-to-make-serverless-platforms-blazing-fast-aea0e9951bd0

68. © 2019 IBM Corporation
Container management
start X
init X X
run X X X
medium.com/openwhisk/squeezing-the-milliseconds-how-to-make-serverless-platforms-blazing-fast-aea0e9951bd0

69. © 2019 IBM Corporation
Container management
start X
init X X
run X X X
medium.com/openwhisk/squeezing-the-milliseconds-how-to-make-serverless-platforms-blazing-fast-aea0e9951bd0

70. © 2019 IBM Corporation
Resource limits
github.com/apache/incubator-openwhisk

71. © 2019 IBM Corporation
Demos – installable

72. © 2019 IBM Corporation
Deployment options

73. © 2019 IBM Corporation
git clone https://github.com/apache/incubator-openwhisk-devtools.git
cd incubator-openwhisk-devtools/docker-compose
make quick-start

74. © 2019 IBM Corporation
OpenWhisk event providers
trigger ruleevent

75. © 2019 IBM Corporation
cd incubator-openwhisk-devtools/docker-compose
make create-provider-cloudant
make create-provider-kafka

76. © 2019 IBM Corporation
git clone https://github.com/apache/incubator-openwhisk-package-kafka.git
cd incubator-openwhisk-package-kafka
export OPENWHISK_AUTH=$OPENWHISK_HOME/ansible/files/auth.whisk.system
./installKafka.sh $OPENWHISK_AUTH $SYSTEM_IP_ADDR
http://$SYSTEM_IP_ADDR:5984
local_
$SYSTEM_IP_ADDR

77. © 2019 IBM Corporation
Demo

78. © 2019 IBM Corporation
Demo
event
Message
produced to
Kafka topic
MY.EVENTS

79. © 2019 IBM Corporation
kafka-topics.sh --create --topic MY.EVENTS
--replication-factor 1 --partitions 1
--zookeeper $SYSTEM_IP_ADDR:3181

80. © 2019 IBM Corporation
Demo
event
Message
produced to
Kafka topic
MY.EVENTS
myKafkaTrigger
trigger

81. © 2019 IBM Corporation
wsk trigger create myKafkaTrigger
-f /whisk.system/messaging/kafkaFeed
-p brokers '["9.174.23.208:3092"]'
-p topic MY.EVENTS

82. © 2019 IBM Corporation
Demo
event
Message
produced to
Kafka topic
MY.EVENTS
myKafkaTrigger
trigger

83. © 2019 IBM Corporation
Demo
event
Message
produced to
Kafka topic
MY.EVENTS
myKafkaTrigger
trigger
sendToSlack
action
action

84. © 2019 IBM Corporation
Demo
event
Message
produced to
Kafka topic
MY.EVENTS
myKafkaTrigger
trigger
sendToSlack
action
action

85. © 2019 IBM Corporation
wsk action create sendToSlack slack-action.js

86. © 2019 IBM Corporation
Demo
event
Message
produced to
Kafka topic
MY.EVENTS
myKafkaTrigger
trigger
sendToSlack
action
action

87. © 2019 IBM Corporation
Demo
ruleevent
Message
produced to
Kafka topic
MY.EVENTS
myKafkaTrigger
trigger
slackOnKafkaMessages
sendToSlack
action
action

88. © 2019 IBM Corporation
wsk rule create slackOnKafkaMessages
myKafkaTrigger
sendToSlack

89. © 2019 IBM Corporation
Demo
ruleevent
Message
produced to
Kafka topic
MY.EVENTS
myKafkaTrigger
trigger
slackOnKafkaMessages
sendToSlack
action
action

90. © 2019 IBM Corporation

91. © 2019 IBM Corporation
Reminder

92. © 2019 IBM Corporation
Demos – using a hosted service

93. © 2019 IBM Corporation
Demo
Orders Voyages Ships Containers Alerts
VoyagesOrder
Order
Management
Microservice
Fleet/Ships
Simulator
Microservice
Voyages
Management
Microservice
Kafka Streams
Kafka

94. © 2019 IBM Corporation

95. © 2019 IBM Corporation
Demo

96. © 2019 IBM Corporation
Demo
ruleevent
Message
produced to
Kafka topic
“alerts”
kafkaAlert
trigger
processAlert
action
action

97. © 2019 IBM Corporation

98. © 2019 IBM Corporation
Demo
ruleevent
Message
produced to
Kafka topic
“alerts”
kafkaAlert
trigger
processAlert
action
action

99. © 2019 IBM Corporation
Orders Voyages Ships Containers Alerts
VoyagesOrder
Order
Management
Microservice
Fleet/Ships
Simulator
Microservice
Voyages
Management
Microservice
Kafka
Kafka Streams

100. © 2019 IBM Corporation
Summary

101. © 2019 IBM Corporation
Summary / roundup
§An architectural pattern where your code is run:
– in response to events
– only on-demand on a per-request basis

102. © 2019 IBM Corporation
Getting started
twitter.com/openwhisk
medium.com/openwhisk
slack.openwhisk.org
youtube.com/channel/UCbzgShnQk8F43NKsvEYA1SA
#openwhisk
developer.ibm.com/code/patterns/category/serverless
serverlessconf.io

103. © 2019 IBM Corporation
Thank you
Dale Lane
Software Engineer, IBM Event Streams
IBM Event Streams - ibm.com/cloud/event-streams
https://slack-invite-ibm-cloud-tech.mybluemix.net/