This document provides an overview and agenda for a workshop on getting started with microservices in Confluent Cloud. It discusses how monolithic architectures can be broken into independent microservices that are easier to deploy and scale. The workshop will include hands-on exercises to create topics in Confluent Cloud, build a ksqlDB application, ingest data using Kafka Connect, and consume topics from Node.js applications. Additional resources are also listed to help with developing event streaming applications on Confluent Cloud.
2. Copyright 2020, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc.
Agenda
2
01. Welcome 👋 and Confluent Team Introductions
02. Presentation: Microservices Overview & Use Cases
03. Workshop: Guided Hands-On Lab
04. Q&A Discussion & Additional Resources
4. Evolution of Monolith
APP APP APP APP
Monolith
Large self contained application
Complex app w/ highly interdependent parts
Poor developer experience and productivity
Slow feature delivery
Difficult to deploy w/o impacting other systems
Requires replication of entire application
Microservices
Multiple smaller, single function apps
Independently deployable and upgradable
Built around solving business capabilities
Can be built w/ different programming languages
Scalable and agile = faster feature delivery
Leverages newer development platforms
To microservices
To Microservices
5. Copyright 2020, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc.
But before they got there...
Their architecture looked something like this.
• Difficult to scale, low throughput
• Multiple single points of failure
• Non-persistent data
• Data integration complexity
• High cost of legacy systems
6. Copyright 2020, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc.
And with some time, it turned into something like this...
6
LINE OF BUSINESS 01 LINE OF BUSINESS 02 PUBLIC CLOUD
7. Copyright 2020, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc.
Confluent for Event Streaming
Real-time
Inventory
Real-time Fraud
Detection
Real-time
Customer 360
Machine
Learning
Models
Real-time Data
Transformation
LOB
apps
Event-Streaming Applications
Universal Event Pipeline
Hadoop ...
Device
Logs ... App ...
Data
Warehouse
Splunk ...
Data Stores Logs 3rd Party Apps Custom Apps /
Microservices
Mainframes Microservice
8. Copyright 2020, Confluent, Inc. All rights reserved. This document may not be reproduced in any manner without the express written permission of Confluent, Inc.
Connect Popular Data Sources & Sinks
Data Diode
175+
pre-built
connectors
80+ Confluent Supported 45+ Partner Supported, Confluent Verified
9. ksqlDB joins the two
streams in real-time
App: Stream of
shipments that arrive
App: Stream of purchases from
online and physical stores
Inventory
on hand
ksqlDB: Stream Processing Example
9
Database
10. Data exploration Data enrichment Streaming ETL
Filter, cleanse, mask Real-time monitoring Anomaly detection
ksqlDB Example Use Cases
10
12. Workshop Agenda
1. Log Into Confluent Cloud
2. Create an Environment and Cluster
3. Create a Topic using the Cloud UI
4. Launch Docker and Test CCloud CLI Utility
5. Create a Topic Using the CCloud Utility
6. Create a ksqlDB Application
7. Create Datagen Connectors for your Topics
8. Consume Topics Using NodeJs
9. Create Streams and Tables using ksqlDB
10. Clean Up Resources
11. Confluent Resources and Further Testing
Architecture Diagram
pageviews
Users
ksqlDB
Confluent
Cloud
13. C O N F I D E N T I A L
Confluent Cloud
Dynamic Performance & Elasticity
Automatic Balancing & Upgrades
Flexible DevOps Automation
CLI | Terraform | Operator | Ansible
GUI-driven Mgmt & Monitoring
Cloud Dashboard | Metrics API
Efficient
Operations at Scale
Freedom of Choice
Committer-driven Expertise
Event Streaming Database
ksql
Rich Pre-built Ecosystem
Connectors | Hub | Schema Registry
Multi-language Development
Non-Java Clients | REST Proxy
Global Resilience
Replicator
Data Compatibility
Schema Registry
Enterprise-grade Security
SSO/SAML | ACLs
ARCHITECT
OPERATOR
DEVELOPER
Open Source | Community licensed
Unrestricted
Developer Productivity
Production-stage
Prerequisites
Fully Managed Cloud Service
Self-managed Software
Training Partners
Enterprise
Support
Professional
Services
Apache Kafka
14. C O N F I D E N T I A L
Additional Resources
● Confluent Cloud Documentation
● Confluent Hub - All fully managed and self managed connectors
● KSQL Recipes Guide - ksqlDB examples and tutorials
● Manage Schemas in Confluent Cloud - Schema Registry
● Best Practices for Developing Apache Kafka Applications on Confluent Cloud - Get
started with connecting your applications to Confluent Cloud