SlideShare uma empresa Scribd logo
1 de 12
Baixar para ler offline
How Level Infinite
Implemented CQRS and Event
Sourcing on Top of Apache
Pulsar and ScyllaDB
Zhiwei Peng, Senior Software Engineer
Zhihao Chen, Senior Software Engineer
Lei Shi, Principal Software Engineer
About Proxima Beta & Level Infinite
■ Proxima Beta aka Tencent Interactive Entertainment Group Global
■ We are part of Tencent Games and in charge of global publishing under the
brand - Level Infinite
■ At this moment, we are running several well-known titles, such as PUBG
Mobile, Arena of Valor, and Tower of Fantasy
■ Everything we do at Level Infinite centers around supporting our teams and
studios to bring unique, exhilarating games to millions of players around
the world
■ Quick recap of our service architecture.
■ Why do we choose ScyllaDB as an event store?
■ What are extra benefits we could get by using ScyllaDB?
Agenda
See Also
https://streamnative.io/blog/case/2022-11-15-how-proxima-beta-implemented-cqrs-and-event-sourcing-on-top-of-apache-pulsar-and-scylladb/
CQRS and Event
Sourcing
Event
Processors
Game Servers Command APIs
Service Architecture
Table
Table
Query APIs
Query
Query
X Y
Y1 Y2
1 2
Y
X
Query Y
Bulk
Batch X
Batch Y
X Y Topic X Y
Topic X Y
Topic X Y
Consume
Produce
Topic
Topic X Y
Y1 Y2
Consume
Update
■ Clients keep sending activity events to
Apache Pulsar through Command APIs.
■ Each type of event has a corresponding
topic.
■ Event processors selectively subscribe
topics to consume activity events and
produce data points of metrics and findings.
■ Data points of metrics and findings are
synchronized into tables as records.
■ Clients periodically query against ScyllaDB
through Query APIs to get results.
Distributed Queue-Like
Event Store
Partition
Time-Series Events
■ Partitioning by sessions and using TimeUUID as
event identifiers.
■ Enabling Time-Window Compaction strategy.
CREATE TABLE IF NOT EXISTS events (
session_id uuid,
event_id timeuuid,
payload blob,
PRIMARY KEY(session_id, event_id)
) WITH gc_grace_seconds = 60 AND
default_time_to_live = 300 AND
compaction = {
'compaction_window_size': '1',
'compaction_window_unit': 'MINUTES',
'class': 'TimeWindowCompactionStrategy'
};
History Future
Phantom-Window
True Present
Pseudo Now
■ Tracking current position at client side
■ Beware of phantom read problem
Smaller EventId Greater EventId
Simplifying Global
Data Management
DC1
DC3
Global Tables
CREATE KEYSPACE "global"
WITH REPLICATION = {
'class' : 'NetworkTopologyStrategy',
'AP_SOUTHEAST_1' : 3, // Datacenter 1
'US_EAST_1' : 3, // Datacenter 2
'EU_CENTRAL_1' : 3, // Datacenter 3
'SA_EAST_1' : 3, // Datacenter 4
};
■ Read and write locally, access data globally.
■ No cloud vendor lock-in.
■ Making global configuration management easier.
DC2 DC4
■ Abstracting complex
configurations and regulations
away from apps.
■ Improving the testability of
complex data distributing
problems by separation of duties.
Keyspaces as Data Containers
DC2 - Region B
DC1 - Region A
Apps
Commands
Queries
Clients
■ When using ScyllaDB to handle time series data, don't
forget TimeWindowCompactionStrategy.
■ Try using keyspaces and data replication as a tool to
separate the duty of data distribution.
Takeaways
Thank you
Stay in touch
Lei Shi https://github.com/shileiyu
lei-shi-b72a0033/
Zhihao Chen
Zhiwei Peng zhiwei-peng-17255694/ https://github.com/zeewinpeng
zhihao-chen-75754713a/ https://github.com/haorenqq

Mais conteúdo relacionado

Semelhante a How Level Infinite Implemented CQRS and Event Sourcing on Top of Apache Pulsar and ScyllaDB

AWS re:Invent 2016| GAM303 | Develop Games Using Lumberyard and Leverage AWS ...
AWS re:Invent 2016| GAM303 | Develop Games Using Lumberyard and Leverage AWS ...AWS re:Invent 2016| GAM303 | Develop Games Using Lumberyard and Leverage AWS ...
AWS re:Invent 2016| GAM303 | Develop Games Using Lumberyard and Leverage AWS ...Amazon Web Services
 
Using Event Streams in Serverless Applications
Using Event Streams in Serverless ApplicationsUsing Event Streams in Serverless Applications
Using Event Streams in Serverless ApplicationsJonathan Dee
 
Building a Real-Time Analytics Application with Apache Pulsar and Apache Pinot
Building a Real-Time Analytics Application with  Apache Pulsar and Apache PinotBuilding a Real-Time Analytics Application with  Apache Pulsar and Apache Pinot
Building a Real-Time Analytics Application with Apache Pulsar and Apache PinotAltinity Ltd
 
OSA Con 2022 - Building a Real-time Analytics Application with Apache Pulsar ...
OSA Con 2022 - Building a Real-time Analytics Application with Apache Pulsar ...OSA Con 2022 - Building a Real-time Analytics Application with Apache Pulsar ...
OSA Con 2022 - Building a Real-time Analytics Application with Apache Pulsar ...Altinity Ltd
 
Build Low Latency, Windowless Event Processing Pipelines with Quine and ScyllaDB
Build Low Latency, Windowless Event Processing Pipelines with Quine and ScyllaDBBuild Low Latency, Windowless Event Processing Pipelines with Quine and ScyllaDB
Build Low Latency, Windowless Event Processing Pipelines with Quine and ScyllaDBScyllaDB
 
CQRS and Event Sourcing
CQRS and Event Sourcing CQRS and Event Sourcing
CQRS and Event Sourcing Inho Kang
 
[Hands-on] CQRS(Command Query Responsibility Segregation) 와 Event Sourcing 패턴 실습
[Hands-on] CQRS(Command Query Responsibility Segregation) 와 Event Sourcing 패턴 실습[Hands-on] CQRS(Command Query Responsibility Segregation) 와 Event Sourcing 패턴 실습
[Hands-on] CQRS(Command Query Responsibility Segregation) 와 Event Sourcing 패턴 실습Oracle Korea
 
SamzaSQL QCon'16 presentation
SamzaSQL QCon'16 presentationSamzaSQL QCon'16 presentation
SamzaSQL QCon'16 presentationYi Pan
 
Apache Pinot Meetup Sept02, 2020
Apache Pinot Meetup Sept02, 2020Apache Pinot Meetup Sept02, 2020
Apache Pinot Meetup Sept02, 2020Mayank Shrivastava
 
Mobile game architecture on GCP
Mobile game architecture on GCPMobile game architecture on GCP
Mobile game architecture on GCP명근 최
 
Azure Stream Analytics
Azure Stream AnalyticsAzure Stream Analytics
Azure Stream AnalyticsMarco Parenzan
 
WSO2Con USA 2015: WSO2 Analytics Platform - The One Stop Shop for All Your Da...
WSO2Con USA 2015: WSO2 Analytics Platform - The One Stop Shop for All Your Da...WSO2Con USA 2015: WSO2 Analytics Platform - The One Stop Shop for All Your Da...
WSO2Con USA 2015: WSO2 Analytics Platform - The One Stop Shop for All Your Da...WSO2
 
Craft CMS: Beyond the Small Business; Advanced tools and configurations
Craft CMS: Beyond the Small Business; Advanced tools and configurationsCraft CMS: Beyond the Small Business; Advanced tools and configurations
Craft CMS: Beyond the Small Business; Advanced tools and configurationsNate Iler
 
Kafka as an Event Store - is it Good Enough?
Kafka as an Event Store - is it Good Enough?Kafka as an Event Store - is it Good Enough?
Kafka as an Event Store - is it Good Enough?Guido Schmutz
 
Self Service Agile Infrastructure for Product Teams - Pop-up Loft Tel Aviv
Self Service Agile Infrastructure for Product Teams - Pop-up Loft Tel AvivSelf Service Agile Infrastructure for Product Teams - Pop-up Loft Tel Aviv
Self Service Agile Infrastructure for Product Teams - Pop-up Loft Tel AvivAmazon Web Services
 
Scaling up uber's real time data analytics
Scaling up uber's real time data analyticsScaling up uber's real time data analytics
Scaling up uber's real time data analyticsXiang Fu
 
Devoxx 2018 - Pivotal and AxonIQ - Quickstart your event driven architecture
Devoxx 2018 -  Pivotal and AxonIQ - Quickstart your event driven architectureDevoxx 2018 -  Pivotal and AxonIQ - Quickstart your event driven architecture
Devoxx 2018 - Pivotal and AxonIQ - Quickstart your event driven architectureBen Wilcock
 

Semelhante a How Level Infinite Implemented CQRS and Event Sourcing on Top of Apache Pulsar and ScyllaDB (20)

AWS re:Invent 2016| GAM303 | Develop Games Using Lumberyard and Leverage AWS ...
AWS re:Invent 2016| GAM303 | Develop Games Using Lumberyard and Leverage AWS ...AWS re:Invent 2016| GAM303 | Develop Games Using Lumberyard and Leverage AWS ...
AWS re:Invent 2016| GAM303 | Develop Games Using Lumberyard and Leverage AWS ...
 
Using Event Streams in Serverless Applications
Using Event Streams in Serverless ApplicationsUsing Event Streams in Serverless Applications
Using Event Streams in Serverless Applications
 
Building a Real-Time Analytics Application with Apache Pulsar and Apache Pinot
Building a Real-Time Analytics Application with  Apache Pulsar and Apache PinotBuilding a Real-Time Analytics Application with  Apache Pulsar and Apache Pinot
Building a Real-Time Analytics Application with Apache Pulsar and Apache Pinot
 
OSA Con 2022 - Building a Real-time Analytics Application with Apache Pulsar ...
OSA Con 2022 - Building a Real-time Analytics Application with Apache Pulsar ...OSA Con 2022 - Building a Real-time Analytics Application with Apache Pulsar ...
OSA Con 2022 - Building a Real-time Analytics Application with Apache Pulsar ...
 
Build Low Latency, Windowless Event Processing Pipelines with Quine and ScyllaDB
Build Low Latency, Windowless Event Processing Pipelines with Quine and ScyllaDBBuild Low Latency, Windowless Event Processing Pipelines with Quine and ScyllaDB
Build Low Latency, Windowless Event Processing Pipelines with Quine and ScyllaDB
 
CQRS and Event Sourcing
CQRS and Event Sourcing CQRS and Event Sourcing
CQRS and Event Sourcing
 
[Hands-on] CQRS(Command Query Responsibility Segregation) 와 Event Sourcing 패턴 실습
[Hands-on] CQRS(Command Query Responsibility Segregation) 와 Event Sourcing 패턴 실습[Hands-on] CQRS(Command Query Responsibility Segregation) 와 Event Sourcing 패턴 실습
[Hands-on] CQRS(Command Query Responsibility Segregation) 와 Event Sourcing 패턴 실습
 
SamzaSQL QCon'16 presentation
SamzaSQL QCon'16 presentationSamzaSQL QCon'16 presentation
SamzaSQL QCon'16 presentation
 
Apache Pinot Meetup Sept02, 2020
Apache Pinot Meetup Sept02, 2020Apache Pinot Meetup Sept02, 2020
Apache Pinot Meetup Sept02, 2020
 
Mobile game architecture on GCP
Mobile game architecture on GCPMobile game architecture on GCP
Mobile game architecture on GCP
 
Implementing Real-Time IoT Stream Processing in Azure
Implementing Real-Time IoT Stream Processing in Azure Implementing Real-Time IoT Stream Processing in Azure
Implementing Real-Time IoT Stream Processing in Azure
 
Azure Stream Analytics
Azure Stream AnalyticsAzure Stream Analytics
Azure Stream Analytics
 
WSO2Con USA 2015: WSO2 Analytics Platform - The One Stop Shop for All Your Da...
WSO2Con USA 2015: WSO2 Analytics Platform - The One Stop Shop for All Your Da...WSO2Con USA 2015: WSO2 Analytics Platform - The One Stop Shop for All Your Da...
WSO2Con USA 2015: WSO2 Analytics Platform - The One Stop Shop for All Your Da...
 
Craft CMS: Beyond the Small Business; Advanced tools and configurations
Craft CMS: Beyond the Small Business; Advanced tools and configurationsCraft CMS: Beyond the Small Business; Advanced tools and configurations
Craft CMS: Beyond the Small Business; Advanced tools and configurations
 
Kafka as an Event Store - is it Good Enough?
Kafka as an Event Store - is it Good Enough?Kafka as an Event Store - is it Good Enough?
Kafka as an Event Store - is it Good Enough?
 
Self Service Agile Infrastructure for Product Teams - Pop-up Loft Tel Aviv
Self Service Agile Infrastructure for Product Teams - Pop-up Loft Tel AvivSelf Service Agile Infrastructure for Product Teams - Pop-up Loft Tel Aviv
Self Service Agile Infrastructure for Product Teams - Pop-up Loft Tel Aviv
 
Scaling up uber's real time data analytics
Scaling up uber's real time data analyticsScaling up uber's real time data analytics
Scaling up uber's real time data analytics
 
Devoxx 2018 - Pivotal and AxonIQ - Quickstart your event driven architecture
Devoxx 2018 -  Pivotal and AxonIQ - Quickstart your event driven architectureDevoxx 2018 -  Pivotal and AxonIQ - Quickstart your event driven architecture
Devoxx 2018 - Pivotal and AxonIQ - Quickstart your event driven architecture
 
Patterns for Building Streaming Apps
Patterns for Building Streaming AppsPatterns for Building Streaming Apps
Patterns for Building Streaming Apps
 
Siddhi - cloud-native stream processor
Siddhi - cloud-native stream processorSiddhi - cloud-native stream processor
Siddhi - cloud-native stream processor
 

Mais de ScyllaDB

What Developers Need to Unlearn for High Performance NoSQL
What Developers Need to Unlearn for High Performance NoSQLWhat Developers Need to Unlearn for High Performance NoSQL
What Developers Need to Unlearn for High Performance NoSQLScyllaDB
 
Low Latency at Extreme Scale: Proven Practices & Pitfalls
Low Latency at Extreme Scale: Proven Practices & PitfallsLow Latency at Extreme Scale: Proven Practices & Pitfalls
Low Latency at Extreme Scale: Proven Practices & PitfallsScyllaDB
 
Dissecting Real-World Database Performance Dilemmas
Dissecting Real-World Database Performance DilemmasDissecting Real-World Database Performance Dilemmas
Dissecting Real-World Database Performance DilemmasScyllaDB
 
Beyond Linear Scaling: A New Path for Performance with ScyllaDB
Beyond Linear Scaling: A New Path for Performance with ScyllaDBBeyond Linear Scaling: A New Path for Performance with ScyllaDB
Beyond Linear Scaling: A New Path for Performance with ScyllaDBScyllaDB
 
Dissecting Real-World Database Performance Dilemmas
Dissecting Real-World Database Performance DilemmasDissecting Real-World Database Performance Dilemmas
Dissecting Real-World Database Performance DilemmasScyllaDB
 
Database Performance at Scale Masterclass: Workload Characteristics by Felipe...
Database Performance at Scale Masterclass: Workload Characteristics by Felipe...Database Performance at Scale Masterclass: Workload Characteristics by Felipe...
Database Performance at Scale Masterclass: Workload Characteristics by Felipe...ScyllaDB
 
Database Performance at Scale Masterclass: Database Internals by Pavel Emelya...
Database Performance at Scale Masterclass: Database Internals by Pavel Emelya...Database Performance at Scale Masterclass: Database Internals by Pavel Emelya...
Database Performance at Scale Masterclass: Database Internals by Pavel Emelya...ScyllaDB
 
Database Performance at Scale Masterclass: Driver Strategies by Piotr Sarna
Database Performance at Scale Masterclass: Driver Strategies by Piotr SarnaDatabase Performance at Scale Masterclass: Driver Strategies by Piotr Sarna
Database Performance at Scale Masterclass: Driver Strategies by Piotr SarnaScyllaDB
 
Replacing Your Cache with ScyllaDB
Replacing Your Cache with ScyllaDBReplacing Your Cache with ScyllaDB
Replacing Your Cache with ScyllaDBScyllaDB
 
Powering Real-Time Apps with ScyllaDB_ Low Latency & Linear Scalability
Powering Real-Time Apps with ScyllaDB_ Low Latency & Linear ScalabilityPowering Real-Time Apps with ScyllaDB_ Low Latency & Linear Scalability
Powering Real-Time Apps with ScyllaDB_ Low Latency & Linear ScalabilityScyllaDB
 
7 Reasons Not to Put an External Cache in Front of Your Database.pptx
7 Reasons Not to Put an External Cache in Front of Your Database.pptx7 Reasons Not to Put an External Cache in Front of Your Database.pptx
7 Reasons Not to Put an External Cache in Front of Your Database.pptxScyllaDB
 
Getting the most out of ScyllaDB
Getting the most out of ScyllaDBGetting the most out of ScyllaDB
Getting the most out of ScyllaDBScyllaDB
 
NoSQL Database Migration Masterclass - Session 2: The Anatomy of a Migration
NoSQL Database Migration Masterclass - Session 2: The Anatomy of a MigrationNoSQL Database Migration Masterclass - Session 2: The Anatomy of a Migration
NoSQL Database Migration Masterclass - Session 2: The Anatomy of a MigrationScyllaDB
 
NoSQL Database Migration Masterclass - Session 3: Migration Logistics
NoSQL Database Migration Masterclass - Session 3: Migration LogisticsNoSQL Database Migration Masterclass - Session 3: Migration Logistics
NoSQL Database Migration Masterclass - Session 3: Migration LogisticsScyllaDB
 
NoSQL Data Migration Masterclass - Session 1 Migration Strategies and Challenges
NoSQL Data Migration Masterclass - Session 1 Migration Strategies and ChallengesNoSQL Data Migration Masterclass - Session 1 Migration Strategies and Challenges
NoSQL Data Migration Masterclass - Session 1 Migration Strategies and ChallengesScyllaDB
 
ScyllaDB Virtual Workshop
ScyllaDB Virtual WorkshopScyllaDB Virtual Workshop
ScyllaDB Virtual WorkshopScyllaDB
 
DBaaS in the Real World: Risks, Rewards & Tradeoffs
DBaaS in the Real World: Risks, Rewards & TradeoffsDBaaS in the Real World: Risks, Rewards & Tradeoffs
DBaaS in the Real World: Risks, Rewards & TradeoffsScyllaDB
 
Build Low-Latency Applications in Rust on ScyllaDB
Build Low-Latency Applications in Rust on ScyllaDBBuild Low-Latency Applications in Rust on ScyllaDB
Build Low-Latency Applications in Rust on ScyllaDBScyllaDB
 
NoSQL Data Modeling 101
NoSQL Data Modeling 101NoSQL Data Modeling 101
NoSQL Data Modeling 101ScyllaDB
 
Top NoSQL Data Modeling Mistakes
Top NoSQL Data Modeling MistakesTop NoSQL Data Modeling Mistakes
Top NoSQL Data Modeling MistakesScyllaDB
 

Mais de ScyllaDB (20)

What Developers Need to Unlearn for High Performance NoSQL
What Developers Need to Unlearn for High Performance NoSQLWhat Developers Need to Unlearn for High Performance NoSQL
What Developers Need to Unlearn for High Performance NoSQL
 
Low Latency at Extreme Scale: Proven Practices & Pitfalls
Low Latency at Extreme Scale: Proven Practices & PitfallsLow Latency at Extreme Scale: Proven Practices & Pitfalls
Low Latency at Extreme Scale: Proven Practices & Pitfalls
 
Dissecting Real-World Database Performance Dilemmas
Dissecting Real-World Database Performance DilemmasDissecting Real-World Database Performance Dilemmas
Dissecting Real-World Database Performance Dilemmas
 
Beyond Linear Scaling: A New Path for Performance with ScyllaDB
Beyond Linear Scaling: A New Path for Performance with ScyllaDBBeyond Linear Scaling: A New Path for Performance with ScyllaDB
Beyond Linear Scaling: A New Path for Performance with ScyllaDB
 
Dissecting Real-World Database Performance Dilemmas
Dissecting Real-World Database Performance DilemmasDissecting Real-World Database Performance Dilemmas
Dissecting Real-World Database Performance Dilemmas
 
Database Performance at Scale Masterclass: Workload Characteristics by Felipe...
Database Performance at Scale Masterclass: Workload Characteristics by Felipe...Database Performance at Scale Masterclass: Workload Characteristics by Felipe...
Database Performance at Scale Masterclass: Workload Characteristics by Felipe...
 
Database Performance at Scale Masterclass: Database Internals by Pavel Emelya...
Database Performance at Scale Masterclass: Database Internals by Pavel Emelya...Database Performance at Scale Masterclass: Database Internals by Pavel Emelya...
Database Performance at Scale Masterclass: Database Internals by Pavel Emelya...
 
Database Performance at Scale Masterclass: Driver Strategies by Piotr Sarna
Database Performance at Scale Masterclass: Driver Strategies by Piotr SarnaDatabase Performance at Scale Masterclass: Driver Strategies by Piotr Sarna
Database Performance at Scale Masterclass: Driver Strategies by Piotr Sarna
 
Replacing Your Cache with ScyllaDB
Replacing Your Cache with ScyllaDBReplacing Your Cache with ScyllaDB
Replacing Your Cache with ScyllaDB
 
Powering Real-Time Apps with ScyllaDB_ Low Latency & Linear Scalability
Powering Real-Time Apps with ScyllaDB_ Low Latency & Linear ScalabilityPowering Real-Time Apps with ScyllaDB_ Low Latency & Linear Scalability
Powering Real-Time Apps with ScyllaDB_ Low Latency & Linear Scalability
 
7 Reasons Not to Put an External Cache in Front of Your Database.pptx
7 Reasons Not to Put an External Cache in Front of Your Database.pptx7 Reasons Not to Put an External Cache in Front of Your Database.pptx
7 Reasons Not to Put an External Cache in Front of Your Database.pptx
 
Getting the most out of ScyllaDB
Getting the most out of ScyllaDBGetting the most out of ScyllaDB
Getting the most out of ScyllaDB
 
NoSQL Database Migration Masterclass - Session 2: The Anatomy of a Migration
NoSQL Database Migration Masterclass - Session 2: The Anatomy of a MigrationNoSQL Database Migration Masterclass - Session 2: The Anatomy of a Migration
NoSQL Database Migration Masterclass - Session 2: The Anatomy of a Migration
 
NoSQL Database Migration Masterclass - Session 3: Migration Logistics
NoSQL Database Migration Masterclass - Session 3: Migration LogisticsNoSQL Database Migration Masterclass - Session 3: Migration Logistics
NoSQL Database Migration Masterclass - Session 3: Migration Logistics
 
NoSQL Data Migration Masterclass - Session 1 Migration Strategies and Challenges
NoSQL Data Migration Masterclass - Session 1 Migration Strategies and ChallengesNoSQL Data Migration Masterclass - Session 1 Migration Strategies and Challenges
NoSQL Data Migration Masterclass - Session 1 Migration Strategies and Challenges
 
ScyllaDB Virtual Workshop
ScyllaDB Virtual WorkshopScyllaDB Virtual Workshop
ScyllaDB Virtual Workshop
 
DBaaS in the Real World: Risks, Rewards & Tradeoffs
DBaaS in the Real World: Risks, Rewards & TradeoffsDBaaS in the Real World: Risks, Rewards & Tradeoffs
DBaaS in the Real World: Risks, Rewards & Tradeoffs
 
Build Low-Latency Applications in Rust on ScyllaDB
Build Low-Latency Applications in Rust on ScyllaDBBuild Low-Latency Applications in Rust on ScyllaDB
Build Low-Latency Applications in Rust on ScyllaDB
 
NoSQL Data Modeling 101
NoSQL Data Modeling 101NoSQL Data Modeling 101
NoSQL Data Modeling 101
 
Top NoSQL Data Modeling Mistakes
Top NoSQL Data Modeling MistakesTop NoSQL Data Modeling Mistakes
Top NoSQL Data Modeling Mistakes
 

Último

UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.francesco barbera
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
RAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AIRAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AIUdaiappa Ramachandran
 
Spring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfSpring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfAnna Loughnan Colquhoun
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataSafe Software
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?SANGHEE SHIN
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesThousandEyes
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 

Último (20)

UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
RAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AIRAG Patterns and Vector Search in Generative AI
RAG Patterns and Vector Search in Generative AI
 
Spring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfSpring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdf
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 

How Level Infinite Implemented CQRS and Event Sourcing on Top of Apache Pulsar and ScyllaDB

  • 1. How Level Infinite Implemented CQRS and Event Sourcing on Top of Apache Pulsar and ScyllaDB Zhiwei Peng, Senior Software Engineer Zhihao Chen, Senior Software Engineer Lei Shi, Principal Software Engineer
  • 2. About Proxima Beta & Level Infinite ■ Proxima Beta aka Tencent Interactive Entertainment Group Global ■ We are part of Tencent Games and in charge of global publishing under the brand - Level Infinite ■ At this moment, we are running several well-known titles, such as PUBG Mobile, Arena of Valor, and Tower of Fantasy ■ Everything we do at Level Infinite centers around supporting our teams and studios to bring unique, exhilarating games to millions of players around the world
  • 3. ■ Quick recap of our service architecture. ■ Why do we choose ScyllaDB as an event store? ■ What are extra benefits we could get by using ScyllaDB? Agenda See Also https://streamnative.io/blog/case/2022-11-15-how-proxima-beta-implemented-cqrs-and-event-sourcing-on-top-of-apache-pulsar-and-scylladb/
  • 5. Event Processors Game Servers Command APIs Service Architecture Table Table Query APIs Query Query X Y Y1 Y2 1 2 Y X Query Y Bulk Batch X Batch Y X Y Topic X Y Topic X Y Topic X Y Consume Produce Topic Topic X Y Y1 Y2 Consume Update ■ Clients keep sending activity events to Apache Pulsar through Command APIs. ■ Each type of event has a corresponding topic. ■ Event processors selectively subscribe topics to consume activity events and produce data points of metrics and findings. ■ Data points of metrics and findings are synchronized into tables as records. ■ Clients periodically query against ScyllaDB through Query APIs to get results.
  • 7. Partition Time-Series Events ■ Partitioning by sessions and using TimeUUID as event identifiers. ■ Enabling Time-Window Compaction strategy. CREATE TABLE IF NOT EXISTS events ( session_id uuid, event_id timeuuid, payload blob, PRIMARY KEY(session_id, event_id) ) WITH gc_grace_seconds = 60 AND default_time_to_live = 300 AND compaction = { 'compaction_window_size': '1', 'compaction_window_unit': 'MINUTES', 'class': 'TimeWindowCompactionStrategy' }; History Future Phantom-Window True Present Pseudo Now ■ Tracking current position at client side ■ Beware of phantom read problem Smaller EventId Greater EventId
  • 9. DC1 DC3 Global Tables CREATE KEYSPACE "global" WITH REPLICATION = { 'class' : 'NetworkTopologyStrategy', 'AP_SOUTHEAST_1' : 3, // Datacenter 1 'US_EAST_1' : 3, // Datacenter 2 'EU_CENTRAL_1' : 3, // Datacenter 3 'SA_EAST_1' : 3, // Datacenter 4 }; ■ Read and write locally, access data globally. ■ No cloud vendor lock-in. ■ Making global configuration management easier. DC2 DC4
  • 10. ■ Abstracting complex configurations and regulations away from apps. ■ Improving the testability of complex data distributing problems by separation of duties. Keyspaces as Data Containers DC2 - Region B DC1 - Region A Apps Commands Queries Clients
  • 11. ■ When using ScyllaDB to handle time series data, don't forget TimeWindowCompactionStrategy. ■ Try using keyspaces and data replication as a tool to separate the duty of data distribution. Takeaways
  • 12. Thank you Stay in touch Lei Shi https://github.com/shileiyu lei-shi-b72a0033/ Zhihao Chen Zhiwei Peng zhiwei-peng-17255694/ https://github.com/zeewinpeng zhihao-chen-75754713a/ https://github.com/haorenqq