SlideShare uma empresa Scribd logo
1 de 22
Baixar para ler offline
Piotr Sarna - Principal Software Engineer, ScyllaDB
Felipe Cardeneti Mendes - Solutions Architect, ScyllaDB
Andrew Brown - Director of Product Growth, ScyllaDB
Virtual Developer Workshop
Build Low-Latency
Applications in Rust on ScyllaDB
Presenters
2
Piotr Sarna
Piotr is a software engineer very keen on open-source projects, C++
and Rust.
Felipe Cardeneti Mendes
Felipe Mendes is an IT Specialist with years of experience with Linux. In
ScyllaDB, he works as a Solutions Architect
Andrew Brown
Andrew has been working with distributed systems in roles across
product development and product marketing.
Agenda ● Background: Why Rust, why ScyllaDB?
● The ScyllaDB Rust driver
● The Rust sample app walkthrough
● Profiling Rust performance
IOT Rust Application Setup
$ git clone https://github.com/fee-mendes/rust-driver-example/
$ cd rust-driver-example/docker-compose
$ docker-compose up -d
$ docker exec -it rust-app /bin/bash
Minimum requirements:
- Linux or OSX x86
- Docker installed and setup
- Quadcore CPU
- 2 GB of memory available
- Windows is not supported
Join the #rust-driver channel on ScyllaDB Slack to discuss any issues or questions you might have.
Low latency, close
to hardware schedulers
Perfect horizontal & Vertical scale
5
1000 Nodes Cluster
2000 Cluster
K8S Deployment
60TB per Node 256 Cores per Node
1B Operations
per Second
About ScyllaDB: Fast and Scalable
6
Poll:
How proficient are you with the
Rust language?
+ Asynchronous, non-blocking runtimes
+ Tokio: Most widely used Rust runtime
+ Seastar: C++ runtime for ScyllaDB
+ Fast, flexible, and reliable
+ Scalable, allows high concurrency and low latency
+ Green and sustainable
Why Rust? Why Tokio? Why ScyllaDB?
Rust Driver ScyllaDB
The effort started during a 2020 internal hackathon at ScyllaDB.
The existing CQL drivers (cassandra-cpp, cdrs) in the Rust ecosystem were not good
enough for us:
● Not fully async
● Unsatisfactory performance
● Not scalable enough
● Known, long-standing bugs
● No support for ScyllaDB-specific optimizations
○ e.g., Shard-per-core
● etc.
Some history
Our design goals:
● fully asynchronous
● token-aware, shard-aware
● ease of use
● maximizing performance
● minimizing overheads
● extensibility
● an extremely cool logo
Based on the principles above, we created the ScyllaDB Rust Driver.
Some history
● Most popular async runtime, which should translate to best adoption
● It would be tempting to write the driver in a runtime-agnostic way, but
it's hard:
○ Not all API's are well defined yet
○ Tokio offers quite complete support for TCP communication,
timeouts and other useful abstractions
● Very actively developed
Why Tokio?
Let’s Code
11
IOT Application Overview
Metric Collector Metric Reader UUID Finder
Write to ScyllaDB in parallel
Deploy schema (ks/tables)
Generate data:
100 devices
3 days
Every 5 minutes
Device metrics aggregator
Analytics sample
Split token-ring in small parts:
Efficient full table scan
token() function usage
BYPASS CACHE
Single device queries
Real-time sample
Partition scan:
MAX(), AVG(), MIN()
Range queries
Date/Time handling
Poll:
What databases do you use
(or are planning to use)
for Rust Apps?:
13
Profiling
14
tokio-console is a very cool project which brings perf/top capabilities to async Rust.
Prerequisites:
Installation:
$ cargo install tokio-console
Cargo.toml:
[dependencies]
console-subscriber = "0.1"
Code:
console_subscriber::init();
Compilation flags:
RUSTFLAGS="--cfg tokio_unstable" cargo run <your-project>
Profiling: tokio-console
Demo
Profiling: tokio-console
Rust ecosystem has a very convenient way of generating flamegraphs: cargo
flamegraph
Usage:
$ cargo install flamegraph
$ cargo flamegraph your-project
Profiling: cargo flamegraph
Demo
Profiling: cargo flamegraph
On Linux, flamegraph scripts use perf underneath. Perf is a well-known and very powerful
profiling tool with direct support in the kernel. Rust programs, since they're compiled via LLVM,
are perfectly capable candidates for profiling with perf.
Prerequisites:
Cargo.toml (the root one in case you use multiple workspaces):
[profile.release]
lto = false
debug = true
System (Linux-specific):
$ echo 0 | sudo tee /proc/sys/kernel/perf_event_paranoid
$ echo 0 | sudo tee /proc/sys/kernel/kptr_restrict
Profiling: perf
By the way, when run on Linux, cargo flamegraph leaves a side effect - a perf.data file,
which is a result format from calling perf record. Here's another example:
$ perf record -g ./target/release/metric-collector
$ perf report -g
Profiling: perf
Felipe Cardeneti
Mendes
felipemendes@scylladb.com
Piotr Sarna
sarna@scylladb.com
Q&A
Andrew Brown
andrew.brown@scylladb.com
United States
2445 Faber St, Suite #200
Palo Alto, CA USA 94303
Israel
Maskit 4
Herzliya, Israel 4673304
www.scylladb.com
@scylladb
Thank You!

Mais conteúdo relacionado

Mais procurados

Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
TO THE NEW | Technology
 

Mais procurados (20)

RocksDB detail
RocksDB detailRocksDB detail
RocksDB detail
 
BPF / XDP 8월 세미나 KossLab
BPF / XDP 8월 세미나 KossLabBPF / XDP 8월 세미나 KossLab
BPF / XDP 8월 세미나 KossLab
 
Under the Hood of a Shard-per-Core Database Architecture
Under the Hood of a Shard-per-Core Database ArchitectureUnder the Hood of a Shard-per-Core Database Architecture
Under the Hood of a Shard-per-Core Database Architecture
 
Scylla Summit 2022: How to Migrate a Counter Table for 68 Billion Records
Scylla Summit 2022: How to Migrate a Counter Table for 68 Billion RecordsScylla Summit 2022: How to Migrate a Counter Table for 68 Billion Records
Scylla Summit 2022: How to Migrate a Counter Table for 68 Billion Records
 
Implementing Highly Performant Distributed Aggregates
Implementing Highly Performant Distributed AggregatesImplementing Highly Performant Distributed Aggregates
Implementing Highly Performant Distributed Aggregates
 
Starting up Containers Super Fast With Lazy Pulling of Images
Starting up Containers Super Fast With Lazy Pulling of ImagesStarting up Containers Super Fast With Lazy Pulling of Images
Starting up Containers Super Fast With Lazy Pulling of Images
 
OpenShift Container Platform 4.12 Release Notes
OpenShift Container Platform 4.12 Release NotesOpenShift Container Platform 4.12 Release Notes
OpenShift Container Platform 4.12 Release Notes
 
Scylla Summit 2022: Scylla 5.0 New Features, Part 1
Scylla Summit 2022: Scylla 5.0 New Features, Part 1Scylla Summit 2022: Scylla 5.0 New Features, Part 1
Scylla Summit 2022: Scylla 5.0 New Features, Part 1
 
MySQL InnoDB Cluster HA Overview & Demo
MySQL InnoDB Cluster HA Overview & DemoMySQL InnoDB Cluster HA Overview & Demo
MySQL InnoDB Cluster HA Overview & Demo
 
LISA2019 Linux Systems Performance
LISA2019 Linux Systems PerformanceLISA2019 Linux Systems Performance
LISA2019 Linux Systems Performance
 
The Columnar Era: Leveraging Parquet, Arrow and Kudu for High-Performance Ana...
The Columnar Era: Leveraging Parquet, Arrow and Kudu for High-Performance Ana...The Columnar Era: Leveraging Parquet, Arrow and Kudu for High-Performance Ana...
The Columnar Era: Leveraging Parquet, Arrow and Kudu for High-Performance Ana...
 
Understanding Data Partitioning and Replication in Apache Cassandra
Understanding Data Partitioning and Replication in Apache CassandraUnderstanding Data Partitioning and Replication in Apache Cassandra
Understanding Data Partitioning and Replication in Apache Cassandra
 
Introduction to Redis
Introduction to RedisIntroduction to Redis
Introduction to Redis
 
Faster Container Image Distribution on a Variety of Tools with Lazy Pulling
Faster Container Image Distribution on a Variety of Tools with Lazy PullingFaster Container Image Distribution on a Variety of Tools with Lazy Pulling
Faster Container Image Distribution on a Variety of Tools with Lazy Pulling
 
Kubernetes 101 for Beginners
Kubernetes 101 for BeginnersKubernetes 101 for Beginners
Kubernetes 101 for Beginners
 
USENIX ATC 2017: Visualizing Performance with Flame Graphs
USENIX ATC 2017: Visualizing Performance with Flame GraphsUSENIX ATC 2017: Visualizing Performance with Flame Graphs
USENIX ATC 2017: Visualizing Performance with Flame Graphs
 
Introduction to Kafka and Zookeeper
Introduction to Kafka and ZookeeperIntroduction to Kafka and Zookeeper
Introduction to Kafka and Zookeeper
 
Docker, Linux Containers (LXC), and security
Docker, Linux Containers (LXC), and securityDocker, Linux Containers (LXC), and security
Docker, Linux Containers (LXC), and security
 
Deep Dive into Apache Kafka
Deep Dive into Apache KafkaDeep Dive into Apache Kafka
Deep Dive into Apache Kafka
 
Scaling for Performance
Scaling for PerformanceScaling for Performance
Scaling for Performance
 

Semelhante a Build Low-Latency Applications in Rust on ScyllaDB

Semelhante a Build Low-Latency Applications in Rust on ScyllaDB (20)

Build Low-Latency Applications in Rust on ScyllaDB
Build Low-Latency Applications in Rust on ScyllaDBBuild Low-Latency Applications in Rust on ScyllaDB
Build Low-Latency Applications in Rust on ScyllaDB
 
Build Low-Latency Applications in Rust on ScyllaDB
Build Low-Latency Applications in Rust on ScyllaDBBuild Low-Latency Applications in Rust on ScyllaDB
Build Low-Latency Applications in Rust on ScyllaDB
 
ScyllaDB V Developer Deep Dive Series: Rust-Based Drivers and UDFs with WebAs...
ScyllaDB V Developer Deep Dive Series: Rust-Based Drivers and UDFs with WebAs...ScyllaDB V Developer Deep Dive Series: Rust-Based Drivers and UDFs with WebAs...
ScyllaDB V Developer Deep Dive Series: Rust-Based Drivers and UDFs with WebAs...
 
Optimizing Performance in Rust for Low-Latency Database Drivers
Optimizing Performance in Rust for Low-Latency Database DriversOptimizing Performance in Rust for Low-Latency Database Drivers
Optimizing Performance in Rust for Low-Latency Database Drivers
 
DevEx | there’s no place like k3s
DevEx | there’s no place like k3sDevEx | there’s no place like k3s
DevEx | there’s no place like k3s
 
Embedded Rust
Embedded RustEmbedded Rust
Embedded Rust
 
Flutter Vikings 2022 - Full Stack Dart
Flutter Vikings 2022  - Full Stack DartFlutter Vikings 2022  - Full Stack Dart
Flutter Vikings 2022 - Full Stack Dart
 
Accelerate your development with Docker
Accelerate your development with DockerAccelerate your development with Docker
Accelerate your development with Docker
 
Accelerate your software development with Docker
Accelerate your software development with DockerAccelerate your software development with Docker
Accelerate your software development with Docker
 
Using Docker For Development
Using Docker For DevelopmentUsing Docker For Development
Using Docker For Development
 
Scylla Summit 2022: ScyllaDB Rust Driver: One Driver to Rule Them All
Scylla Summit 2022: ScyllaDB Rust Driver: One Driver to Rule Them AllScylla Summit 2022: ScyllaDB Rust Driver: One Driver to Rule Them All
Scylla Summit 2022: ScyllaDB Rust Driver: One Driver to Rule Them All
 
Настройка окружения для кросскомпиляции проектов на основе docker'a
Настройка окружения для кросскомпиляции проектов на основе docker'aНастройка окружения для кросскомпиляции проектов на основе docker'a
Настройка окружения для кросскомпиляции проектов на основе docker'a
 
QConSF 2022 - Backends in Dart
QConSF 2022 - Backends in DartQConSF 2022 - Backends in Dart
QConSF 2022 - Backends in Dart
 
Spark Summit EU 2015: Lessons from 300+ production users
Spark Summit EU 2015: Lessons from 300+ production usersSpark Summit EU 2015: Lessons from 300+ production users
Spark Summit EU 2015: Lessons from 300+ production users
 
Dependencies Managers in C/C++. Using stdcpp 2014
Dependencies Managers in C/C++. Using stdcpp 2014Dependencies Managers in C/C++. Using stdcpp 2014
Dependencies Managers in C/C++. Using stdcpp 2014
 
Scaleable PHP Applications in Kubernetes
Scaleable PHP Applications in KubernetesScaleable PHP Applications in Kubernetes
Scaleable PHP Applications in Kubernetes
 
State of Containers and the Convergence of HPC and BigData
State of Containers and the Convergence of HPC and BigDataState of Containers and the Convergence of HPC and BigData
State of Containers and the Convergence of HPC and BigData
 
Learning Rust the Hard Way for a Production Kafka + ScyllaDB Pipeline
Learning Rust the Hard Way for a Production Kafka + ScyllaDB PipelineLearning Rust the Hard Way for a Production Kafka + ScyllaDB Pipeline
Learning Rust the Hard Way for a Production Kafka + ScyllaDB Pipeline
 
[RUHR] DRPSL- Drupal on kubernetes in production. What should you do_know_.pdf
[RUHR] DRPSL- Drupal on kubernetes in production. What should you do_know_.pdf[RUHR] DRPSL- Drupal on kubernetes in production. What should you do_know_.pdf
[RUHR] DRPSL- Drupal on kubernetes in production. What should you do_know_.pdf
 
UplinQ - qualcomm® hexagon™ sdk optimize your multimedia solutions
UplinQ - qualcomm® hexagon™ sdk optimize your multimedia solutionsUplinQ - qualcomm® hexagon™ sdk optimize your multimedia solutions
UplinQ - qualcomm® hexagon™ sdk optimize your multimedia solutions
 

Mais de ScyllaDB

Mais de ScyllaDB (20)

Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
What Developers Need to Unlearn for High Performance NoSQL
What Developers Need to Unlearn for High Performance NoSQLWhat Developers Need to Unlearn for High Performance NoSQL
What Developers Need to Unlearn for High Performance NoSQL
 
Low Latency at Extreme Scale: Proven Practices & Pitfalls
Low Latency at Extreme Scale: Proven Practices & PitfallsLow Latency at Extreme Scale: Proven Practices & Pitfalls
Low Latency at Extreme Scale: Proven Practices & Pitfalls
 
Dissecting Real-World Database Performance Dilemmas
Dissecting Real-World Database Performance DilemmasDissecting Real-World Database Performance Dilemmas
Dissecting Real-World Database Performance Dilemmas
 
Beyond Linear Scaling: A New Path for Performance with ScyllaDB
Beyond Linear Scaling: A New Path for Performance with ScyllaDBBeyond Linear Scaling: A New Path for Performance with ScyllaDB
Beyond Linear Scaling: A New Path for Performance with ScyllaDB
 
Dissecting Real-World Database Performance Dilemmas
Dissecting Real-World Database Performance DilemmasDissecting Real-World Database Performance Dilemmas
Dissecting Real-World Database Performance Dilemmas
 
Database Performance at Scale Masterclass: Workload Characteristics by Felipe...
Database Performance at Scale Masterclass: Workload Characteristics by Felipe...Database Performance at Scale Masterclass: Workload Characteristics by Felipe...
Database Performance at Scale Masterclass: Workload Characteristics by Felipe...
 
Database Performance at Scale Masterclass: Database Internals by Pavel Emelya...
Database Performance at Scale Masterclass: Database Internals by Pavel Emelya...Database Performance at Scale Masterclass: Database Internals by Pavel Emelya...
Database Performance at Scale Masterclass: Database Internals by Pavel Emelya...
 
Database Performance at Scale Masterclass: Driver Strategies by Piotr Sarna
Database Performance at Scale Masterclass: Driver Strategies by Piotr SarnaDatabase Performance at Scale Masterclass: Driver Strategies by Piotr Sarna
Database Performance at Scale Masterclass: Driver Strategies by Piotr Sarna
 
Replacing Your Cache with ScyllaDB
Replacing Your Cache with ScyllaDBReplacing Your Cache with ScyllaDB
Replacing Your Cache with ScyllaDB
 
Powering Real-Time Apps with ScyllaDB_ Low Latency & Linear Scalability
Powering Real-Time Apps with ScyllaDB_ Low Latency & Linear ScalabilityPowering Real-Time Apps with ScyllaDB_ Low Latency & Linear Scalability
Powering Real-Time Apps with ScyllaDB_ Low Latency & Linear Scalability
 
7 Reasons Not to Put an External Cache in Front of Your Database.pptx
7 Reasons Not to Put an External Cache in Front of Your Database.pptx7 Reasons Not to Put an External Cache in Front of Your Database.pptx
7 Reasons Not to Put an External Cache in Front of Your Database.pptx
 
Getting the most out of ScyllaDB
Getting the most out of ScyllaDBGetting the most out of ScyllaDB
Getting the most out of ScyllaDB
 
NoSQL Database Migration Masterclass - Session 2: The Anatomy of a Migration
NoSQL Database Migration Masterclass - Session 2: The Anatomy of a MigrationNoSQL Database Migration Masterclass - Session 2: The Anatomy of a Migration
NoSQL Database Migration Masterclass - Session 2: The Anatomy of a Migration
 
NoSQL Database Migration Masterclass - Session 3: Migration Logistics
NoSQL Database Migration Masterclass - Session 3: Migration LogisticsNoSQL Database Migration Masterclass - Session 3: Migration Logistics
NoSQL Database Migration Masterclass - Session 3: Migration Logistics
 
NoSQL Data Migration Masterclass - Session 1 Migration Strategies and Challenges
NoSQL Data Migration Masterclass - Session 1 Migration Strategies and ChallengesNoSQL Data Migration Masterclass - Session 1 Migration Strategies and Challenges
NoSQL Data Migration Masterclass - Session 1 Migration Strategies and Challenges
 
ScyllaDB Virtual Workshop
ScyllaDB Virtual WorkshopScyllaDB Virtual Workshop
ScyllaDB Virtual Workshop
 
DBaaS in the Real World: Risks, Rewards & Tradeoffs
DBaaS in the Real World: Risks, Rewards & TradeoffsDBaaS in the Real World: Risks, Rewards & Tradeoffs
DBaaS in the Real World: Risks, Rewards & Tradeoffs
 
NoSQL Data Modeling 101
NoSQL Data Modeling 101NoSQL Data Modeling 101
NoSQL Data Modeling 101
 
Top NoSQL Data Modeling Mistakes
Top NoSQL Data Modeling MistakesTop NoSQL Data Modeling Mistakes
Top NoSQL Data Modeling Mistakes
 

Último

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Victor Rentea
 

Último (20)

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Vector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptxVector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptx
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 

Build Low-Latency Applications in Rust on ScyllaDB

  • 1. Piotr Sarna - Principal Software Engineer, ScyllaDB Felipe Cardeneti Mendes - Solutions Architect, ScyllaDB Andrew Brown - Director of Product Growth, ScyllaDB Virtual Developer Workshop Build Low-Latency Applications in Rust on ScyllaDB
  • 2. Presenters 2 Piotr Sarna Piotr is a software engineer very keen on open-source projects, C++ and Rust. Felipe Cardeneti Mendes Felipe Mendes is an IT Specialist with years of experience with Linux. In ScyllaDB, he works as a Solutions Architect Andrew Brown Andrew has been working with distributed systems in roles across product development and product marketing.
  • 3. Agenda ● Background: Why Rust, why ScyllaDB? ● The ScyllaDB Rust driver ● The Rust sample app walkthrough ● Profiling Rust performance
  • 4. IOT Rust Application Setup $ git clone https://github.com/fee-mendes/rust-driver-example/ $ cd rust-driver-example/docker-compose $ docker-compose up -d $ docker exec -it rust-app /bin/bash Minimum requirements: - Linux or OSX x86 - Docker installed and setup - Quadcore CPU - 2 GB of memory available - Windows is not supported Join the #rust-driver channel on ScyllaDB Slack to discuss any issues or questions you might have.
  • 5. Low latency, close to hardware schedulers Perfect horizontal & Vertical scale 5 1000 Nodes Cluster 2000 Cluster K8S Deployment 60TB per Node 256 Cores per Node 1B Operations per Second About ScyllaDB: Fast and Scalable
  • 6. 6 Poll: How proficient are you with the Rust language?
  • 7. + Asynchronous, non-blocking runtimes + Tokio: Most widely used Rust runtime + Seastar: C++ runtime for ScyllaDB + Fast, flexible, and reliable + Scalable, allows high concurrency and low latency + Green and sustainable Why Rust? Why Tokio? Why ScyllaDB? Rust Driver ScyllaDB
  • 8. The effort started during a 2020 internal hackathon at ScyllaDB. The existing CQL drivers (cassandra-cpp, cdrs) in the Rust ecosystem were not good enough for us: ● Not fully async ● Unsatisfactory performance ● Not scalable enough ● Known, long-standing bugs ● No support for ScyllaDB-specific optimizations ○ e.g., Shard-per-core ● etc. Some history
  • 9. Our design goals: ● fully asynchronous ● token-aware, shard-aware ● ease of use ● maximizing performance ● minimizing overheads ● extensibility ● an extremely cool logo Based on the principles above, we created the ScyllaDB Rust Driver. Some history
  • 10. ● Most popular async runtime, which should translate to best adoption ● It would be tempting to write the driver in a runtime-agnostic way, but it's hard: ○ Not all API's are well defined yet ○ Tokio offers quite complete support for TCP communication, timeouts and other useful abstractions ● Very actively developed Why Tokio?
  • 12. IOT Application Overview Metric Collector Metric Reader UUID Finder Write to ScyllaDB in parallel Deploy schema (ks/tables) Generate data: 100 devices 3 days Every 5 minutes Device metrics aggregator Analytics sample Split token-ring in small parts: Efficient full table scan token() function usage BYPASS CACHE Single device queries Real-time sample Partition scan: MAX(), AVG(), MIN() Range queries Date/Time handling
  • 13. Poll: What databases do you use (or are planning to use) for Rust Apps?: 13
  • 15. tokio-console is a very cool project which brings perf/top capabilities to async Rust. Prerequisites: Installation: $ cargo install tokio-console Cargo.toml: [dependencies] console-subscriber = "0.1" Code: console_subscriber::init(); Compilation flags: RUSTFLAGS="--cfg tokio_unstable" cargo run <your-project> Profiling: tokio-console
  • 17. Rust ecosystem has a very convenient way of generating flamegraphs: cargo flamegraph Usage: $ cargo install flamegraph $ cargo flamegraph your-project Profiling: cargo flamegraph
  • 19. On Linux, flamegraph scripts use perf underneath. Perf is a well-known and very powerful profiling tool with direct support in the kernel. Rust programs, since they're compiled via LLVM, are perfectly capable candidates for profiling with perf. Prerequisites: Cargo.toml (the root one in case you use multiple workspaces): [profile.release] lto = false debug = true System (Linux-specific): $ echo 0 | sudo tee /proc/sys/kernel/perf_event_paranoid $ echo 0 | sudo tee /proc/sys/kernel/kptr_restrict Profiling: perf
  • 20. By the way, when run on Linux, cargo flamegraph leaves a side effect - a perf.data file, which is a result format from calling perf record. Here's another example: $ perf record -g ./target/release/metric-collector $ perf report -g Profiling: perf
  • 22. United States 2445 Faber St, Suite #200 Palo Alto, CA USA 94303 Israel Maskit 4 Herzliya, Israel 4673304 www.scylladb.com @scylladb Thank You!