StorPool is a leading software provider that develops the most reliable and speedy storage platform on the market. The software comes as an utterly hands-off solution – StorPool’s team architects, deploys, tunes, monitors, and maintains the system. The software has a native plug-in for CloudStack, and it is the ideal solution for companies that aim to optimize and grow their business, eliminate data storage issues, or secure fast and reliable access to data to achieve their business goals. In this talk, you’ll find out more about StorPool’s functionality, architecture, and use cases. You’ll also learn about the new features of the product and its deeper integration into CloudStack.
Slavka Peleva is a software developer at StorPool Storage and has been a passionate CloudStack Commiter since 2021. She is part of the CloudStack community and is excited to work on new features and capabilities to improve the open-source cloud management solution. At StorPool Slavka is responsible for the product integration with Cloudstack and is working on its native implementation in the technology. With her dedication and efforts in creating the StorPool’s plugin for CloudStack, companies can build reliable and speedy storage that is powered by the two technologies.
Venko Moyankov is the Solutions Lead at StorPool Storage, a fully managed data storage solution that transforms commodity hardware into a shared, powerful, highly available, and reliable storage system. He has experience with managing large virtualizations, working in telcos, designing and supporting the infrastructure of large enterprises. In the last year his focus has been in helping companies globally to build the best storage solution according to their needs and projects.
Alex Ivanov is the Product Lead at StorPool Storage and is responsible for the product strategy and tactics at the company. He has experience in the storage market and a deep understanding of the needs and priorities of businesses managing large-scale clouds to run diverse, mission-critical workloads. At StorPool, he works closely with the team to evolve the StorPool primary storage platform and expand its capabilities and usefulness for various use cases. His goal - ensure StorPool cost-effectively delivers above and beyond what is possible with other primary storage products to help customers achieve their commercial and technical goals.
-----------------------------------------
CloudStack Collaboration Conference 2022 took place on 14th-16th November in Sofia, Bulgaria and virtually. The day saw a hybrid get-together of the global CloudStack community hosting 370 attendees. The event hosted 43 sessions from leading CloudStack experts, users and skilful engineers from the open-source world, which included: technical talks, user stories, new features and integrations presentations and more.
2. 2
Alex Ivanov
Product Lead
StorPool Company Intro
Products and Services
Use Cases and Case
Studies
Venko Moyankov
Solutions Lead
StorPool Storage Intro
CloudStack Integration
Overview
Key Technical Benefits
Slavka Peleva
CloudStack Tech Lead
Quick Intro to the Speakers
StorPool Integration with
Apache CloudStack
Plug-in Features and
Benefits
3. 3
of Flash Storage Deployments
Primary Workload Experts
Years in Business
Proven Track Record
Countries of Deployment
Global Software Company
Team Members
Stable and Profitable Partner
Accelerating Human Activity with Faster Storage
100s
11
30+
50+
4. 4
Software Defined Storage (SDS) Vendor of the Year at 2020
Storage Awards
“The heated battle for the SDS vendor of the year award between the tech
giant IBM and the storage market disruptor, StorPool, has ended with a
well-deserved victory for the innovative scale-up company.
IBM took the place of runner-up for the category and this showed us that
modern business has become more accepting in the adoption of next-gen
technologies.”
Source:
https://containerjournal.com/news/news-releases/storpool-named-software-defined-storage-sds-vendor-of-the-year-at-2020-storage-awards/
6. 6
StorPool Storage Products and Services
Enablers of our Complete Storage Solution for CloudStack Environments
StorPool Storage
Converts customers’ servers into high-performance,
linearly scalable primary storage systems.
StorPool Analytics
Collects millions of metrics daily for visualizations and
deep insights into each StorPool storage system.
StorPool Alerts & Reporting
Uses StorPool Analytics’ metrics for per-node, per-host,
per-volume, and whole-system alerts and reports.
StorPool VolumeCare
Periodic snapshotting and replication of incremental
changes to local or remote lower storage tiers.
StorPool Managed Service
Our experts leverage the other StorPool products to
proactively open tickets and deal with potential issues.
7. 7
We analyse and tune your
StorPool storage system so
that it runs optimally and
reliably for your end-users’
workloads
Design Deploy
Fine-tune Monitor Maintain
StorPool Helps Get Your Data In Order
We help you select the ideal
architecture for your cloud at
the physical, network, and
logical levels, using only
standard hardware.
We install StorPool Storage
in your servers and connect
your StorPool storage
system to one or your
CloudStack cluster.
We monitor thousands of
metrics per second to
proactively open support
tickets and deal with any
issues that arise
We ensure that your storage
system always runs optimally
by installing non-disruptive
updates and helping with
expansion when needed.
Deploy Reliable, Agile, Speedy, and Cost-effective Clouds
8. 8
Typical Customers and Other Use Cases
Typical Customers:
● Follow Modern Cloud Infrastructure
Practices
API-driven, Automated, DevOps, SREs,
Cloud Management Platforms
● Offer Public Cloud Services
IaaS, PaaS, hosted SaaS, virtual private
servers, ecommerce, shared hosting,
desktop as a service, BC/DR
● Offer Private Cloud Services
Tailored solutions for SMBs and Enterprises -
virtual private clouds, managed/hosted
private clouds
Primary Use Cases:
● Media: SATA/SAS/NVMe SSD-based
storage served to workloads in bare-metal
hosts, virtual machines, and containers
● Workloads: Databases, VM disks, VDI, large
apps or SaaS, video game hosting,
persistent K8s volumes, etc.
● Scale: 100TB to 10s of PB stored across
multiple data centers
● Example Projects: Convert Legacy IT Stack
to SDDC Design; Consolidate Мultiple IT
systems to One Stack
9. 9
Case Study - Amito
The Project:
● "Тhe most difficult part of the infrastructure stack for high
availability virtualized servers is storage. Amito was using
quite a few different solutions, from big-name vendors to in
house developed HA SANs based on ZFS. None of these
solutions brought the expected results."
The goals:
● Implement a reliable, high-performance shared storage in
multiple data centers - support a broad range of apps.
The solution:
● “... our shortlist didn’t contain any of the vendors with older
legacy solutions – the approach to storage has moved on,
and they have not all kept up.”, Ed Butler, CEO, Amito.
● KVM, CloudStack, StorPool, supporting a large number of
Windows and Linux virtual machines.
Company Profile:
● UK-based Cloud and Managed
Service Provider
● Founded in 2000
● Presence in 28 data centers
worldwide
Case study:
https://storpool.com/amito-cas
e-study-storpool
10. 10
Use Cases / Customers / Workloads Summary
Top Use Cases
Selected Customers
Workloads of
Virtual Disks for VMs,
Databases, VDI, K8s
Large SaaS / Web Apps
Test/Dev and Production
Upgrade legacy IT
systems to SDDC design
12. 12
StorPool Storage Technical Overview
Software deployed on customer’s servers running
standard Linux - CentOS, RHEL, Ubuntu, Debian.
Converts off-the-shelf x86-64 servers into a
high-performance primary storage system
Distributed, shared-nothing architecture linearly
scales to petabytes without performance
bottlenecks.
Decoupled from compute or hyper-converged
Thin-provisioned volumes for virtual machines,
containers, and bare-metal hosts.
Fastest, Most Reliable Primary Storage Software on the Market
15. 15
● Thin provisioning
● Efficient snapshots
● Clones
● Per volume QoS
● Storage tiering
● One volume per virtual disk
● RESTful JSON API
Data Management
16. 16
Performance in Large-scale Environments
StorPool is so efficient that
84% of end-to-end latency is
caused by the laws of physics
Measured with All-NVMe cluster using Micron 9300 Pro, 25Gbit/s Ethernet. (lower is better)
Test using FIO on a raw block device, random read, 4k block size, QD=1
19. 19
The Fastest Cloud is Powered by StorPool
https://storpool.com/wp-content/uploads/2022/01/StorPool-Public-Cloud-Performance-Measurement-Report_2022.pdf
Unfit for Databases
PostgreSQL Benchmark
20. 20
The Fastest Cloud is Powered by StorPool
https://storpool.com/wp-content/uploads/2022/01/StorPool-Public-Cloud-Performance-Measurement-Report_2022.pdf
PostgreSQL Benchmark
2-3x lower latency at
fixed load
21. 21
The Fastest Cloud is Powered by StorPool
https://storpool.com/wp-content/uploads/2022/01/StorPool-Public-Cloud-Performance-Measurement-Report_2022.pdf
2.6-4x transactions
with same CPU, RAM
PostgreSQL Benchmark
24. 24
CloudStack versions
● From CloudStack v4.11.3 to the main
branch of CloudStack
● Included in the CloudStack v4.17.0
packages
Separate jar
● On all versions
on the managements’ host
/usr/share/cloudstack-management/lib
on the agents’ hosts
/usr/share/cloudstack-agent/lib
Prerequisites and requirements
25. 25
What’s the Impact for Service Providers?
Direct Technical Benefits of the StorPool Plug-in for CloudStack
Offloading Secondary Storage
Creating snapshots and templates based on
items in the CloudStack marketplace is done
directly in StorPool, accelerating all operations.
Offloading Hypervisor Network
Since snapshots and templates are created in
StorPool, the private and public compute node
networks are used only for user traffic.
Storage Usage Efficiency
StorPool creates a volume for each virtual disk
by cloning pre-existing snapshots. VM data is
kept only on the primary storage and provisioned
stored ratios are optimised – 2:1 to 6:1.
26. 26
StorPool plug-in features
Simplified Cloud Architecture
Both primary and secondary storage can be
provided with a single StorPool storage system
Reduce the usage of Secondary storage
● No need to download VM templates for VM
deployments
● No snapshot transfers
● No volume migration
27. 27
The ideal foundation for large-scale clouds running diverse,
mission-critical workloads
Thanks
Slavka, Venko, Alex
30. 30
Case Study #2 - Katapult
The Project:
● Virtual IaaS platform built for extreme performance and
simplicity
● Build the fastest public cloud using standard hardware
The goals:
● Modern linearly scalable storage solution
● Unmatched performance
● High availability
● A high level of flexibility
● Guaranteed data integrity
The solution:
● Tailored IaaS platform, focused on extreme performance
● Automation and API-first approach
● Extremely high level of data protection offered by
StorPool’s end-to-end data integrity and 3x copies of data
Company Profile:
● Katapult by Krystal Hosting
● One of the largest independent
UK web hosting companies
● Founded in 2002 and steadily
grown over the last 20 years
● Comprehensive portfolio of
hosting, cloud and VPS
services
Case study:
https://storpool.com/krystal-kat
apult-case-study
31. 31
Comparing Cloud Performance and Cost
How to Make Sure you are Comparing Apples to Apples
Use Equivalent Technical
Setup
Work off the Same Cost
Basis
Run Tests Similar to
Production Workloads
Example from our tests:
Storage – 1,024 GiB
Memory – 16GB DDR4
vCPUs – 8 vCPUs
If VMs have more memory,
pre-allocate excess memory
as a file to even the playing
field.
Experiment with vCPUs and
memory to ensure they are
not the bottleneck.
Make sure to account for the
total monthly cost of the VM.
Example from our tests:
Katapult – $274 / m
AWS with io2 – $1,348 / m
AWS with gp3 – $402 / m
Azure – $358 / m
GCP – $299 / m
Remember to consider
ingress/egress charges as they
lead to cloud lock-in
Pgbench – PostgreSQL
benchmark, closest to
application performance
Sysbench – MySQL
benchmark, control for
Pgbench results
fio – synthetic benchmarks that
test the limits of the VM
(latency, random read/write,
sequential read/write)
rsync – simulates rapid
deployment and backups
32. 32
Cost/Performance Comparison $ / TPS / Month
Small OLTP Database Large OLTP Database
Sub-2ms Sub-3ms Sub-2ms Sub-3ms Sub-4ms
Katapult $0.028 / TPS $0.024 / TPS $0.031 / TPS $0.029 / TPS $0.028 / TPS
GCP SSD
Persistent
$0.687 / TPS $0.470 / TPS $0.620 / TPS $0.404 / TPS $0.413 / TPS
Amazon EBS
io2
Over
Threshold
$0.250 / TPS $0.880 / TPS $0.457 / TPS $0.314 / TPS
Amazon EBS
gp3
Over
Threshold
$0.151 / TPS
Over
Threshold
$0.390 / TPS $0.083 / TPS
Azure Premium
SSD
Over
Threshold
Over
Threshold
Over
Threshold
Over
Threshold
Over
Threshold
Katapult Delivers up to 20 Times Better Cost per TPS
Running OLTP Databases in Katapult vs. Hyperscale Public Clouds