An introduction to Zenko at the Cloud Native Foundation in London.

1. Cloud Native London
March 6th 2017
Zenko:
MultiCloud Data
Controller
Laure
Vergeron
Technology Evangelist
Software Engineer
@zenko
@LaureVergeron

2. Who is Scality?

3. ▪The RING: unified file and object storage at
Petabyte Scale
▪ File storage and object storage are both still relevant today -
they simply fit different usecases
▪ Scality’s original software, the RING, scales seamlessly at
petabyte scale
▪Software-Defined: freedom of choice
▪ Most storage solutions today are black box, meaning you
don’t get to change your hardware without buying new
software, and vice versa
▪ Scality RING doesn’t lock you in: you may choose any x86
hardware you like, and change it
▪Gartner Leader: 2 years in a row
▪ Gartner analysts ranked us leader 2 years in a row! (and that
quadrant is only 2 years old ;) )
▪ And here’s where it becomes interesting to you: our open
source stack played a big role in this placement
Scality is a Software-Defined Storage Leader

4. Scality is also a bold, disruptive, and responsible StartUp

5. ▪The RING is sailing
▪ Our customer base is growing steadily
▪ Our customers are happy about the product
▪Our teams are struggling, growth is hard
▪ Most of our engineering is lacking vision, which is new
▪ Processes, while necessary, have lowered the morale
▪ QA is complaining that the code is less and less good
▪How do we foster innovation in our startup? How
do we reignite our talents?
▪ It’s not a skills problem: our engineers are bright
▪ It’s not a technology problem: analysts see us as innovative
and promising
▪ It’s not a market problem: the demand for storage is ever
increasing
Back in time: 2015

6. ▪The RING is sailing
▪ Our customer base is growing steadily
▪ Our customers are happy about the product
▪Our teams are struggling, growth is hard
▪ Most of our engineering is lacking vision, which is new
▪ Processes, while necessary, have lowered the morale
▪ QA is complaining that the code is less and less good
▪How do we foster innovation in our startup? How
do we reignite our talents?
▪ It’s not a skills problem: our engineers are bright
▪ It’s not a technology problem: analysts see us as innovative
and promising
▪ It’s not a market problem: the demand for storage is ever
increasing
Back in time: 2015
Let’s create a startup inside the
startup. And let’s use that inner
startup to revamp our S3 API to the
RING, as it has become the de facto
standard for object storage. Let’s also
make it multi cloud.
And let’s open source the inner
startup work.
Jérôme Lecat, CEO
Giorgio Regni, CTO

7. ▪We achieved the most thorough open source S3 standalone API : Cloudserver
▪ We had it out in 8 months, which was permitted by technologies (node.js vs C/C++ historically)
▪ We were named Gartner leaders from the moment they had a leader quadrant in our field
▪ We hit 1.6M Docker pulls a month ago, and we’re not stopping
▪We had our growth pains too
▪ We do open source fairly: we use the exact same code in our enterprise stack, so we had to converge back with the
rest of the company to go from project to product… innovative ideas and processes had a delayed cost.
▪ We thought we could have a global community, but we really have local communities that share a common goal
(enabling people who create value with data), and work mostly independently
▪We would do it again and again!
▪ The win outweighs the costs by a lot
▪ This open source project has paved the way for a lot of other innovations, both open source or enterprise
▪ We have reenergized our company with this new team’s mindset
2017: 2 years of open source in a nutshell

8. Zenko:
A MultiCloud Data
Controller

9. Who is multi cloud?
?

10. Who is multi cloud?
(and that’s only storage)

11. What’s the big deal?

12. What’s the big deal?

13. What’s the bigger deal?

14. ▪Content Distribution
▪ Media companies have tens of thousands of movies, which they store on Private Cloud for
control. When it is time to publish a movie, it makes sense to copy it to a public cloud to use
its transcoding and CDN services.
▪Compute Bursting
▪ Banks have to do risk analysis leveraging thousands of CPU every night. These intense
computation only run for a few hours. Rather than having idle servers for the rest of the day, it
makes sense to use Public Cloud services for the computation
▪Analytics
▪ E-commerce company do more and more machine learning on their very large data lake.
Rather than setting up Hadoop infrastructures in-house, the company can copy just a data set
to an Hadoop cloud, compute the appropriate algorithm, and get back the result and destroy
the cloud copy of the data to save on storage cost.
▪Long-term Archival / cold storage
▪ While storing data which is regularly accessed is cheaper in a private cloud, long term archive
of never accessed data is cheaper in long term archive cloud offering. Automatic archival of
never accessed data would save a lot of money.
Examples of Use-Cases for Multi-Cloud

15. Zenko Open Source: Features & Capabilities
METADATA
DATA STORAGE
DMD REST/Sproxyd AWS S3 API AZURE BLOB API
Shared Local
Storage
S3 API
APP
METADATA
APP
S3 CALLS
Zenko Open Source
S3 API—Single API set and 360° access to any cloud
Native format—Data written through Zenko is stored
in the native format of the target cloud storage and
can be read directly, without going through Zenko.
Project Backbeat for data workflow—Policy-based
data management engine
Project Clueso for metadata search— Apache
Spark-based metadata search tool for optimal data
insight
HA/Failover – Deployed as dual-containers managed
by Docker Swarm for HA, but not full scale-out
Simple Security –single-tenant credentials managed
locally
S3 API
S3 CALLS
METADATA DATA
CLUESO
Metadata Search
Bucket LOCATION
BACKBEAT
Data Policy Engine
Bucket LOCATION
CRR/DATADATA
Google CS API

16. Current Release API Support
Core S3 APIs
• Bucket and object operations (PUT,
GET, DELETE, HEAD)
• Multi-Part Upload (MPU) for efficient
ingest of large objects
Advanced S3 APIs
• Bucket Website
• Bucket CORS
• Bucket Versioning
• Bucket Cross Region Replication (CRR)
Extended APIs
• Utilization API for metering of capacity,
#objects, bandwidth & ops
2018 Roadmap
Bucket Lifecycle
• Expiration policies - Q1 2018
Extended APIs
• Metadata search through extended
GET Bucket API – Q1 2018
• CRR one-to-many – Q1 2018
16

17. - Microsoft Azure
- AWS S3
- AWS S3 compatible
• Wasabi
• Digital Ocean
• yours?
- Backblaze B2
- Google Cloud
17

18. Ease of use
- No more manual edition of JSON files
- Get a sandbox in 1 click
- No more service restart for configuration updates
- Download your Cyberduck profile to get started
Aggregated stats over all of your storage
- At user, bucket, account, and service level, as permitted by UTAPI
Monitoring made simple
- Straightforward graphs
- Healthcheck indicators at a glance
- S3 browser: bucket listing across clouds
- Manage IAM access and secret key in a few clicks
- Manage your locations and replication workflows with Orbit’s wizards
- See your changes applied in real time
18
Zenko EE and Open Source: Zenko Orbit
one UI to rule them* all *(them = storage vendors)

19. Zenko EE: Enterprise Security, File & Scale-Out
METADATA: HA/Consistency Cluster
DATA STORAGE
DMD REST/Sproxyd AWS S3 API AZURE BLOB API
Shared Local
Storage
S3 API
APP APP
DATA
CLUESO
Metadata Search
S3 CALLS
Zenko Enterprise Edition
Multi-tenancy & Enterprise Security – Full IAM support
of Multi Accounts, Users, Groups, Policies & Single-Sign
On (SSO) to AD & LDAP security servers
Scale-Out – N-Way scale-out to any number of servers to
deliver capacity AND performance for massive workloads,
leverages the Metadata engine cluster from S3 Connector
File & S3 Shared Access – bi-directional file & object
sharing with NFS v4/v3 & SMB for legacy apps
Enables full Scale-Out for all key Zenko Services:
• Native Cloud Storage — Support for multiple public
clouds and Scality RING in native data format
• Backbeat for data workflow—Policy-based data
management engine
• Clueso for metadata search— Apache Spark-based
metadata search tool for optimal data insight
S3 API S3 API← Scale Out →
S3 CALLS
← Scale Out →
NFS/SMB
Google CS API
← Enterprise Apps →
Legacy
App
NFS/ / SMB
Identity & Access Management (IAM): SAML 2.0/SSO with AD/LDAP
BACKBEAT
Data Policy Engine
METADATA DATACRR/DAT
A
METADATA
LOCATION LOCATION LOCATION LOCATION

20. • Integrated NFS v4/v3 stack, namespace & data
compatible with S3
• S3 Buckets are exported as Volumes, which can be
mounted over NFS
• Bi-directional reads & writes of files ←→ objects
• Core NFS functionality - limitations on POSIX
functionality (for example, no hard links & renames,
no NFS v4 ACLs, no S3 metadata access)
Zenko EE - File Services
20

21. Enterprise Security Model
• Based on our S3 IAM
• Multi-tenancy is based on the AWS IAM (Identity & Access
Management)
• Multiple Accounts, Users, Groups and IAM User/Group Policies
• Access control to clouds restricted through User and Group IAM Policies
• Authentication to Zenko through AWS Signature v4
• App uses standard S3 access keys
• Zenko manages target Cloud account locations and credentials
• Note: not all S3 IAM features can be mapped to other clouds
Multi-Tenancy
• Logical separation of Accounts through Auth/Access control
• Cannot authenticate requests into Account without keys
Zenko EE: Enterprise Security Model
21

22. ▪Developers / QA departments
▪ Free, 100% AWS S3 compatible testing running locally, on your machine
▪ Using CloudServer as your API to AWS S3 gives you buffer if AWS S3 changes
▪Small enterprise / startups
▪ Works perfectly until 500 TB
▪ Only need a server / an external hard drive
▪ Every company has someone who knows S3: no learning curve to start having proper storage
▪ISVs / Partners / Prospects
▪Applications
▪ Give your app the freedom to run against multiple data backends
▪ Give yourself the opportunity to change vendor without updating code
Zenko Open Source: Who uses it?

23. Community Meetups
• Initiated prior to our CloudServer launch
• Participating at open source events
for Docker, Nodejs, etc...
Developper “Hackathons”
• Paris and San Francisco in 2015 & 2016 & 2017
• Co-sponsoring with partners –
focused on a specific project goal (e.g., IP Drives, S3 API, Encryption)
• Great for building visibility & community participation
One example: The Paris 42 School hackathon
• 42 Paris (free coding university); bringing in Backblaze B2 support
• Nov 6-10 in Paris, France: https://www.zenko.io/hackathon-42-paris/
23
Building a Developer community

24. How can I get involved with Zenko?
• Let us know what you do with Zenko stack!
▪ zenko@scality.com
▪ Get your project/company featured on the website in a quote
• Contribute tutorials
▪ Get a blogpost featuring your introduction of your tutorial
▪ Become part of our readTheDocs hosted documentation
• Contribute code
▪ It’s an opportunity to drive the roadmap with us !
▪ Join the team and be part of the Zenko craze !
▪ We have Contributing Guidelines on the GitHub repos, and we’ll answer your
questions via GitHub issues or our forum forum.scality.com
• Meet us at AWS Re:invent, Cloud Native London Meetup, DockerCon...
▪ All info is on www.zenko.io
24

25. ▪Zenko: MultiCloud Data Controller
▪ http://zenko.readthedocs.io/en/latest
▪ https://github.com/scality/Zenko
▪ https://forum.scality.com/
▪ https://www.zenko.io/
▪CloudServer (S3 API standalone)
▪ http://s3-server.readthedocs.io/en/latest/
▪ https://github.com/scality/S3/
▪ https://hub.docker.com/r/scality/s3server/
▪Backbeat (data driven event manager)
▪ https://github.com/scality/Backbeat
▪Clueso (cross-cloud metadata search)
▪ https://github.com/scality/Clueso
Zenko: everything you need
@zenko
@scality
@LaureVergeron
@GiorgioRegni

26. Demo, 1:
Getting a Zenko
sandbox and managing
it on Orbit

27. Demo, 2:
Deploying Zenko
yourself (and managing
it with Zenko Orbit)

28. Q&A

29. Forum: forum.scality.com
Email: zenko@scality.com
Thank You