Assista a apresentação em: https://www.eventials.com/locaweb/cloud-storage-com-ceph Italo Santos é Administrador de Sistemas na Locaweb e entusiasta a programador. Locaweber há mais de 8 anos passando pelas equipes de suporte e operações, sempre focado na implementação de ferramentas de monitoramento e automatização de tarefas, atualmente focado na administração da infraestrutura de storages e sempre buscando novas tecnologias. Cloud storage vem se popularizando a cada dia no mercado e nessa apresentação pretende mostrar como é possível criar uma infraestrutura de mercado utilizando solução open source Ceph.

1. Cloud storage com Ceph

2. ● Italo Santos
● @ Locaweb desde 2007
● Sysadmin @ CT-Storage
# whoami

4. IBM 350
• 13 de Setembro, 1956
• 152cm de comprimento
172cm de altura
74cm de largura
• 50 discos de 60cm 1.2k RPM
• 3.75MB de espaço
• Alugado $3200 /mês

5. Era uma vez…

6. 1880's
Era uma vez…

7. 1880's
1950's
Era uma vez…

8. 1880's
1950's
Era uma vez…
1956
IBM 350

9. 1880's
1950's 1963
Era uma vez…
1956
IBM 350

10. 1880's
1950's 1963
1971
Era uma vez…
1956
IBM 350

11. 1880's
1950's 1963
1971
1982
Era uma vez…
1956
IBM 350

12. 1880's
1950's 1963
1971
1982
1994
Era uma vez…
1956
IBM 350

13. 1880's
1950's 1963
1971
1982
1994
2000's
Era uma vez…
1956
IBM 350

14. Cloud Storage

15. Cloud Storage

16. Cloud Storage

17. Cloud Storage

18. Cloud Storage

19. “Modelo de Serviço onde os dados são
armazenados e gerenciados remotamente
através de rede e acessados via API.”
Cloud Storage

20. API
Datacenter #2Datacenter #1
Pools
PUT / GET https://cloud.storage.com/bucket/file.txt
Cloud Storage

21. API
Datacenter #2Datacenter #1
Pools
PUT / GET https://cloud.storage.com/bucket/file.txt
Cloud Storage

22. Ceph
• Software Defined Storage
• Multi-Protocolo
• Escalável
• Auto Gerenciável
• Sem ponto único de falha

23. Arquitetura
RADOS
A reliable, autonomous, distributed object store comprised of self-healing, self-
managing, intelligent storage nodes
LIBRADOS
A library allowing
apps to directly
access RADOS,
with support for
C, C++, Java,
Python, Ruby,
and PHP
APP
RADOSGW
A bucket-based
REST gateway,
compatible with S3
and Swift
APP
RBD
A reliable and fully-
distributed block
device, with a Linux
kernel client and a
QEMU/KVM driver
HOST/VM
CEPH FS
A POSIX-compliant
distributed file
system, with a Linux
kernel client and
support for FUSE
CLIENT

24. Arquitetura
RADOS
A reliable, autonomous, distributed object store comprised of self-healing, self-
managing, intelligent storage nodes
LIBRADOS
A library allowing
apps to directly
access RADOS,
with support for
C, C++, Java,
Python, Ruby,
and PHP
APP
RADOSGW
A bucket-based
REST gateway,
compatible with S3
and Swift
APP
RBD
A reliable and fully-
distributed block
device, with a Linux
kernel client and a
QEMU/KVM driver
HOST/VM
CEPH FS
A POSIX-compliant
distributed file
system, with a Linux
kernel client and
support for FUSE
CLIENT

25. Arquitetura
RADOS
A reliable, autonomous, distributed object store comprised of self-healing, self-
managing, intelligent storage nodes
LIBRADOS
A library allowing
apps to directly
access RADOS,
with support for
C, C++, Java,
Python, Ruby,
and PHP
APP
RADOSGW
A bucket-based
REST gateway,
compatible with S3
and Swift
APP
RBD
A reliable and fully-
distributed block
device, with a Linux
kernel client and a
QEMU/KVM driver
HOST/VM
CEPH FS
A POSIX-compliant
distributed file
system, with a Linux
kernel client and
support for FUSE
CLIENT

26. Arquitetura
RADOS
A reliable, autonomous, distributed object store comprised of self-healing, self-
managing, intelligent storage nodes
LIBRADOS
A library allowing
apps to directly
access RADOS,
with support for
C, C++, Java,
Python, Ruby,
and PHP
APP
RADOSGW
A bucket-based
REST gateway,
compatible with S3
and Swift
APP
RBD
A reliable and fully-
distributed block
device, with a Linux
kernel client and a
QEMU/KVM driver
HOST/VM
CEPH FS
A POSIX-compliant
distributed file
system, with a Linux
kernel client and
support for FUSE
CLIENT

27. Arquitetura
RADOS
A reliable, autonomous, distributed object store comprised of self-healing, self-
managing, intelligent storage nodes
LIBRADOS
A library allowing
apps to directly
access RADOS,
with support for
C, C++, Java,
Python, Ruby,
and PHP
APP
RADOSGW
A bucket-based
REST gateway,
compatible with S3
and Swift
APP
RBD
A reliable and fully-
distributed block
device, with a Linux
kernel client and a
QEMU/KVM driver
HOST/VM
CEPH FS
A POSIX-compliant
distributed file
system, with a Linux
kernel client and
support for FUSE
CLIENT

28. Arquitetura
RADOS
A reliable, autonomous, distributed object store comprised of self-healing, self-
managing, intelligent storage nodes
LIBRADOS
A library allowing
apps to directly
access RADOS,
with support for
C, C++, Java,
Python, Ruby,
and PHP
APP
RADOSGW
A bucket-based
REST gateway,
compatible with S3
and Swift
APP
RBD
A reliable and fully-
distributed block
device, with a Linux
kernel client and a
QEMU/KVM driver
HOST/VM
CEPH FS
A POSIX-compliant
distributed file
system, with a Linux
kernel client and
support for FUSE
CLIENT

29. Componentes
OSDs
M
Monitors MDS

30. Componentes
OSDs
M
Monitors MDS

31. Componentes
OSDs
M
Monitors MDS

32. Componentes
OSDs
M
Monitors MDS

33. RADOS
A reliable, autonomous, distributed object store comprised of self-healing, self-managing, intelligent storage nodes
LIBRADOS
A library allowing
apps to directly
access RADOS,
with support for
C, C++, Java,
Python, Ruby,
and PHP
RBD
A reliable and fully-
distributed block
device, with a Linux
kernel client and a
QEMU/KVM driver
CEPH FS
A POSIX-compliant
distributed file
system, with a Linux
kernel client and
support for FUSE
APP HOST/VM CLIENT
Object Storage Gateway
RADOSGW
A bucket-based
REST gateway,
compatible with S3
and Swift
APP

34. RadosGW
• Interface Object Storage REST
• Amazon S3
• OpenStack Swift
• FastCGI ou Embedded
• Utiliza cluster Ceph

35. Arquitetura
LIBRADOS
RADOSGW
APP
socket (Autenticação Ceph key)
REST (S3 / Swift)

36. Funcionalidades
• Autenticação
• Buckets (ou containers)
• Gerenciamento de quotas
• Relatórios de utilização
• Replicação (zona e/ou região)
• Regras por usuário
• API de administração

37. Autenticação

38. Buckets
Usuário
Global

39. Quotas - usuário

40. Quotas - bucket

41. Relatórios
• Usuário
• Bucket
• Período
• Tipo de operação
• Buckets
• Sumário

42. Replicação
• Disponibilidade / Failover
• Proteção / Backup
• Multi-Site / Zona
• Multi-Região

43. Multi-Site / Zona
APP
LIBRADOS
RADOSGW
Master Zone
APP
LIBRADOS
RADOSGW
Failover Zone
Read OnlyRead / Write
Data Sync
Master Region

44. Multi-Região
APP
LIBRADOS
RADOSGW
Master Zone
APP
LIBRADOS
RADOSGW
Master Zone
Read / Write
Metadata Sync
Read / Write
Master Region Secundary Region

45. Regras

46. API de Administração
• Começa em “/admin"
• Autenticação via S3
• Permissões pré-definidas
• Suporte a JSON e XML

47. API de Administração

48. API de Administração

49. Compatibilidade
Amazon S3 Ceph
Buckets
Bucket Lifecycle
Policy
Bucket Website
ACLs
Location
Notification
Versioning
Request Payment
Multipart Uploads
Replication

50. Website

51. Website

52. https://www.flickr.com/photos/maayc/6786074583/

53. Obrigado!
Italo Santos @ CT-Storage