TLUG Meeting 2008/09/13 - Gosuke Miyashita Discusses Scalable Storage Solutions

•Transferir como PPT, PDF•

9 gostaram•2,066 visualizações

The document discusses Gosuke Miyashita's goal of building a scalable storage system for his company's web hosting service. He is exploring the use of several open source technologies including cman, CLVM, GFS2, GNBD, DRBD, and DM-MP to create a storage system that provides high availability, flexible I/O distribution, and easy extensibility without expensive hardware. He outlines how each technology works and shows some example configurations, but notes that integrating many components may introduce issues around complexity, overhead, performance, stability and compatibility with non-Red Hat Linux.

Tecnologia Negócios

TLUG Meeting 2008/09/13 Gosuke Miyashita

My company ,[object Object],[object Object],[object Object],[object Object]

My goal of a scalable storage system ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Technologies

cman ,[object Object],[object Object],[object Object],[object Object],[object Object]

CLVM ,[object Object],[object Object],[object Object],[object Object]

CLVM clvmd distributes metadata among cluster nodes Logical volumes presented to each cluster node Logical volume on shared storage LVM2 Metadata clvmd LVM2 Metadata clvmd LVM2 Metadata clvmd

GNBD ,[object Object],[object Object],[object Object],[object Object]

GNBD TCP/IP network GNBD client GNBD client GNBD client GNBD Server Exported block device

GFS2 ,[object Object],[object Object],[object Object],[object Object],[object Object]

GFS2 These nodes can access to the GFS2 file system simultaneously GNBD Server GFS2 GNBD client cman GNBD client cman GNBD client cman

DRBD ,[object Object],[object Object],[object Object],[object Object]

DRBD Replication Server Block Device Server Block Device

DM-MP ,[object Object],[object Object],[object Object]

DM-MP with SAN storage /dev/sda1 /dev/sdb1 Seen as one device /dev/mapper/mpath0 active/passive or active/active Node HBA1 HBA2 SAN swtich 1 SAN swtich 2 Storage CNTRLR1 CNTRLR2

A scalable storage system /dev/VG0/LV0 (CLVM) mount /dev/VG0/LV0 /mnt cman GNBD cman GNBD GNBD Server GFS2 GNBD Server GFS2 Replication (DRBD) /dev/mapper/mpath0 (DM-MP) /dev/gnbd0 /dev/gnbd1 GNBD Server GFS2 GNBD Server GFS2 Replication (DRBD) /dev/mapper/mpath1 (DM-MP) /dev/gnbd2 /dev/gnbd3

How to extend /dev/VG0/LV0 (CLVM) mount /dev/VG0/LV0 /mnt cman GNBD cman GNBD GNBD Server GFS2 GNBD Server GFS2 /dev/mapper/mpath0 /dev/gnbd0 /dev/gnbd1 GNBD Server GFS2 GNBD Server GFS2 /dev/mapper/mpath1 /dev/gnbd2 /dev/gnbd3 GNBD Server GFS2 GNBD Server GFS2 /dev/mapper/mpath2 /dev/gnbd4 /dev/gnbd5

I wonder ... ,[object Object],[object Object],[object Object],[object Object],[object Object]

Mais conteúdo relacionado

Mais procurados

Ceph on 64-bit ARM with X-GeneCeph Community

MySQL Head-to-HeadPatrick McGarry

Ceph on Intel: Intel Storage Components, Benchmarks, and ContributionsColleen Corrice

Stabilizing CephCeph Community

Ceph: Open Source Storage Software Optimizations on Intel® Architecture for C...Odinot Stanislas

Quick-and-Easy Deployment of a Ceph Storage Cluster with SLESJan Kalcic

Cephfs jewel mds performance benchmarkXiaoxi Chen

Ceph Day Melbourne - Ceph on All-Flash Storage - Breaking Performance BarriersCeph Community

HKG15-401: Ceph and Software Defined Storage on ARM serversLinaro

QCT Ceph Solution - Design Consideration and Reference ArchitecturePatrick McGarry

Storage tiering and erasure coding in Ceph (SCaLE13x)Sage Weil

Ceph Performance Profiling and ReportingCeph Community

Accelerating Ceph with RDMA and NVMe-oFinside-BigData.com

Build a High Available NFS Cluster Based on CephFS - Shangzhong ZhuCeph Community

New Ceph capabilities and Reference ArchitecturesKamesh Pemmaraju

Ceph on All Flash Storage -- Breaking Performance BarriersCeph Community

Ceph Day Beijing - Ceph All-Flash Array Design Based on NUMA ArchitectureDanielle Womboldt

Red Hat Ceph Storage Acceleration Utilizing Flash Technology Red_Hat_Storage

Journey to Stability: Petabyte Ceph Cluster in OpenStack CloudPatrick McGarry

Red Hat® Ceph Storage and Network Solutions for Software Defined InfrastructureIntel® Software

Mais procurados (20)

Ceph on 64-bit ARM with X-Gene

MySQL Head-to-Head

Ceph on Intel: Intel Storage Components, Benchmarks, and Contributions

Stabilizing Ceph

Ceph: Open Source Storage Software Optimizations on Intel® Architecture for C...

Quick-and-Easy Deployment of a Ceph Storage Cluster with SLES

Cephfs jewel mds performance benchmark

Ceph Day Melbourne - Ceph on All-Flash Storage - Breaking Performance Barriers

HKG15-401: Ceph and Software Defined Storage on ARM servers

QCT Ceph Solution - Design Consideration and Reference Architecture

Storage tiering and erasure coding in Ceph (SCaLE13x)

Ceph Performance Profiling and Reporting

Accelerating Ceph with RDMA and NVMe-oF

Build a High Available NFS Cluster Based on CephFS - Shangzhong Zhu

New Ceph capabilities and Reference Architectures

Ceph on All Flash Storage -- Breaking Performance Barriers

Ceph Day Beijing - Ceph All-Flash Array Design Based on NUMA Architecture

Red Hat Ceph Storage Acceleration Utilizing Flash Technology

Journey to Stability: Petabyte Ceph Cluster in OpenStack Cloud

Red Hat® Ceph Storage and Network Solutions for Software Defined Infrastructure

Destaque

S de0882 new-generation-tiering-edge2015-v3Tony Pearson

Tony blogging-tips-itso30-v1310eTony Pearson

IBM Solid State in eX5 serversTony Pearson

Delitos cibernéticosYaquelina Bermejo

Planetasaaronlongueira

SAP HANA Runs Better, Faster, Stronger on IBM PowerDynamix

Infographic OpenStack - Deployment ToolsStratoscale

Complete dd ex5s1170131

IBM's Pure and Flexible Integrated SolutionTony Pearson

The Pendulum Swings Back: Converged and Hyperconverged EnvironmentsTony Pearson

S cv3179 spectrum-integration-openstack-edge2015-v5Tony Pearson

Sg248107 Implementing the IBM Storwize V3700Wellington Torrejais da Silva

S ss0885 spectrum-scale-elastic-edge2015-v5Tony Pearson

Choosing the Right Storage for your Server Virtualization EnvironmentTony Pearson

IBM Spectrum Scale Overview november 2015Doug O'Flaherty

Clever safe Azad Mukhurov

Backup Options for IBM PureData for Analytics powered by NetezzaTony Pearson

IBM's Cloud Storage OptionsTony Pearson

Has Your Data Gone Rogue?Tony Pearson

Cleversafe.PPTXPatrick Berghaeger

Destaque (20)

S de0882 new-generation-tiering-edge2015-v3

Tony blogging-tips-itso30-v1310e

IBM Solid State in eX5 servers

Delitos cibernéticos

Planetas

SAP HANA Runs Better, Faster, Stronger on IBM Power

Infographic OpenStack - Deployment Tools

Complete dd ex5

IBM's Pure and Flexible Integrated Solution

The Pendulum Swings Back: Converged and Hyperconverged Environments

S cv3179 spectrum-integration-openstack-edge2015-v5

Sg248107 Implementing the IBM Storwize V3700

S ss0885 spectrum-scale-elastic-edge2015-v5

Choosing the Right Storage for your Server Virtualization Environment

IBM Spectrum Scale Overview november 2015

Clever safe

Backup Options for IBM PureData for Analytics powered by Netezza

IBM's Cloud Storage Options

Has Your Data Gone Rogue?

Cleversafe.PPTX

Semelhante a TLUG Meeting 2008/09/13 - Gosuke Miyashita Discusses Scalable Storage Solutions

High Availability Storage (susecon2016)Roger Zhou 周志强

Red Hat Global File System (GFS)Schubert Zhang

SUSE Expert Days Paris 2018 - SUSE HA Cluster Multi-DeviceSUSE

Linux containers – next gen virtualization for cloud (atl summit) ar4 3 - copyBoden Russell

OpenNebulaConf 2016 - The DRBD SDS for OpenNebula by Philipp Reisner, LINBITOpenNebula Project

Distributed replicated block deviceChanaka Lasantha

Disaster recovery of OpenStack Cinder using DRBDViswesuwara Nathan

Linux High Availability Overview - openSUSE.Asia Summit 2015 Roger Zhou 周志强

GlusterFS and Openstack StorageDeepak Shetty

Rhel cluster basics 1Manoj Singh

Realizing Linux Containers (LXC)Boden Russell

HIGH AVAILABLE CLUSTER IN WEB SERVER WITH HEARTBEAT + DRBD + OCFS2Utah Networxs Consultoria e Treinamento

Rhel cluster basics 2Manoj Singh

Paper on RDMA enabled Cluster FileSystem at Intel Developer Forumsomenathb

Lisa 2015-gluster fs-introductionGluster.org

drbd9_and_drbdmanage_may_2015Alexandre Huynh

Using CloudStack With Clustered LVMMarcus L Sorensen

Redhat ha cluster with pacemakerIndika Dias

GlusterFS CTDB IntegrationEtsuji Nakai

Lavigne bsdmag-jan2012Dru Lavigne

Semelhante a TLUG Meeting 2008/09/13 - Gosuke Miyashita Discusses Scalable Storage Solutions (20)

High Availability Storage (susecon2016)

Red Hat Global File System (GFS)

SUSE Expert Days Paris 2018 - SUSE HA Cluster Multi-Device

Linux containers – next gen virtualization for cloud (atl summit) ar4 3 - copy

OpenNebulaConf 2016 - The DRBD SDS for OpenNebula by Philipp Reisner, LINBIT

Distributed replicated block device

Disaster recovery of OpenStack Cinder using DRBD

Linux High Availability Overview - openSUSE.Asia Summit 2015

GlusterFS and Openstack Storage

Rhel cluster basics 1

Realizing Linux Containers (LXC)

HIGH AVAILABLE CLUSTER IN WEB SERVER WITH HEARTBEAT + DRBD + OCFS2

Rhel cluster basics 2

Paper on RDMA enabled Cluster FileSystem at Intel Developer Forum

Lisa 2015-gluster fs-introduction

drbd9_and_drbdmanage_may_2015

Using CloudStack With Clustered LVM

Redhat ha cluster with pacemaker

GlusterFS CTDB Integration

Lavigne bsdmag-jan2012

Mais de Gosuke Miyashita

Walter ファミリーの紹介 at Shibuya.go#1Gosuke Miyashita

Serverspec at Testing Framework MeetingGosuke Miyashita

Serverspec at July Tech Festa 2013Gosuke Miyashita

Serverspec at hbstudy #45Gosuke Miyashita

NoSQLに関するまとめGosuke Miyashita

イベント駆動プログラミングとI/O多重化Gosuke Miyashita

Maglica - A Simple Internal Cloud Tool at #techkayacGosuke Miyashita

DevOps とは何か何であるべきかGosuke Miyashita

Inside Sqale's Backend at RubyConf Taiwan 2012Gosuke Miyashita

Ia型超新星とチャンドラセカール限界Gosuke Miyashita

How Perl Changed My LifeGosuke Miyashita

Inside Sqale's Backend at YAPC::Asia Tokyo 2012Gosuke Miyashita

Inside Sqale's Backend at Sapporo Ruby Kaigi 2012Gosuke Miyashita

Assurer - a pluggable server testing/monitoring frameworkGosuke Miyashita

Open Source System Administration Framework - FuncGosuke Miyashita

10分でわかるDevOpsGosuke Miyashita

DevOpsって何？Gosuke Miyashita

PuppetのススメGosuke Miyashita

Puppet Best Practices? at COOKPADGosuke Miyashita

How Danga::Socket handles asynchronous processing and how to write asynchrono...Gosuke Miyashita

Mais de Gosuke Miyashita (20)

Walter ファミリーの紹介 at Shibuya.go#1

Serverspec at Testing Framework Meeting

Serverspec at July Tech Festa 2013

Serverspec at hbstudy #45

NoSQLに関するまとめ

イベント駆動プログラミングとI/O多重化

Maglica - A Simple Internal Cloud Tool at #techkayac

DevOps とは何か何であるべきか

Inside Sqale's Backend at RubyConf Taiwan 2012

Ia型超新星とチャンドラセカール限界

How Perl Changed My Life

Inside Sqale's Backend at YAPC::Asia Tokyo 2012

Inside Sqale's Backend at Sapporo Ruby Kaigi 2012

Assurer - a pluggable server testing/monitoring framework

Open Source System Administration Framework - Func

10分でわかるDevOps

DevOpsって何？

Puppetのススメ

Puppet Best Practices? at COOKPAD

How Danga::Socket handles asynchronous processing and how to write asynchrono...

Último

TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc

How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesThousandEyes

Generative Artificial Intelligence: How generative AI works.pdfIngrid Airi González

The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3

Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Mark Goldstein

(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...AliaaTarek5

Assure Ecommerce and Retail Operations Uptime with ThousandEyesThousandEyes

Modern Roaming for Notes and Nomad – Cheaper Faster Better Strongerpanagenda

New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada

Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3

Potential of AI (Generative AI) in Business: Learnings and InsightsRavi Sanghani

Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3

Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA

Why device, WIFI, and ISP insights are crucial to supporting remote Microsoft...panagenda

Enhancing User Experience - Exploring the Latest Features of Tallyman Axis Lo...Scott Andery

The Future Roadmap for the Composable Data Stack - Wes McKinney - Data Counci...Wes McKinney

Data governance with Unity Catalog PresentationKnoldus Inc.

Rise of the Machines: Known As Drones...Rick Flair

So einfach geht modernes Roaming fuer Notes und Nomad.pdfpanagenda

TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey

TLUG Meeting 2008/09/13 - Gosuke Miyashita Discusses Scalable Storage Solutions

1. TLUG Meeting 2008/09/13 Gosuke Miyashita

8. CLVM clvmd distributes metadata among cluster nodes Logical volumes presented to each cluster node Logical volume on shared storage LVM2 Metadata clvmd LVM2 Metadata clvmd LVM2 Metadata clvmd

10. GNBD TCP/IP network GNBD client GNBD client GNBD client GNBD Server Exported block device

11.

12. GFS2 These nodes can access to the GFS2 file system simultaneously GNBD Server GFS2 GNBD client cman GNBD client cman GNBD client cman

13.

14. DRBD Replication Server Block Device Server Block Device

15.

16. DM-MP with SAN storage /dev/sda1 /dev/sdb1 Seen as one device /dev/mapper/mpath0 active/passive or active/active Node HBA1 HBA2 SAN swtich 1 SAN swtich 2 Storage CNTRLR1 CNTRLR2

17.

18. A scalable storage system /dev/VG0/LV0 (CLVM) mount /dev/VG0/LV0 /mnt cman GNBD cman GNBD GNBD Server GFS2 GNBD Server GFS2 Replication (DRBD) /dev/mapper/mpath0 (DM-MP) /dev/gnbd0 /dev/gnbd1 GNBD Server GFS2 GNBD Server GFS2 Replication (DRBD) /dev/mapper/mpath1 (DM-MP) /dev/gnbd2 /dev/gnbd3

19. How to extend /dev/VG0/LV0 (CLVM) mount /dev/VG0/LV0 /mnt cman GNBD cman GNBD GNBD Server GFS2 GNBD Server GFS2 /dev/mapper/mpath0 /dev/gnbd0 /dev/gnbd1 GNBD Server GFS2 GNBD Server GFS2 /dev/mapper/mpath1 /dev/gnbd2 /dev/gnbd3 GNBD Server GFS2 GNBD Server GFS2 /dev/mapper/mpath2 /dev/gnbd4 /dev/gnbd5

20.

21.

22.

TLUG Meeting 2008/09/13 - Gosuke Miyashita Discusses Scalable Storage Solutions

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (20)

Semelhante a TLUG Meeting 2008/09/13 - Gosuke Miyashita Discusses Scalable Storage Solutions

Semelhante a TLUG Meeting 2008/09/13 - Gosuke Miyashita Discusses Scalable Storage Solutions (20)

Mais de Gosuke Miyashita

Mais de Gosuke Miyashita (20)

Último

Último (20)

TLUG Meeting 2008/09/13 - Gosuke Miyashita Discusses Scalable Storage Solutions