One method KVM users have to utilize their shared block storage (SAN) is via CLVM. This document describes how.

1. Using CloudStack
with
Clustered LVM
Marcus Sorensen
BetterServers, Inc

2. What's LVM?
● Logical Volume Manager
– Built on Linux device mapper modules
– Physical volumes create pools called volume
groups
– Volume groups provide extents from which we
obtain logical volumes
– Provides features like resizing, redundancy, thin
provisioning, snapshots

3. What about the clustered part?
● LVM can utilize a cluster manager such as corosync
to syncronize LVM metadata between nodes
● LVM has a configurable locking mechanism for
metadata changes
– /etc/lvm/lvm.conf
● locking_type
– 1 is local (default)
– 2 is external locking library
– 3 is built-in clustered locking (depends on how LVM was compiled)
● Locking handled by Corosync and DLM
– CMAN in RHEL/CentOS 6.x is corosync
– Ubuntu provides corosync as well
● Read more about configuring a Red Hat cluster

4. Why use CLVM for Primary Storage?
● Provides a way to utilize shared, block-based
storage, such as iSCSI or Fibre Channel
● Allows CloudStack to have a pool from which to
carve out volumes
● Block-based storage is high performance
● Shared storage is required for live migrations
● CLVM is relatively simple; it doesn't require HA
resources or any of the more difficult configurations
of a typical RedHat cluster.

5. Setting up the storage
1. Install a base RHEL/CentOS operating system
on all cluster nodes
2. Install cluster software
1. RHEL/CentOS: “yum install lvm2-cluster”
2. Ubuntu: “apt-get install clvm”
3. Edit /etc/lvm/lvm.conf to change locking_type
4. Edit or create /etc/cluster/cluster.conf
5. Start services
6. Create volume group(s)
7. Add into CloudStack

6. /etc/lvm/lvm.conf
● Set locking_type to 3
● Other tunables to be aware of:
– wait_for_locks
– filter

7. /etc/cluster/cluster.conf
Example:
<?xml version="1.0"?>
<cluster config_version="1" name="kvmcluster">
<cman transport="udpu"/>
<clusternodes>
<clusternode name="192.168.10.11" nodeid="1">
<fence><method name="1"> <device name="fence_ipmitool" nodename="host1"/></method></fence>
</clusternode>
<clusternode name="192.168.10.12" nodeid="2">
<fence><method name="1"> <device name="fence_ipmitool" nodename="host2"/></method></fence>
</clusternode>
<clusternode name="192.168.10.13" nodeid="3">
<fence> <method name="1"> <device name="fence_ipmitool" nodename="host3"/></method></fence>
</clusternode>
</clusternodes>
<fencedevices><fencedevice agent="fence_ipmitool" name="fence_ipmitool"/></fencedevices>
<fence_daemon post_join_delay="60"/>
<rm/>
</cluster>

8. Start Services
● RHEL/CentOS
– service cman start
– service clvmd start
● Ubuntu
– service cman start
– service clvm start
● Check services:
– cman_tool status
– cman_tool services
● Create volume groups
– Just use normal LVM commands on shared block devices

9. Add into CloudStack

10. Under the hood... briefly
● Volume groups and logical volumes are managed as storage
pools by libvirt in LibvirtStorageAdaptor
● Pools are activated/deactivated, created/destroyed
dynamically as needed

11. Limitations/bugs
● Snapshots consist of LVM snapshot, copy
snapshot to secondary, delete snapshot
● Due to nature of templating, templates
unnecessarily copied to primary storage
before use (fixed in master)
● New dm-thin module won't be of much use
with current implementations for thin-
provisioned LVM volumes

12. Looking to the future...
● Refactor to use Edison's storage plugin
framework
● Utilize dm-thin for
backingstore/clone/snapshot functionality
● Could provide (limited) dm-thin based
snapshot functionality directly via device-
mapper

13. Questions