4. The Software-Defined Data Center
Transform storage
by aligning it with
app demands
Management
tools give way
to automation
Expand virtual
compute to all
applications
Virtualize the
network for speed
and efficiency
4
6. Storage Market in Midst of Disruption
6
Key Drivers
Server flash
Falling storage prices
Abundant CPU cycles
Hypervisor-converged
infrastructure
Cloud economics
Server Storage
20-30 years ago
Shared Storage
10-15 years ago
New Forms
Today
7. New Storage Tiers Are Rapidly Growing
Flash: Enables New
Storage Architectures
• Flash is 50x – 2,000x faster than HDD
– 110K/140K IOPs R/W from 360GB MLC PCIe card1
– Less than $0.10 per IOP
• Eliminates the need to stripe across
100s of HDDs
• Enables high performance server-side
storage
Cloud: Enables Cost-Effective
Storage
• Highly scalable, pay-as-you-go
• Access through standard APIs
• Low cost for capacity
– $0.05 per GB per month2
• Forecasted to grow at 40% annually to
20183
7
Cloud Storage
1. Source: FuisionIO ioDrive2, Feb 2014 2. Source: Amazon S3, Feb 2014
3. Source: MarketsandMarkets Cloud Storage report -
http://www.marketsandmarkets.com/PressReleases/cloud-storage.asp
8. Today’s Challenge:
Massive Increase in Storage Demand & Complexity
Source: IDC, Yezhkova, Worldwide Enterprise Storage Systems Forecast, November 2013, #244293
Storage Growth
41%
YoY
Source: IDC, Storage Predictions 2014, January 2014, General Storage QuickPoll, #243511, n=307
8
9. The Hypervisor Opens Up New Opportunities
9
SAN / NAS
x86 Servers Cloud Storage
vSphere
The virtualization platform:
• Knows the needs of all
apps in real time
• Sits directly in the I/O path
• Global view of underlying
infrastructure
• Hardware agnostic
10. Object-based
Pool
SAN/NAS
Pool
Hypervisor
Converged Pool
Leveraging The Hypervisor We Can Transform Storage
Today Software-defined Storage
10
LUN
Array A
LUN
LUN
Array B
LUN
LUN
Abstract and pool
(Virtualized Data Plane)
Automate SLAs via
VM-centric policies
(Policy-based Control Plane)
VM level Data services
(Virtual Data Services)
SAN / NASx86 Servers
Cloud Object
Storage
vSphere
Replication
Snapshots
11. 11
2011-2013
2008-2010
2005-2007
VI 3.x
● VMFS
● Snapshots
● Storage vMotion
● NAS & iSCSI support
vSphere 5.x
● Storage DRS
● Profile-driven Storage
● VASA
● vSphere Storage Appliance
● vSphere Data Protection
● vSphere Replication
● vSphere Flash ReadvSphere 4.x
● Thin Provisioning
● Storage I/O control
● Boot from SAN
● VAAI
● Linked mode
2014+
Software-defined Storage
vSphere 5.5 & VSAN
VMware Leads the Way to A New Approach
Software-Defined Storage represents the next step in Storage Evolution
12. Software-Defined Storage
12
Bringing the efficient operational model of virtualization to storage
Virtual Data Services
Data Protection Mobility Performance
Policy-driven Control Plane
SAN / NAS
SAN/NAS Pool
Virtual Data Plane
x86 Servers
Hypervisor-converged
Storage pool
Object Storage Pool
Cloud Object
Storage
Virtual SAN
14. Virtual SAN: Radically Simple Hypervisor-Converged Storage
14
vSphere + Virtual SAN
…
• Software-defined storage embedded in
vSphere
• Runs on any standard x86 server
• Pools HDD/flash into a shared datastore
• Managed through storage policy-based
management framework
• High performance through flash acceleration
• Highly resilient - zero data loss in the event of
hardware failures
• Deeply integrated with the VMware stack
The Basics
Hard disks
SSD
Hard disks
SSD
Hard disks
SSD
Virtual SAN Shared
Datastore
15. 15
● Installs in two clicks
● Managed from vSphere Client
● Policy-based management
● Self-tuning and elastic
● Deep integration with VMware stack
Radically Simple
● Embedded in vSphere kernel
● Flash-accelerated
● Up to 915K IOPs from 16 nodes
cluster
● Matches the VDI density of all flash
array
● Best price/performance
High Performance Lower TCO
● Eliminates large upfront investments
(CAPEX)
● Grow-as-you-go (OPEX)
● Flexible choice of industry standard
hardware
● Does not require specialized skills
Virtual SAN Key Benefits
16. Virtual SAN is Deeply Integrated with VMware Stack
16
Ideal for VMware Environments
vMotion
vSphere
HA
DRS
Storage vMotion
vSphere
Snapshots
Linked Clones
VDP Advanced
vSphere Replication
Data Protection
VMware View
Virtual Desktop
vCenter Operations Manager
vCloud Automation Center
IaaS
Cloud Ops and Automation
Site Recovery Manager
Disaster Recovery
Site
A
Site
B
Storage Policy-Based Management
17. vSphere + VSAN
19
Virtual SAN resiliency:
• Simple to set up via policy
• Delivered on per VM basis
• zero data loss in case of disk, network or
host failures
• Ensures zero downtime from disk or
network failures
• Interoperable with vSphere HA and
Maintenance Mode
• Modularizes infrastructure for efficient
data center operations through break-fix
model
Virtual SAN Is Highly Resilient Against Any Hardware Failure
Virtual SAN is designed to ensure data is never lost in case of failures
18. High Performance with Elastic and Linear Scalability
20
Notes: based on IOmeter benchmark
Mixed = 70% Read, 4K 80% random Notes: Based on View Planner benchmark
Up to 2M IOPs in 32 Node Cluster Comparable VDI density to an All Flash Array
19. 21
Embedded in the Hypervisor
• No virtual appliance needed
• Streamlines data path
• Makes optimal data placement and I/O
optimizations for enhanced performance
Storage Virtual
Appliance
Read/Write
Caching
Read
Cache
VSAN Datastore
Flash Accelerated Architecture
vSphere + VSAN
• Write buffer accelerates write performance
• SSD accelerate read performance
• Data persists on HDD
Persistency Layer
Architecture Delivers Superior Performance
Virtual SAN is embedded into the vSphere kernel to minimize the I/O data path
20. Scale UP
Add more Disks
IOPSCapacity
40 TB
400 TB
4.4 PB
Scale OUT
Add more nodes
• Elastic
Grow or shrink on demand
• Granular
Add single nodes or disks
• Non-disruptive
No app downtime
Virtual SAN Enables Elastic Linear Scaling
of Performance and Capacity
No More Complex Forecasting & Large Upfront Investments
“Virtual SAN enables us to scale our
storage infrastructure and while
providing the necessary redundancy.
This allows us to be more agile and
bring our solutions to market faster.”
— Frans Van Rooyen
Cloud Architect, Adobe
22
21. Virtual SAN Delivers Enterprise-Grade Scale
2M
IOPS
3,200
VMs
4.4
Petabytes
Maximum Scalability per Virtual SAN Cluster
32
Hosts
22. Virtual SAN Reduces CAPEX and OPEX for Better TCO
24
CAPEX
• Server-side economics
• No Fibre Channel network
• Pay-as-you-grow
OPEX
• Simplified storage configuration
• No LUNs
• Managed directly through
vSphere Web Client
• Automated VM provisioning
• Simplified capacity planning
As Low as
$0.50/GB2
As Low as
$0.25/IOPS
5X Lower
OPEX4
Up to 50%
TCO
Reduction
As Low as
$50/Desktop1
1. Full clones
2. Usable capacity
3. Estimated based on 2013 street pricing, Capex (includes storage hardware + Software License costs)
4. Source: Taneja Group
23. • Compared to external storage at scale
• Estimated based on 2013 street pricing, Capex (includes storage hardware + Software License costs)
• Additional savings come from reduced Opex through automation
• Virtual SAN configuration: 9 VMs per core, with 40GB per VM, 2 copies for availability and 10% SSD for performance
Granular Scaling Eliminates Overprovisioning
Delivers Predictable Scaling and ability to Control Costs
VSAN enables predictable
linear scaling
Spikes correspond to
scaling out due to IOPs
requirements
25
24. Two Ways to Build a Virtual SAN Node
26
Completely Hardware Independent
1. Virtual SAN Ready Node
…with multiple options available at GA + 30
Preconfigured server ready to use Virtual
SAN…
2. Build Your Own
…using the Virtual SAN Compatibility Guide*
Choose individual components …
SSD or PCIe
SAS/NL-SAS/ SATA HDDs
Any Server on vSphere
Hardware Compatibility List
HBA/RAID Controller
⃰ Note: For additional details, please refer to Virtual SAN VMware Compatibility Guide Page
⃰ Components for Virtual SAN must be chosen from Virtual SAN HCL, using any other components is unsupported
25. Flexibly Configure For Performance And Capacity
27
Performance
2xCPU – 8-core
128GB Memory
2xCPU – 8-core
128GB Memory
2xCPU – 8-core
128GB Memory
1x
400GB MLC SSD
(~15% of usable capacity)
1x
400GB MLC SSD
(~10% of usable capacity)
2x
400GB MLC SSD
(~4% of usable capacity)
5x
1.2TB 10K SAS
7x
2TB 7.2K NL-SAS
10x
4TB 7.2K NL-SAS
IOPS1
Raw Capacity
~20-15K
6TB
~15-10K
14TB
~10-5K
40TB
Capacity
1. Mix workload 70% Read, 80% Random
Estimated based on 2013 street pricing, Capex
(includes storage hardware + Software License costs)
26. Virtual SAN Pricing and Packaging
28
Virtual SAN
with Data
Protection
$3,590 /
CPU
Virtual SAN
(1 CPU)
vSphere Data
Protection
Advanced
(1 CPU)
Standalone
• Fully featured, no scale limit.
• Includes vSphere Distributed
Switch
• Applies only to internal drives
(no JBOD/ext. storage)
• Must license all CPU in a
cluster
Standalone
• 10 license package size
aligned with View
For Any Workload For Virtual Desktop Only Launch Promos
VSA to VSAN
upgrade
$11,475 / bundle
Virtual SAN
(6 CPUs per
bundle)
Register and
download promo
(Minimum 10
license
purchase)
Virtual SAN
(1 CPU)
Beta Promo
Virtual SAN for
Desktop
$50 / CCU
Bundle Promos
Virtual
SAN
$2,495 / CPU
20% Discount 20% Discount 20% Discount
Note: Virtual SAN also available on regional price lists on VMware supported local currencies.
28. Virtual SAN is NOT a Virtual Storage Appliance
30
– Virtual SAN is fully integrated with vSphere (ESXi & vCenter)
– Drivers embedded in ESXi 5.5 contain the Virtual SAN smarts
– Kernel modules:
• Provide the shortest path for I/O
• Remove unnecessary management overheads when dealing with an appliance
• Do not consume resources unnecessarily
Virtual SAN – Embedded into vSphereVirtual SAN – Not a VSA
+
VSA
29. 31
Management Clusters
Typical use cases for VSAN
Backup and DR Target
DMZ / Isolated
Tier 2 / Tier 3
Test / Dev / Staging
Private cloud
Virtual Desktop
ROBO
VDI
Site A Site B
vSphere
VSAN
30. Hardware Requirements
32
Any Server on the VMware
Compatibility Guide
• SSD, HDD, and Storage Controllers must be listed on the VMware Compatibility Guide for VSAN
http://www.vmware.com/resources/compatibility/search.php?deviceCategory=vsan
• ESXi 5.5 Hosts: Minimum 3, Maximum 32
1Gb/10Gb NIC
SAS/SATA Controllers (RAID Controllers must work in
“pass-through” or RAID0” mode
SAS/SATA/PCIe SSD
SAS/NL-SAS/SATA HDD
At least 1 of
each
4GB to 8GB USB, SD Cards
31. Boot Devices
● What installation device to use:
– Depends on amount of host memory
– Up to 512 GB
– Use SD/USB devices as the installation media.
– 512 GB or greater
– Use a separate magnetic disk or solid stated disk as the installation
device
33
32. Flash Based Devices
In Virtual SAN ALL read and write operations always go directly to the Flash tier.
Flash based devices serve two purposes in Virtual SAN
1. Non-volatile Write Buffer (30%)
– Writes are acknowledged when they enter prepare stage on SSD.
– Reduces latency for writes
2. Read Cache (70%)
– Cache hits reduces read latency
– Cache miss – retrieve data from HDD
Choice of hardware is the #1 performance
differentiator between Virtual SAN configurations.
34
33. Magnetic Disks (HDD)
• SAS/NL-SAS/SATA HDDs supported
– 7200 RPM for capacity
– 10000 RPM for performance
– 15000 RPM for additional performance
• NL SAS will provide higher HDD controller queue depth at same drive rotational speed and
similar price point
– NL SAS recommended if choosing between SATA and NL SAS
• Differentiate performance between clusters with SSD selection, and SSD:HDD ratio.
• Flash rule of thumb guideline is 10% of anticipated capacity usage
35
34. Storage Controllers
• SAS/SATA Storage Controllers
– Pass-through or “RAID0” mode supported
• Performance using RAID0 mode is controller dependent
– Check with your vendor for SSD performance behind a RAID-controller
• Storage Controller Queue Depth matters
– Higher storage controller queue depth will increase performance
• Validate number of drives supported for each controller
36
35. Network
• 1Gb / 10Gb supported
– 10Gb shared with NIOC for QoS will support most environments
– If 1GB then recommend dedicated links for Virtual SAN
• Jumbo Frames will provide nominal performance increase
– Enable for greenfield deployments
• Virtual SAN supports both VSS & VDS
– NIOC requires VDS
– Nexus 1000v – Should work but hasn't been fully tested
• Network bandwidth performance has more impact on host evacuation, rebuild times
than on workload performance
37
36. Firewalls
• Virtual SAN Vendor Provider (VSANVP)
– Inbound and outbound - TCP 8080
• Cluster Monitoring, Membership, and Monitoring Services (CMMDS)
– Inbound and outbound UDP 12345 - 23451
• Reliable Datagram Transport (RDT)
– Inbound and outbound TCP 2233
38
37. Technical Characteristics
Virtual SAN is a cluster level feature similar to:
– vSphere DRS
– vSphere HA
– Virtual SAN
Deployed, configured and manage from vCenter through the vSphere Web Client (ONLY!).
– Radically simple
• Configure VMkernel interface for Virtual SAN
• Enable Virtual SAN by clicking Turn On
39
38. Virtual SAN Implementation Requirements
• Virtual SAN requires:
– Minimum of 3 hosts in a cluster configuration
– First 3 host MUST!!! contribute storage
• vSphere 5.5 U1 or later
– Maximum of 32 hosts
• Not all hosts must contribute storage (hosts #4 -
#32)
– Locally attached disks
• Magnetic disks (HDD)
• Flash-based devices (SSD)
– Network connectivity
• 1GB Ethernet
• 10GB Ethernet (preferred)
40
esxi-01
local storage local storage local storage
vSphere 5.5 U1 Cluster
esxi-02 esxi-03
cluster
HDDHDD HDD
39. Virtual SAN Policies
• Virtual SAN currently surfaces five unique storage capabilities to vCenter.
43
40. Number of Failures to Tolerate
• Number of failures to tolerate
– Defines the number of hosts, disk or network failures a storage object can tolerate. For “n” failures
tolerated, “n+1” copies of the object are created and “2n+1” host contributing storage are required.
44
vsan network
vmdkvmdk witness
esxi-01 esxi-02 esxi-03 esxi-04
~50% of I/O ~50% of I/O
Virtual SAN Policy: “Number of failures to tolerate = 1”
raid-1
41. Number of Disk Stripes Per Object
• Number of disk stripes per object
– The number of HDDs across which each replica of a storage object is distributed. Higher values may
result in better performance.
45
vsan network
stripe-2b witness
esxi-01 esxi-02 esxi-03 esxi-04
stripe-1b
stripe-1a stripe-2a
raid-0raid-0
VSAN Policy: “Number of failures to tolerate = 1” + “Stripe Width =2”
raid-1
42. Virtual SAN Storage Capabilities
• Force provisioning
– if yes, the object will be provisioned even is the policy specified in the storage policy is not satisfiable
with the resources currently available.
• Flash read cache reservation (%)
– Flash capacity reserved as read cache for the storage object. Specified as a percentage of logical size
of the object.
• Object space reservation (%)
– Percentage of the logical size of the storage object that will be reserved (thick provisioned) upon VM
provisioning. The rest of the storage object is thin provisioned.
46
43. Storage Capabilities Recommended Practices
47
Storage Capability Use Case Value
Number of failures to tolerate
(RAID 1 – Mirror)
Redundancy
Default 1
Max 3
Number of disk stripes per object
(RAID 0 – Stripe)
Performance Default 1
Max 12
Object space reservation Thick Provisioning Default 0
Max 100%
Flash read cache reservation Performance Default 0
Max 100%
Force provisioning Override policy Disabled
44. Virtual SAN Constructs and Artifacts
New Virtual SAN constructs, artifacts and terminologies:
• Disk Groups.
• VSAN Datastore.
• Objects.
• Components.
• Virtual SAN Network.
48
45. Virtual SAN Disk Groups
• Virtual SAN uses the concept of disk groups to pool together flash devices and magnetic disks
as single management constructs.
• Disk groups are composed of at least 1 flash device and 1 magnetic disk.
– Flash devices are use for performance (Read cache + Write buffer).
– Magnetic disks are used for storage capacity.
– Disk groups cannot be created without a flash device.
49
disk group disk group disk group disk group
Each host: 5 disk groups max. Each disk group: 1 SSD + 1 to 7 HDDs
disk group
HDD HDDHDDHDDHDD
46. Virtual SAN Datastore
• Virtual SAN is an object store solution that is presented to vSphere as a file system.
• The object store mounts the VMFS volumes from all hosts in a cluster and presents them as a
single shared datastore.
– Only members of the cluster can access the Virtual SAN datastore
– Not all hosts need to contribute storage, but its recommended.
50
disk group disk group disk group disk group
Each host: 5 disk groups max. Each disk group: 1 SSD + 1 to 7 HDDs
disk group
VSAN network VSAN network VSAN network VSAN networkVSAN network
vsanDatastore
HDD HDDHDDHDDHDD
47. Virtual SAN Objects
• Virtual SAN manages data in the form of flexible data containers called objects. virtual machine
files are referred to as objects.
• Virtual machines files are referred to as objects.
– There are four different types of virtual machine objects:
• VM Home
• VM swap
• VMDK
• Snapshots
• Virtual machine objects are split into multiple
components based on performance and availability
requirements defined in
VM Storage profile.
51
disk group disk group disk group disk group
Each host: 5 disk groups max. Each disk group: 1 SSD + 1 to 7 HDDs
disk group
VSAN network VSAN network VSAN network VSAN networkVSAN network
vsanDatastore
HDD HDD HDD HDD HDD
48. Virtual SAN Components
• Virtual SAN components are chunks of objects distributes across multiple hosts in a cluster in
order to tolerate simultaneous failures and meet performance requirements.
• Virtual SAN utilizes a Distributed RAID architecture to distribute data across the cluster.
• Components are distributed with the use of two
main techniques:
– Striping (RAID0)
– Mirroring (RAID1)
• Number of component replicas
and copies created is based on
the object policy definition.
52
disk group disk group disk group disk group disk group
VSAN network VSAN network VSAN network VSAN networkVSAN network
vsanDatastore
replica-1 replica-2
RAID1
HDD HDD HDD HDD HDD
49. Object and Components Layout
53
VSAN network VSAN network VSAN network VSAN networkVSAN network
Virtual SAN Storage Objects
R1
R0
R0
R0
Availability defined as
number of copies
Low level storage
objects would reside on
different hosts
VMFS VMFS VMFS
rolo2.vmdk
The VM Home directory object is formatted
with VMFS to allow a VM’s configuration files
to be stored on it.
Performance may include a stripe
width
VMFS
rolo1.vmdk
rolo.vmx, .log, etc
/vmfs/volumes/vsanDatastore/rolo/rolo.vmdk
disk group
HDD
disk group
HDD
disk group
HDD
disk group
HDD
disk group
HDD
50. Virtual SAN Network
• New Virtual SAN traffic VMkernel interface.
– Dedicated for Virtual SAN intra-cluster communication and data replication.
• Supports both Standard and Distributes vSwitches
– Leverage NIOC for QoS in shared scenarios
• NIC teaming – used for availability and not for bandwidth aggregation.
• Layer 2 Multicast must be enabled on physical switches.
– Much easier to manage and implement than Layer 3 Multicast
54
Management Virtual Machines vMotion Virtual SAN
Distributed Switch
20 shares 30 shares 50 shares 100 shares
uplink1 uplink2
vmk1 vmk2vmk0
51. Virtual SAN Scalable Architecture
56
• Scale up and Scale out architecture – granular and linearly storage, performance and compute
scaling capabilities
– Per magnetic disks – for capacity
– Per flash based device – for performance
– Per disk group – for performance and capacity
– Per node – for compute capacity
disk group disk group disk group
VSAN network VSAN networkVSAN network
vsanDatastore
HDD
disk group
HDD HDD HDD
disk group
VSAN network
HDD
sc
al
e
u
p
scale out
52. Understanding Failure Events
● Virtual SAN recognized two different types of hardware device events in order to define the
type of failed scenario:
– Absent
– Degraded
● Absent events are responsible to trigger the 60 minutes recovery operations.
– Virtual SAN will wait 60 minutes before starting the object and component recovery operations
– 60 minutes is the default setting for all absent events
– Configurable value via hosts advanced settings
57
53. Understanding Failure Events
● Degraded events are responsible to trigger the immediate recovery operations.
– Triggers the immediate recovery operation of objects and components
– Not configurable
● Any of the following detected I/O errors are always deemed degraded:
– Magnetic disk failures
– Flash based devices failures
– Storage controller failures
● Any of the following detected I/O errors are always deemed absent:
– Network failures
– Network Interface Cards (NICs)
– Host failures
58
59. ● VMUG Offerings (membership is FREE)
• 200 Local Groups
• 44 User Conferences
• Special Interest Groups organized around industry and technology
• Monthly e-Newsletter: VMUG Voice
• Virtual education, including webcasts, session recordings and more
• VMUG Advantage: exclusive discount package
Visit www.vmug.com to join!
The VMware User Group (VMUG) is a global independent customer-led organization which maximizes members’
use of VMware and partner solutions through knowledge sharing, training, collaboration and events.