GDG Cloud Southlake 32: Kyle Hettinger: Demystifying the Dark Web
Storage Area networking and upcoming trends
1. Overview of Storage Area
Networking(SAN) and Upcoming Trends
BY
DEEPANSHU LULLA
2. Contents
• Storage Devices. Why Storage networking is important
• Storage networking protocols: Front end protocols vs back end protocols
• Front end Access to Storage Devices(SAN vs NAS)
• SAN
• SAN Architectures
• SAN protocols
Fiber channel SAN and Fiber Channel protocol
ISCSI
FCOE
• SAN virtualization techniques
• Vmware’s Approach to Storage Virtualization
3. Storage Devices
• Data is growing at an exponential rate.
• Effectively storing, accessing, protecting and
managing data is becoming challenging.
• Big data and Internet of Things(IoT) will
continuously to push the requirements.
4. Storage Networking protocols
Two primary communication types in storage
devices:
• Front End Communication
Protocols: iSCSI, FCoE, FC, NFS, CIFS etc.
• Back end Communication
Protocols: SCSI,IDE/ATA
Focus of presentation: Block based Front end
protocols
5. Front End Access to Storage Devices
• Based on the type of data access
Block-based storage system (SAN) eg iSCSI, FC
File-based storage system (NAS) eg NFS, CIFS, HDFS
Object-based storage system eg. Amazon S3,Openstack
shift
• We will focus on Block based storage systems.
6. Storage Area Networks
• Network used for inter communication between compute and storage systems.
• LUNs(Logical unit Number) fundamental storage unit which is shared over the network.
7. SAN Architectures
• SAN network Architectures are of three types
Point to Point
Arbitrated loop
Switched fabric
8. SAN Access Frontend Protocols
• SCSI
Serial Connection Serial Interface. Command response protocol.
Works in the Application layer of all block based protocols.
• Primary protocols we will discuss are
Fiber Channel(FC)
ISCSI(Internet SCSI)
Fiber Channel Over Ethernet
Front End Block Based Protocols
9. Fiber channel
• Fiber Channel: used to connect computer data storage to servers.
• Ports where the fibre channel connect to devices is known as Host based
Adapter.
• Uses WWN addresses instead of MAC addresses
10. Fiber Channel protocol(FCP)
Fiber Channel Protocol (FCP) is the SCSI interface protocol utilizing an underlying Fiber Channel
connection.
11. IP SAN using ISCSI
• Advantages of IP SAN
Existing IP infrastructure can be leveraged.
IP SANs can be used to connect SANs over long
Distances.
12. ISCSI
•ISCSI
Encapsulates SCSI I/O over TCP/IP.
Client Server Protocol. Used to establish sessions
Between devices.
Handles login, authentication, target discovery, and
session management.
13. Fiber Channel over Ethernet (FCoE)
• Transports FC data along with regular Ethernet traffic
over a Converged network adapter.
• FCoE protocol encapsulates FC fames into Ethernet frames.
14. SAN Virtualization
Block-level storage virtualization Virtual SAN (also called virtual fabric) N_Port ID virtualization (NPIV)
Note VSAN terminology in EMC is different from VSAN in Vmware
15. Vmware’s Approach of Storage
Virtualization
• VMFS stands for Virtual Machine
File system.
• VMFS can be distributed or shared
among Virtual machines.
16. Summary
• Need for storage is constantly rising with increase in data
• SAN is one of the solutions to this fortunate problem.
• Using SAN one accesses data using LUNs(Chunks of storage units).
• Front end SAN protocols: FC, iSCSI, FCOE
• SAN virtualization techniques increase the overall SAN efficiency by hardware reuse, increasing redundancy etc.
By 2016 we almost crossed 3.74 ZB of data.
1 ZB=10^21
So we store our data in these enterprise storage devices
Front End Communication
Communication between compute systems and storage controllers
Back end Communication
Communication between storage controllers and disks
SCSI: Serial communication serial interface
Integrated Drive Electronics or IBM Disc Electronics, IDE is more commonly known as ATA or Parallel ATA (PATA)
difference b/w Block file based storage systems is who is hosting the file systems
The file system has two components: user component and storage component.
The user component of the file system performs functions such as hierarchy management, naming, and user access control.
The storage component maps the files to the physical location on the storage device
Good examples of NAS include HDFS(Hadoop distributed file system) and Google File system.
In Object level access type of access, the file system’s user component resides on the compute system
and the storage component resides on the storage system. Data is accessed over a network in terms of self-contained objects with a unique object identifier.Tersely, Block Storage and File Storage are designed to be consumed by your operating system, Object Storage is designed to be consumed by your application.Another difference between Object Storage and the other storage types is that object metadata lives directly in the object, rather than e.g. in a separate inode.
A storage area network is a network which enables communication between compute and storage system or between two storage systems.
In block access data centers first created "islands" of SCSI disk arrays as direct-attached storage (DAS), each dedicated to an application, often visible as a number of "virtual hard drives" addressed as Logical Unit Numbers (LUNs). Usually LUNS are not shared among Operating systems
The whole concept of SAN was probably designed keeping how could one share a hard drive over a network.
Fiber Channel network topologies consist of the following:
Point to Point: enables direct connection between nodes
Arbitrated Loop—A network consisting of hub which forwards packets to all ports
Switched Fabric—A network consisting of switching elements.
Small Computer System Interface (SCSI) is a set of standards for physically connecting and transferring data between computers and peripheral devices.
SCSI is the command protocol that works at the application layer of the Open SystemInterconnection (OSI) model.
Set of standards defined for physical connection and transferring data between external devices.
FC operates at very high rates (1, 2, 4, 8, 16, 32, and 128 Gbps) though in practice I have seen only upto 16 gbps used till now
Difference between HBA and CNA is that CNA has the ability send traffic through both Ethernet and FC network whereas HBA are limited to FC use only
The FC cable connects to devices using either Host based Adapter or Converged Network Adapter.
Every HBA has a pre configured WWN(World wide name) Address which is similar to MAC address in Ethernet networks.
Each HBA has a unique World Wide Name (WWN), which is similar to an Ethernet MAC address in that it uses an Organizationally Unique Identifier (OUI) assigned by the IEEE. However, WWNs are longer (8 bytes) unlike MAC(6 bytes)
FC ProtocolFiber Channel consists of the following layers:
FC-0 -- The interface to the physical media
FC-1 -- The encoding and decoding of data and out-of-band physical link control information for transmission over the physical media
FC-2 -- The transfer of frames, sequences and Exchanges comprising protocol information units.
FC-3 -- Common Services required for advanced features such as striping, hunt group and multicast.
FC-4 -- Application interfaces that can execute over Fiber Channel such as the Fiber Channel protocol for SCSI (FCP).
IP SANs can be used to connect SANs over long Distances as IP network has no distance limitation unlike FC
Key components for iSCSI communication are:• iSCSI initiators such as an iSCSI HBA• iSCSI targets such as a storage system with an iSCSI port• IP-based network such as a Gigabit Ethernet LANAn iSCSI initiator sends commands and associated data to a target and the targetreturns data and responses to the initiator. iSCSI is the session-layer protocol that initiates a reliable session between devices that recognizeSCSI commands and TCP/IP.
IP SANs can be used to connect SANs over long Distances as IP network has no distance limitation unlike FC
Key components for iSCSI communication are:• iSCSI initiators such as an iSCSI HBA• iSCSI targets such as a storage system with an iSCSI port• IP-based network such as a Gigabit Ethernet LANAn iSCSI initiator sends commands and associated data to a target and the targetreturns data and responses to the initiator. iSCSI is the session-layer protocol that initiates a reliable session between devices that recognizeSCSI commands and TCP/IP.
The FCoE protocol specification replaces the FC-0 and FC-1 layers of the FC stack with Ethernet.This provides the capability to carry the FC-2 to the FC-4 layer over the Ethernet layer.
Block-level storage virtualization aggregates block storage devices (LUNs) and enablesprovisioning of virtual storage volumes, independent of the underlying physical storage. Avirtualization layer, which exists at the SAN, abstracts the identity of block-based storage systemsand creates a storage pool by aggregating LUNs from the storage systems.
Virtual SAN (also called virtual fabric) is a logical fabric on an FC SAN, which enablescommunication among a group of nodes regardless of their physical location in the fabric. In aVSAN, a group of node ports communicate with each other using a virtual topology defined on thephysical SAN
N_Port ID virtualization (NPIV) enables a single N_Port (such as an FC HBA port) tofunction as multiple virtual N_Ports. Each virtual N_Port has a unique WWPN identity inthe FC SAN. This allows a single physical N_Port to obtain multiple FC addresses.Hypervisors leverage NPIV to create virtual N_Ports on the FC HBA and then assign thevirtual N_Ports to virtual machines (VMs). A virtual N_Port acts as a virtual FC HBA port.This enables a VM to directly access LUNs assigned to it (see the figure on the slide).NPIV enables an administrator to restrict access to specific LUNs to specific VMs usingsecurity techniques like zoning and LUN masking; similarly to the assignment of a LUN toa physical compute system.
Datastores are logical containers that hide specifics of each storage device and provide a uniformmodel for storing virtual machine files. Depending on the type of storage that you use, datastorescan be formatted with VMware vSphere® VMFS, or datastores can be formatted with a file systemnative to a storage device that is shared using the NFS protocol.