OpenvSwitch Deep Dive

1. Open vSwitch Overview Compiled by Rajdeep Dua Twitter : @rajdeepdua VMware

2. What is Open vSwitch • Multi layer virtual Switch licensed under Apache 2.0 • Enables massive network automation through programmatic extension • Supports monitoring protocols like NetFlow, sFlow etc • Supports distribution across multiple physical layers like VMware ‘s vNetwork distributed vSwitch or Cisco’s Nexus 1000V • Included in Linux 3.3 Kernel by default • Most users space utilities are available in Linux distributions • Ongoing efforts to port Open vSwitch to Windows

3. Features • Visibility into inter-VM communication via NetFlow, sFlow® etc • Supports LACP - Link Aggregation Control Protocol • Supports VLAN • Provides fine-grained QoS control • Support for HFSC qdisc : QoS across traffic Aggregate connections • Per VM interface traffic policing

4. Supported Platforms • Default Switch in – Xen – KVM • Supported in ESXi • Integrated in – OpenStack, OpenNebula and vSphere • Supports – Ubuntu, Fedora, Debian and also FreeBSD – Currently porting to Windows?

5. Why Open VSwitch • Traffic between VMs and outside world – In Linux Hypervisors, it is through a bridge • Why then Open vSwitch? – Multi-server virtualization deployments – Dynamic end points – Logical abstractions – Integration or offloading with special purpose hardware

6. Design Considerations • • • • • The mobility of state Responding to network dynamics Maintenance of Logical tags Hardware Integration Goals – Keep in-kernel code as small as possible – Re-use existing sub-systems

7. Mobility of State • Network state associated with a network entity, which is a virtual machine – Should be easily identifiable – Migrate between different hosts • State – – – – – Soft state (Entry in L2 learning table) L3 forwarding state ACLs QoS policy Monitoring the configuration (Netflow, IPFIX, sFlow) • Open vSwitch is backed by – Real Data Model, allows development of structured automation systems – Migrate SPAN Rules, ACLs, Qos and live state reconstruction – Fast moving and slow network state between instances

8. Responding to Network Dynamics • Virtual environments are characterized by high-rates of change – VMs coming and going – changes to the logical network environments • Open vSwitch supports – Simple accounting and visibility support such as NetFlow, IPFIX and sFlow – A Network database OVSDB supports remote triggers – Supports OpenFlow as a method of exporting remote access control to traffic

9. Maintenance of Logic tags • Distributed Virtual Switches often maintain logical tags for a network context by appending and manipulating tags in the network packets – Used for uniquely identifying a VM – Efficiently and correctly manage these tags • Open vSwitch supports – Multiple methods of specifying and maintaining tagging rules – Tagging rules are stored in an optimized form so they don't have to be coupled with a heavyweight network device – Supports GRE, STT and VXLAN

10. Hardware integration • Virtualized hosting environments can be managed using the same mechanism for automated network control • Datapath in hardware instead of kernel • Ongoing efforts to port Open vSwitch to hardware chipsets.

11. Centralized Control • One OpenFlow connection per datapath – Exports idealized view of switch’s datapath • • • • Lookup based on L2-L4 Fill wildcarding and priorities Actions: forward, drop, modify, and queue Missed flows go to central controller • One Management channel per system – Switch-level configuration – Resources – Counters

12. Open vSwitch Data Structures

13. OpenvSwitch Internals

14. OpenvSwitch Daemon ovs-vswitchd implements the switch talks to the kernel via the netlink protocol

15. ovs-ofctl, ovs-dpctl ovs-ofctl – Management Utility for Open Flow ovs-dpctl – Open vSwitch datapath management utility

16. ovs-ofctl • Command Line for Managing the Open Flow related tasks show SWITCH show OpenFlow information dump-ports SWITCH [PORT] print port statistics dump-flows SWITCH print all flow entries queue-stats SWITCH [PORT [QUEUE]] dump queue stats add-flow SWITCH FLOW add flow described by FLOW add-flows SWITCH FILE add flows from FILE mod-flows SWITCH FLOW modify actions of matching FLOWs SWITCH  PORT  FLOW 

17. ovs-dpctl • Open vSwitch datapath management utility usage: ovs-dpctl [OPTIONS] COMMAND [ARG...] add-dp DP [IFACE...] add new datapath DP(with IFACEs) del-dp DP delete local datapath DP add-if DP IFACE... add each IFACE as a port on DP set-if DP IFACE... reconfigure each IFACE within DP del-if DP IFACE... delete each IFACE from DP dump-dps display names of all datapaths show show basic info on all datapaths show DP... show basic info on each DP dump-flows DP display flows in DP del-flows DP delete all flows from DP

18. OpenvSwitch: ovs-vswitchd ovs-vswitchd saves and changes the switch configuration into a database and talks to ovsdb-server, which manages ovsdb

19. OpenvSwitch : ovs-vsctl ovs-vsctl manages the switch through interaction with ovsdb-server

20. ovs-vsctl – Bridge commands : Manage the bridge – Port commands : Manage the Port – Interface commands : Manages the Interfaces – Controller commands : Get controller details – Manager commands : get manager – SSL commands : Configure SSL – Switch commands : Reset the Switch – Database commands : Get table details of ovsdb

21. OpenvSwitch Internals ovs-dpctl - Monitor and Administer Switch, works with any OpenFlow Switch ovs-appctl – Utility for managing logging levels ovs-vsctl manages the switch through ovsdb-server ovs-dbclient – manipulate database entries directly without ovsdbserver

22. ovsdb-client ovsdb-client: Open vSwitch database JSON-RPC client usage: ovsdb-client [OPTIONS] COMMAND [ARG...] Valid commands are: list-dbs [SERVER] list databases available on SERVER get-schema [SERVER] [DATABASE] retrieve schema for DATABASE from SERVER get-schema-version [SERVER] [DATABASE] retrieve schema for DATABASE from SERVER and report only its version number on stdout list-tables [SERVER] [DATABASE] list tables for DATABASE on SERVER

23. OpenvSwitch Internals ovsdb-server - Monitor and Administer Switch, works with any OpenFlow Switch ovsdb-tool – command line tool to manage database ovsdb - persists the data across reboots; configures ovs-vswitchd

24. OpenvSwitch Internals Kernel module – Designed to be fast and simple; Handles switching and tunneling Knows nothing about openflow, if flow found, actions are executed otherwise passed to the user space; Implements tunnels and caches flows

25. Forwarding Components • ovs-vswitchd (Slow Path) – Forwarding logic (learning, mirroring, VLANs and bonding) – Remote configuration and visibility • openvswitch_mod.ko (Fast Path) – Packet lookup, modification, and forwarding – Tunnel encapsulation/decapsulation

26. Forwarding Flows • The first packet in the flow is sent to the controller • The controller programs the data path's actions for a flow – Usually one, but may be a list – Action include: • Forward to port port or ports, mirror • Encapsulate and forward to controller • Drop • Returns the packet to the data path • Subsequent packets are handled by the data path

30. Example • Mininet – Run a Simple Topology with One Open vSwitch, 3 hosts • Pox Controller

31. ovs-vsctl Used to Manage bridges, ports, Interfaces • List Bridges for a Switch • List Ports associated with s1 $ sudo ovs-vsctl list-br s1 $ sudo ovs-vsctl list-ports s1 s1-eth1 s1-eth2 s1-eth3

32. ovs-vsctl Manage bridges, ports, Interfaces • List of Interfaces for a Switch • Each port has a single Interface in this case $ sudo ovs-vsctl list-ifaces s1 s1-eth1 s1-eth2 s1-eth3

33. ovs-dpctl Shows the data path $ ovs-dpctl show system@s1: lookups: hit:0 missed:33 lost:0 flows: 0 port 0: s1 (internal) port 1: s1-eth1 port 2: s1-eth2 port 3: s1-eth3

34. ovs-ofctl Overall Openflow Management $ sudo ovs-ofctl dump-flows s1 NXST_FLOW reply (xid=0x4): cookie=0x0, duration=2.507s, table=0, n_packets=1, n_bytes=98, idle_timeout=10,hard_timeout=30,priority=65535,icmp,in_port=2,vlan_tci=0x0000,dl_src=00:00:00:00:00:02,d l_dst=00:00:00:00:00:01,nw_src=10.0.0.2,nw_dst=10.0.0.1,nw_tos=0,icmp_type=8,icmp_code=0 actions=output:1 cookie=0x0, duration=2.492s, table=0, n_packets=1, n_bytes=98, idle_timeout=10,hard_timeout=30,priority=65535,icmp,in_port=3,vlan_tci=0x0000,dl_src=00:00:00:00:00:03,d l_dst=00:00:00:00:00:01,nw_src=10.0.0.3,nw_dst=10.0.0.1,nw_tos=0,icmp_type=8,icmp_code=0 actions=output:1 cookie=0x0, duration=2.496s, table=0, n_packets=1, n_bytes=98, idle_timeout=10,hard_timeout=30,priority=65535,icmp,in_port=3,vlan_tci=0x0000,dl_src=00:00:00:00:00:03,d l_dst=00:00:00:00:00:02,nw_src=10.0.0.3,nw_dst=10.0.0.2,nw_tos=0,icmp_type=0,icmp_code=0 actions=output:2 ….

35. ovsdb-client JSON RPC client for ovsdb $ sudo ovsdb-client list-dbs Open_vSwitch $ sudo ovsdb-client list-tables Table -----------Capability SSL Bridge Controller NetFlow Port Mirror Queue QoS Interface Open_vSwitch sFlow Manager

36. OVS in OpenStack • • • • Use Case One Private Subnet One VM Single Node Installation

37. Network Topology

38. Summary • OVS is an open source software switch implementing Open Flow • Is supported in most of the hypervisors • Runs in the Kernel and User space of Linux

39. Features • NIC bonding – with source-MAC load balancing (L2) – Active backup – L4 hashing – to achieve Load Balancing using TCP/UDP layers • OpenFlow protocol support (including many extensions for virtualization) • IPv6 support • Multiple tunneling protocols – – – – GRE VXLAN IPsec GRE and VXLAN over Ipsec

40. Features • Remote configuration protocol with C and Python bindings • Kernel and user-space forwarding engine options – Kernel space forwarding if there is a flow entry – Else goes to User space in the switch and eventually to controller • Multi-table forwarding pipeline with flow-caching engine – Standard requirement of OpenFlow 1.3 • Forwarding layer abstraction to ease porting to new software and hardware platforms