In this 90-minute hands-on workshop, some of the key contributors to OpenNebula will walk attendees through the configuration and integration aspects of the networking subsystem in OpenNebula. The session will also include lightning talks by community members describing aspects related to Networking, NFVs and SDNs with OpenNebula: - Deployment scenarios - Integration - Tuning & debugging - Best practices

1. Networking, NVFs & SDN Workshop
OpenNebulaConf 2016
4th
edition

2. Agenda for the Workshop
● OpenNebula Network Stack
● Hacking the Network Drivers
● Virtual Routers
● IPAM Drivers
● Use Cases
○ OnLife: OpeNebula & ONOS for CORD
○ BEACON: SDN for Networks across Clouds

3. OpenNebula Network Stack
OpenNebulaConf 2016
4th
edition

4. OpenNebula Network Model: View
Hyper. Hyper. Hyper. Hyper.
Virtual Network
VM VM VM VM
Datacenter Network
● Leaf-Spine switched or routed backbone
● Under-control of SDN
● Most deployments just few TOR switches
Virtual switches
Virtual Network
OpenNebula
SDN

5. OpenNebula Network Model: “built-in SDN”
Provide a central point to define, consume and set up Virtual
Networks for Virtual Machines
XML-RPC API
OpenNebula daemon
Infrastructure Drivers
Northbound Interface
● Virtual Network abstract management
● Lease addresses to VMs
● Reserve addresses for users or groups
Network
Southbound Interface
● Uniform interface to interact with network
elements
● Specialized for VM Networking

6. Northbound Interface
Logical Attributes
Addresses Space (multiple, disjoint ranges)
● IPv4
● IPv6 (Global & ULA)
● Ethernet - MAC addresses
Security Groups
● Managed as a separate entity
● Inbound & Outbound, TCP/UDP/ICMP
Custom Tags
Implementation & Physical Attributes
● VNET driver* (e.g. bridged,vxlan)
● Network specific (e.g. BRIDGE, VLAN_ID*)
● VM Context (e.g. DNS, GATEWAY)
VirtualNetworkDefinition

7. Virtual Network Tips
● Can be used as a logical SuperNet
○ Each AR can be a different entity (drivers, IPs, context…)
○ Example: Public Network as a set of AR’s
● Non VM leases can be put on hold or left out of the range
● Use reservations:
○ special purposes leases (e.g. routers, well-known services)
○ VDC or user assignment
○ Labels come in handy to organize VNETs

8. Southbound Interface: The Network Drivers
Three-phase setup
● pre, post and clean right before, after VM boot and shutdown
● It considers live-migrations
● Includes update_sg to push Security Group changes
OpenvSwtich IPAM
Linux Bridge
802.1Q Bridged
Security Group
iptables rules
VXLAN ebtables Bridged VLAN dummy
NFV Virtual Router
Setup networking for VMs interfacing with network & hypervisors
Stock Network drivers

9. VLAN Driver Structure
def activate
lock
process do |nic|
get_vlan_dev_name #eth0.200
create_bridge #brctl addbr onebr.200
create_vlan_dev #ip link add ...
exec_and_log("#{command(:brctl)} addif"
" #{nic[:bridge]} #{nic[:vlan_dev]}")
end
unlock
#!/usr/bin/env ruby
template64 = ARGV[0]
deploy_id = ARGV[1]
hm = VXLANDriver.from_base64(...)
exit hm.activate

10. The Drivers in Action: Example VXLAN
ens3 IP:10.4.4.11 (this IP is used to encapsulate
VM traffic in a UDP datagram, can be in a VLAN)
ens3.100 vxlan encap.
239.0.0.100 mc group
ens3.200 vxlan encap.
239.0.0.200 mc group
onbr.100 (bridge)
one-<vm_id>-<nic_id>
Infrastructure set in pre
operation, removed in clean
Predictable interface namesVM
switching/routing fabric:
● multicast
● MTU

11. Virtual Routers
OpenNebulaConf 2016
4th
edition

12. Virtual Routers (new in 5.0)
Virtual
Router
Virtual
Router
Virtual
Router
VR cluster
Virtual NetworkVirtual Network
● Alpine Linux
● Link any VLAN network
● HA configuration
● Support for reconfiguration
● First class management entity
● Sunstone specialized views
● CLI for vrouters management
● Can be configured per VDC

13. The Virtual Routers in Action

14. ● Approach: Virtualize some network functions by packaging
into an appliance
● Performance tuning
○ Hugepages*,
○ NUMA scheduling*,
○ PCI PF/SR-IOV co-allocation (fully supported in 5.0.2)
* Supported only to some extent in 4.14
● Other common pitfalls
○ Network MTU
○ virtio and segmentation offloading
NFV Implementation Tips

15. Security Groups
OpenNebulaConf 2016
4th
edition

16. Security Groups
● Defines a set of rules to accept specific traffic to/from a VM
● Rule
● SG’s can be applied to Virtual Networks or Address Ranges
● DEFAULT SG allows all traffic!
● A SG update will be propagated to running VMs (new 5.0.2)
● New SG cannot be added to VM NICs (detach/attach)
Traffic Type
● Protocol (TCP, UDP, IPSEC, ICMP)
● Port or ICMP type
Traffic Origin/Destination
● INBOUND/OUTBOUND
● IP (or IP set)
● OpenNebula Virtual Netwok

17. Security Groups
● Implemented using iptables kernel facilities & ipset
● Only for Linux bridge based virtual networks
Match outbound traffic for VM (at bridge port)
Match inbound traffic for VM (at bridge port)
opennebula chain
inbound chainMatch protocol, port, src or dst (RETURN)
ACCEPT (by default)
DROP (by default)
Match bridged packets

18. Security Groups in Action

19. IP Address Management
OpenNebulaConf 2016
4th
edition

20. IP Address Management
● Awesome contribution by
Christophe Palanché & Guillaume Oberlé
● Delegate IP management to external component per AR
● IPAM options in 5.2:
○ Built-in module for address management
○ Develop custom one based on provided template
○ Addons are welcome!

21. IP Address Management: Network
allocate_address get_address free_address
IPAM driver
AR definition with as many details as needed
AR=[
IPAM_MAD="dummy", TYPE="IP4", SIZE="20",
IP="10.0.0.2", GATEWAY="10.0.0.1",
DNS="10.0.0.1", ANY_IPAM_ATTR="doamin.com"
]
register_address_range
AR = [ IPAM_MAD="dummy", TYPE="IP4", SIZE="20" ]

22. IP Address Management: Addresses
register_address_range allocate_address get_address free_address
IPAM driver
Return ADDRESS requested by VM or Reservation
ADDRESS = [ IP = "10.0.0.10", SIZE = "5" ]
ADDRESS = [ SIZE = "5" ] (IP optional for allocate)

23. IPAM Drivers: Integration Sample
Guillaume Oberlé

24. OnLife: OpenNebula & ONOS for
COORD
OpenNebulaConf 2016
4th
edition

25. OnLife: OpenNebula & ONOS for CORD
Central Office Re-architected as Datacenter
● NFVs
● SDN
● Cloud
● Real-time configuration of new customer
services

26. OnLife: OpenNebula & ONOS for CORD

27. OnLife: OpenNebula & ONOS for CORD
VPdC

28. OnLife: OpenNebula & ONOS for CORD
VPdC VPdCVPdC
VPdC VPdC VPdC
VPdC
VPdC
VPdC
VPdC

29. OnLife: OpenNebula & ONOS for CORD
VPdC VPdCVPdC
VPdC VPdC VPdC
VPdC
VPdC
VPdC
VPdC

30. OnLife: OpenNebula & ONOS for CORD
VPdC VPdCVPdC
VPdC VPdC VPdC
VPdC
VPdC
VPdC
VPdC

31. OnLife: OpenNebula & ONOS for CORD
VPdC VPdCVPdC
VPdC VPdC VPdC
VPdC
VPdC
VPdC
VPdC

32. OnLife: OpenNebula & ONOS for CORD
one-onos-driver
├── clean
├── clean.d
│ └── onos.sh
├── onos.conf
├── post
├── post.d
├── pre
└── pre.d
└── onos.sh

33. OnLife: OpenNebula & ONOS for CORD
onos.conf
ONOS_URL=http://<ip>:8181/onos
ONOS_USER=<user>
ONOS_PASSWORD=<password>
OLTAPP_URL=$ONOS_URL/ctpd-olt-app/oltapp
CLOSFWD_URL=$ONOS_URL/closfwd-app/closfwdapp

34. OnLife: OpenNebula & ONOS for CORD
pre.d/onos.sh (194 lines)
1. OLTAPP (olt_dev, olt_port, bandwidth) ⇒ VLAN
2. Update VM template with VLAN through onegate
3. Create vPDC bridge
4. Tag veth (veth_clos ↔ veth_vpdc)
5. Add tagged veth to bridge
6. CLOSFWD. args: vlan_id
7. CLOSFWD_MULTI Internet. args: vlan_id, mac
8. CLOSFWD_MULTI Other services. args: vlan_id, mac

35. OnLife: OpenNebula & ONOS for CORD
# Get VLAN ID
URL="$OLTAPP_URL/$OLN_OLT_DEV/$OLN_OLT_PORT/$NO_VLAN_TAG/$OLN_B
ANDWIDTH"
RESPONSE=$(curl -s --user $ONOS_USER:$ONOS_PASSWORD -X POST -H
"Content-Type: application/json" "$URL")
VLAN_ID=$(echo "$RESPONSE" | tr , 'n' | grep vlanOut | tr -dc
0-9)

36. OnLife: OpenNebula & ONOS for CORD
# create bridge
sudo brctl addbr $BR_VPDC
sudo ip link set dev $BR_VPDC up
# tag vpdc side of veth
sudo ip link add link veth_vpdc veth_vpdc.$VLAN_ID type vlan id
$VLAN_ID
sudo ip link set dev veth_vpdc.$VLAN_ID up
# add ifaces to bridges
sudo brctl addif $BR_VPDC veth_vpdc.$VLAN_ID

37. OnLife: OpenNebula & ONOS for CORD
closfwd() {
SRC_DEV="$1"; DST_DEV="$2"; SRC_PORT="$3"; DST_PORT="$4";
VLAN="$5"; MAC="$6"
URL="$CLOSFWD_URL/$SRC_DEV/$DST_DEV/$SRC_PORT/$DST_PORT/$VLAN/$
MAC"
curl -s --user $ONOS_USER:$ONOS_PASSWORD -X POST -H
"Content-Type: application/json" "$URL"
}
# OLT - L2
closfwd "$OLN_CLOS_IN_DEV"
"$OLN_VPDC_IN_DEV"
"$OLN_CLOS_IN_PORT"
"$OLN_VPDC_IN_PORT"
"$VLAN_ID"
"$ANY_MAC"

38. OnLife: OpenNebula & ONOS for CORD
closfwd_multi() {
# regular fwd
closfwd "$SRC_DEV" "$DST_DEV"...
# multicast fwd
MAC=$(echo "$MAC" | awk -F ":" '{print
"33:33:ff:"$4":"$5":"$6}')
closfwd "$SRC_DEV" ...
}
# L3 - VR - INTERNET
closfwd_multi "$OLN_VPDC_OUT_DEV"
"$OLN_CLOS_OUT_DEV"
"$OLN_VPDC_OUT_PORT"
"$OLN_CLOS_OUT_PORT"
"$OLN_CLOS_OUT_VLAN_INTERNET"
"$OLN_VPDC_OUT_MAC_INTERNET"

39. OnLife: OpenNebula & ONOS for CORD
pre.d/clean.sh (131 lines)
1. Delete veth
2. OLTAPP: DELETE. args: port
3. CLOSFWD_DEL. args: vlan_id
4. CLOSFWD_MULTI_DEL Internet. args: vlan_id, mac
5. CLOSFWD_MULTI_DEL Other services. args:
vlan_id, mac

40. BEACON: SDN for Networks
across Clouds
OpenNebulaConf 2016
4th
edition

41. BEACON: Project Architecture

42. BEACON: Federated SDN
Virtual RouterVirtual Router
VM
Virtual Network (VLAN)Virtual Network (VLAN)
VM VM
Cross-site
Network
VM
Ethernet over IP (GRETAP) + IPSec
● Link multiple virtual networks (L3/L2)
● Support for hybrid configurations
● Support for L3 inter-DC

43. BEACON: Federated Cloud Networks
OpenNebula
KVM Host
EU
EC2
US
EC2 AWS
us_east
Demo: federated network overlay between local
OpenNebula and EC2 VPC (Virtual Private Cloud)

44. BEACON: Federated Cloud Networks
KVM Host
EU
EC2
US
InternetPrivate
bridge
Public
bridge
VM VM FA
192.168.0.x
VM VMFA
AWS VPC
10.0.1.x
147.96.81.134 52.71.148.30

45. BEACON: Federated Cloud Networks
KVM Host
EU
EC2
US
InternetPrivate
bridge
Public
bridge
VM VM
192.168.0.x
VM VM
AWS VPC
10.0.1.x
147.96.81.134 52.71.148.30
GRE tunnels
192.168.0.x
Network overlay
FA
FA

46. OpenNebulaConf 2016
4th
edition
Platinum
Gold
Silver
Community
THANKS!