Technical Presentation about the MidoNet architecture and in-depth discussion about MidoNet features like Distributed Layer 2 Switching, Distributed Layer 3 Routing, Firewall, NAT and Distributed Flow State.
About MidoNet
Taking an overlay-based approach to network virtualization, MidoNet sits on top of any IP-connected network, and pushes the network intelligence to the edge of the network, in software. MidoNet makes it possible to build an IaaS cloud with fully virtualized and distributed scale-out L2-L4 networking.
Presenter: Taku Fukushima, Midokura Engineering
4. Why do we need MidoNet?
• Demands for the virtualised networking
• Faster and more flexible provisioning
• Cloud IaaS requires virtualised networking
• Multi-tenancy
• Complete software-based solution
5. MidoNet Features
• L2- L3 Logical Switching
• Logical Routing
• State-less and Stateful NAT
• Logical and distributed Firewall
• L4 Load Balancing
• BGP and its ECMP multiplexing
• GRE and VXLAN tunneling
6. MidoNet Features
• OpenStack Neutron integration and MidoStack
• REST API
• VTEP support with OVSDB protocol
• Partial Docker integration
7. History of MidoNet (a dev’s perspective)
• Started with Midolman written in Python,
OpenStack Austin, Open vSwitch (including
userland)
• MidoNet 1.x
• Re-written with Java
• Scala was partially introduced
• Open-sourced in Nov, 2014 New!
• MidoNet 2.0 (WIP)
12. Datapath control via Netlink by Midolman
Open vSwitch Datapath
IF IF
Interfaces on the host
IF
VM VM VM Midolman
(MidoNet
agent)
Flow Table
Watch/modify
Add/remove flows
Host
Cache
Store virtu
Nova compute
13. GRE/VXLAN Tunneling
NSDB NSDB
NSDB
Private
Network
Host
Midol
man
Cache
Datapath
VM VM VM
Flow Table
Nova compute
MidoNet APINova
API
Neutron API
MidoNet Plugin
Host
Midol
man
Cache
Datapath
VM VM VM
Flow Table
Nova compute
BGP Gateway
Midol
man
Datapath
Flow Table
BGP Gateway
Midol
man
Datapath
Flow Table
GRE/VXLAN Tunneling
Internet
14. NSDB and Cluster API
NSDB NSDB
NSDB
dd/remove flows
Store virtual
topology
information
NSDB and Cluster API
15. OpenStack integration and APIs
NSDB
NSDB
Network
MidoNet APINova
API
Horizon MidoNet CLI
Add/remove flows
Neutron API
MidoNet Plugin
Host
Clients / Users
OpenStack
integration
and
MidoNet API
16. BGP with ECMP
NSDB NSDB
NSDB
Private
Network
Host
Midol
man
Cache
Datapath
VM VM VM
Flow Table
Nova compute
MidoNet APINova
API
Neutron API
MidoNet Plugin
Host
Midol
man
Cache
Datapath
VM VM VM
Flow Table
Nova compute
BGP Gateway
Midol
man
Datapath
Flow Table
BGP Gateway
Midol
man
Datapath
Flow Table
GRE/VXLAN Tunneling
Internet
20. MidoNet 101! Face-to-Face with the Distributed SDN・FOSDEM 2015
Distributed L2 Switching
20
VM 1 VM 2
Virtual Tenant
Router B
Virtual Topology
Physical Topology
ARP Request
Virtual
Switch B1
VM 1 VM 2
State Cluster
Virtual Switch B1
MAC Port Host
AC:CA:BA:00:00:01
AC:CA:BA:00:00:02
vPort 0
vPort 1
Host 0
Host 1
Tunnel Zone
GRE / VXLAN IPv4Host
192.168.0.1
10.0.0.1
Host 0
Host 1
MAC AC:CA:BA:00:00:01 MAC AC:CA:BA:00:00:02
vPort 1vPort 0
Host 0 Host 1
• State cluster based on ZooKeeper
• Stores the virtual topology
• Topology is cached by the MidoNet Agent
• Agents access data using publish-subscribe
21. MidoNet 101! Face-to-Face with the Distributed SDN・FOSDEM 2015
Layer 2 Gateways
21
VM 1 VM 2
Virtual Tenant
Router B
Virtual Topology
Physical Topology
Virtual
Switch B1
vPort 1vPort 0
Virtual Provider
Router
vPort L3GW
vPort L2GW
Layer 2 Network
VM 1 Host 0 Hardware VTEP
State Cluster
Layer 2 Network
VXLAN
L2 gateway for VXLAN
• The state cluster adds L2 gateway
functions
• Exchange state data with hardware
VXLAN tunnel end-points (VTEPs)
• Leverages virtualization at the edge to
optimize the traffic flow
L2 VXLAN
Gateway
22. MidoNet 101! Face-to-Face with the Distributed SDN・FOSDEM 2015
Distributed Layer 2 Networks
22
Private IP Network
Virtual Servers
VM 1
VM 2
Hardware VTEP
L2 Network
Hardware VTEP
Hardware VTEP
State Cluster
Virtual
Switch B1
VM 1 VM 2
vPort 1vPort 0
L2 Network
vPort L2GW 0 vPort L2GW 1 vPort L2GW 2
Physical Topology Virtual Topology
Scalability and High
23. MidoNet 101! Face-to-Face with the Distributed SDN・FOSDEM 2015
Distributed Layer 3 Routing
23
Private IP Network
Virtual Servers
VM 1
VM 2
Provider
Network
State Cluster
Virtual
Switch B1
VM 1 VM 2
vPort 1vPort 0
Physical Topology Virtual Topology
Scalability and High
Border Node
Border Node
Border Node
Virtual Tenant
Router B
Virtual Provider
Router
vPort L3GW
vPort L3GW
Provider
Network BGP Peer
BGP Peer
BGP Peer
24. MidoNet 101! Face-to-Face with the Distributed SDN・FOSDEM 2015
Firewall
24
• MidoNet supports OpenStack/Neutron Security Groups
• Apply to each network port bound to a VM, inbound or outbound
• Any forward traffic not explicitly allowed by a rule is dropped
• Return traffic is allowed
VM 1 VM 2
Virtual Tenant
Router A
Virtual
Switch A1
Virtual Provider
Router
Virtual
Switch A2
vPort 1vPort 0
Port-level firewall
$ neutron security-group-rule-create --protocol tcp
--port-range-min 22 --port-range-max 22
-—direction ingress security-group-1
SG-1 Allowing SSH inbound traffic
$ neutron security-group-rule-create --protocol icmp
--direction ingress security-group-2
SG-2 Allowing ICMP inbound traffic
Chains
Rules
• Anti-spoofing
• L2 - L4 header fields
• Wildcards
• Ranges
MidoNet Models
25. CHAIN vPort0 ingress
MidoNet 101! Face-to-Face with the Distributed SDN・FOSDEM 2015
Firewall
25
VM 1 VM 2
Virtual Tenant
Router A
Virtual
Switch A1
Virtual Provider
Router
Virtual
Switch A2
vPort 1vPort 0
$ neutron security-group-rule-create --protocol tcp
--port-range-min 22 --port-range-max 22
-—direction ingress security-group-1
SG1 Allowing SSH inbound traffic
$ neutron security-group-rule-create --protocol icmp
--direction ingress security-group-2
SG2 Allowing ICMP inbound traffic
SG-1
SG-1
SG-2
DROP
if not
MAC1 AC:CA:BA:
00:00:01
MAC2 AC:CA:BA:
00:00:02
DROP
if not IP1
ACCEPT
return
JUMP
SG-1
DROP
everything
CHAIN SG-1 ingress
ACCEPT
TCP port range
26. • Different agents must exchange flow
information
• Drop not allowed packets at the ingress
host
• Protects the private underlay
MidoNet 101! Face-to-Face with the Distributed SDN・FOSDEM 2015
Network Address Translation
26
Virtual
Switch B1
VM 1 VM 2
Virtual Tenant
Router B
Virtual Provider
Router
Provider
Network
Private Network
Public Network
10.0.0.100:1234
151.16.16.1:370
Forwardflow
Returnflow
L4 NAT for a TCP connection
Private IP Network
VM 1
Border Router
Virtual Topology Physical Topology
27. MidoNet 101! Face-to-Face with the Distributed SDN・FOSDEM 2015
Distributed Flow State
27
VM 1 VM 2
Virtual
Switch B1
VM 1
VM 2
Virtual Tenant
Router B
Private Network
Public Network
Physical Topology Virtual Topology
Forward flow
Fwd outFwd in
Flow state
Return flow Ret inRet out
Ingress host
Possible return
flow ingress
Possible forward
flow ingress
Egress host
Ingress host Egress host
Forward flow
Fwd out
Fwd in
Ingress host
Possible return
flow ingress
Possible forward
flow ingress
Egress host
1
2
3
• Flow state forwarded to
possible interested hosts
• No delay for simulating flow
ingress packets at other hosts
• State backup in cluster
State Cluster
38. Distributed architecture of MidoNet
• Each compute node has MidoNet agent
• MidoNet handles L2 - L4, NAT, LB, … at the edge
• MidoNet agent has cached virtual networking
topology information and synchronises with
Network State Database (NSDB)
• MidoNet agent adds/removes flows to/from the
local Open vSwitch datapath based on simulations
of packets
39. The rise of
OpenFlow
It brought a simple and
flexible idea to decouple
control planes from data
planes. However, OpenFlow
controllers can be a SPoF.