Open vSwitch Fall Conference 2017

1. Optimizing OvS using DPDK
Membership Library
Intel Labs
Yipeng Wang & Sameh Gobriel

2. 2
Legal Disclaimers
No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.
Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a
particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in
trade.
This document contains information on products, services and/or processes in development. All information provided here is subject
to change without notice. Contact your Intel representative to obtain the latest forecast, schedule, specifications and roadmaps.
Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service
activation. Performance varies depending on system configuration. No computer system can be absolutely secure. Check with your
system manufacturer or retailer or learn more at intel.com.
© 2017 Intel Corporation. Intel, the Intel logo, Intel. Experience What’s Inside, and the Intel. Experience What’s Inside logo are
trademarks of Intel. Corporation in the U.S. and/or other countries.
*Other names and brands may be claimed as the property of others.

3. Contributors
Charlie Tai Charlie.tai@intel.com
Ren Wang ren.wang@intel.com
Antonio Fischetti antonio.fischetti@intel.com

4. OvS De Facto Virtual Switch for NFV Environments
4
Memory
C
$ LLC
$ Lx
C
$ Lx
Memory
C
$ LLC
$ Lx
C
$ Lx
NIC NIC
• General purpose processors withCache/memoryhierarchycan
support muchlargerflowtables.
• Multicores architectureprovide a scalablecompetitiveflow
classificationperformance.
TEM/OEM
Proprietary OS
ASIC, DSP,
FPGA, ASSP
• Network appliances use purpose-built H/W&
ASICs (e.g., TCAM) forflowclassification
• Cost & power consumption are limiting factors to
support large number offlows
Monolithic Purpose-built Boxes
NFV
Hypervisor (e.g. ESXi, KVM,.. etc.)
O
Open vSwitch
Networking VMs on Standard Servers

5. 5
Open vSwitch Flow Lookup
Mask N
1xxx xxxx
0xxx xxxx
Flow Mask Mask L
01xx xxxx
10xx xxxx
1110 0000
110x xxxx
101x xxxx
111x xxxx
011x xxxx
1111 0000
1010 xxxx
0011 xxxx
1011 xxxx
Rules Match
Packet Header
1. Set of disjoint sub-table withnopriority
2. Rule is only insertedintoone sub-table (lookupterminates after first match)
3. Lookup is donebysequentially searcheach sub-table untila matchis found
Fig. Vtunes OVS flow lookup process (bypass EMC). Test case: 20 sub-tables, each has 100 rules.
OvS Flow Classification is a
bottleneck

6. 6
Membership Test Usage (example)
Blacklisted Flow 1
Blacklisted Flow 2
Blacklisted Flow 3
Blacklisted Flow N
Clients
Incoming Flows
Legitimate Flows
are Forwarded to
Backend Server
Build
Set {..}
Set of Blacklisted Flows
to be Dropped
Membership Test ?
Check if Flow Belongs to
Blacklisted Set
SN
S2
S1
Set Summary
A Summary Instead Of
Storing Original ListMembership Library is a DPDK Library to Provide Users the Functionality to
Create Different Types of Set-Summaries

7. 7
Overview of DPDK Membership Library in V17.11
Set
Summary
Is X in set?
No
Is X in set?
Very
probable yes
Get X
Summary of items in Probabilistic data structure
• Handle membership test questions
• Much smaller storage
• Much faster than huge set lookup
• [Multi-Set]: Returns X is not found or which set it
belongs to (with high probability)
Huge Set
[Millions of
Entries]
1- Too Much Storage
2- Slow Lookup
Bloom Filter
Vector Bloom
Filter
Hash Table Set
Summary
Cuckoo
Distributor
Membership Library

8. 8
Two Level Lookup for MFC
Mask N
1xxx xxxx
0xxx xxxx
Flow Mask Mask L
01xx xxxx
10xx xxxx
1110 0000
110x xxxx
101x xxxx
111x xxxx
011x xxxx
1111 0000
1010 xxxx
0011 xxxx
1011 xxxx
Rules Match
Packet Header
1st
Level of
Indirection
Set Summary Provided
by Membership Library
Mask N
1xxx xxxx
0xxx xxxx
Mask L
01xx xxxx
10xx xxxx
1110 0000
110x xxxx
101x xxxx
111x xxxx
011x xxxx
1111 0000
1010 xxxx
0011 xxxx
1011 xxxx
Match
Packet Header
• Membership library used to create a 1st
level set-summary indirection
• Flow Keys are looked up in set-
summaries:
• Hits: directs to the correct sub-table for searching
(correct 97%)
• Misses: “New” flow default sequential search &
upcall if needed

9. 9
Dynamic Operation & Sub-Table Ranking
Mask N
1xxx xxxx
0xxx xxxx
Flow Mask Mask L
01xx xxxx
10xx xxxx
1110 0000
110x xxxx
101x xxxx
111x xxxx
011x xxxx
1111 0000
1010 xxxx
0011 xxxx
1011 xxxx
Rules Match
Packet Header
1st
Level of
Indirection
Set Summary Provided
by Membership Library
Number of Sub-tables
Traversed
Mask N
1xxx xxxx
0xxx xxxx
Mask L
01xx xxxx
10xx xxxx
1110 0000
110x xxxx
101x xxxx
111x xxxx
011x xxxx
1111 0000
1010 xxxx
0011 xxxx
1011 xxxx
Match
Packet Header
1
2
• Sub-table Ranking:
• Based on number of hits per sub-table à
optimize the order of sequential search.
• First level is switched ON/OFF
• If average number of sub-tables (without first
level) traversed is small à turn off

10. 10
Implementation Overview
Rte_member_lookup
Sequential search of tuples
(upcall possible)
Pkt miss emc
hit
miss
Rte_member_add()
Initilization:
Rte_member_create() to create
set-summary
Emc lookup
packets
tuple lookup miss
Hit
Return rules
New ML Code
OvS Code
Legend

11. 11
Performance Gain
2X-3X Throughput Improvement for OvS using
DPDK Membership Library
1.8
1.6
5
4.4
0
1
2
3
4
5
6
EMC OFF EMC ON
MaxForwardingRate(MPPS)
20 Sub-Table - 10k flow – Uniform Traffic
Orig OvS-DPDK OvS-DPDK + ML Library
2.7X
2.7X

12. 12
Conclusion
• MegaFlow Lookup has scalability bottleneck, especially with uniform distribution
traffic patterns.
• The membership structure optimizes flow lookup in OvS and avoids the sequential
search of the sub-tables.
• Using DPDK Membership Library, first level of indirection is created to direct flow to
the correct sub-table.
• Dynamic turning on/off to avoid overhead of first level when not needed.
• DPDK V17.11 released with Membership Library … Patch to be submitted to the
mailing list, please review and test in your workload.

13. Questions?
sameh.gobriel@intel.com
charlie.tai@intel.com