Presentation by Richard Bayliss and Shishio Tsuchiya at APRICOT 2018 on Monday, 26 February 2018.

1. Copyright © Arista 2018. All rights reserved.Copyright © Arista 2018. All rights reserved.1
Data Plane Evolution:
Towards Openness
and Flexibility
Rich Bayliss bayliss@arista.com
Shishio Tsuchiya shtsuchi@arista.com

2. Copyright © Arista 2018. All rights reserved.
Agenda
• Data Plane Today (Recap)
• IETF NVO3 Encapsulation Consideration
• Future Data Plane Implementations
2

3. Copyright © Arista 2018. All rights reserved.
2010/11 – New Solutions For DC Overlay (Recap)
Technology VXLAN NVGRE STT OTV
Intend to
on 1st draft
Experimental
Aug 2011
Informational
Sep 2011
Informational
Feb 2012
Standard
Apr 2010
Status
RFC7348
(Information)
RFC7637
(Information)
Expired Expired
IPR NA Microsoft Nicira Cisco
Implementation
Linux 3.7
OVS
Hyper-V
DPDK
Broadcom Trident2
Hyper-V
DPDK
Broadcom Trident2
OVS
Hyper-V
Cisco NX
Transport UDP GRE TCP Like UDP
Load balance depends on intermediate node
need calculate include Key Field
3
2018: VXLAN Is The Defacto Standard For Modern L3 Leaf/Spine Cloud

4. Copyright © Arista 2018. All rights reserved.
Agenda
• Data Plane Today (Recap)
• IETF NVO3 Encapsulation Consideration
• Future Data Plane Implementations
4

5. Copyright © Arista 2018. All rights reserved.
IETF NVO3 (Network Virtualization Overlays) WG
• Working Group Charter
- NVO3 WG is to develop a set of protocols and/or protocol extensions that enable network
virtualization within a data center (DC) environment that assumes an IP-based underlay.
- NVO3 solution provides layer 2 and/or layer 3 services for virtual networks enabling multi-
tenancy and workload mobility.
• IETF NVO3 goal is to define a standard encapsulation, Today there are three
candidate protocols:
- Geneve: Generic Network Virtualization Encapsulation
- GUE: Generic UDP Encapsulation
- VXLAN-GPE: Generic Protocol Extension for VXLAN
• All of them have hardware implementation and backward compatibility issues
that are worth exploring and understanding.
5
https://datatracker.ietf.org/wg/nvo3/about/

6. Copyright © Arista 2018. All rights reserved.
NVO3 Encapsulation Considerations
https://tools.ietf.org/html/draft-ietf-nvo3-encap
• NVO3 initiated a design team to compare and discuss current proposal encapsulation
• Geneve seen as most suitable as a starting point for proposed standard for network virtualization - DT
initial recommendation - because the flexibility of TLV, ease of implementation by total option length, and
VNI is included in the header.
6
Technology Geneve GUE VXLAN-GPE
Transport IP/UDP IP/UDP IP/UDP
Base Header
Length
8 bytes 4 bytes
8 bytes
(16byte combine with NSH)
Extensibility
Variable Length
Options
Extensions Fields
VXLAN-GPE is not inherently
extensible. Need to combine with NSH
Maximum Header
length
260 bytes 128 bytes
8 bytes
(264 byte NSH)
Next Protocol
Identifier
Ether type Protocol ID New Registry

7. Copyright © Arista 2018. All rights reserved.
Status of NVO3 Encapsulations & Pain-Points
• Variable Header Length: introduces new challenges – and impacts traditional fixed length pipeline processing.
• Parallel Processing Requirements: Not applicable to all applications/use cases.
• Diverging Requirements: Who will be mainstream in next data center?
7
Technology Geneve GUE VXLAN-GPE
Author
Intel
VMware
Facebook
Huawei
Microsoft
Cisco
Intel
IPR VMware
(Royalty-Free)
Cisco N/A
Maximum Header length 260 bytes 128 bytes
8 bytes
(264 bytes with NSH)
Next Protocol
Identifier
Ether type Protocol ID New Registry
Author
Intel
VMware
Facebook
Huawei
Microsoft
Cisco
Intel
Software Implementation
3.18
OVS
DPDK
Hyper-V
3.18
4.6
OVS
DPDK
Ethernet
Header
Outer IP
Header
NVO3
Header
Original
Packets
Extension
Header
Extension
Header

8. Copyright © Arista 2018. All rights reserved.
Extension Headers
8

9. Copyright © Arista 2018. All rights reserved.
What’s About MPLS/SR
• Are any of the NVO3 encapsulations
the ‘final’ solution?
• Lots of interest in moving towards
SRv6 too!!
- https://tools.ietf.org/html/draft-ietf-dmm-srv6-mobile-uplane
- https://portal.3gpp.org/ngppapp/CreateTdoc.aspx?mode=view&
contributionId=835814
9
Ethernet
Header
Outer IP
Header
NVO3 Header
Original
Packets
Extension
Header
Extension
Header
Ethernet
Header
IPv6
SRH
Original
Packets
SRL1
SRL2
NVO3 SRv6

10. Copyright © Arista 2018. All rights reserved.
Where Are We Today?
Connecting VXLAN Pods Has Created New Complex Possibilities
L3 EVPN MPLS DCI
Simplification Q: Will VXLAN Enter The WAN Or Will MPLS Enter The DC?
EVPN VXLAN over L3
IP / MPLS
SP NFV
DC 2
(AZ 2)
DC 1
MEC
DC 2
(AZ 1)

11. Copyright © Arista 2018. All rights reserved.
Agenda
• Data Plane Today (Recap)
• IETF NVO3 Encapsulation Consideration
• Future Data Plane Implementations
11

12. Copyright © Arista 2018. All rights reserved.
Why New Data Plane Encapsulations Are Tricky
• Work continues to develop new data-plane encapsulations – success is not assured.
• New encapsulations have traditionally required:
- Green-field deployment: Day One support for new encapsulation (best case)
- Tunneling: Additional overhead between aware nodes – greater traffic engineering complexity
- Forklift: Replace existing hardware (worst case)
• Emerging field upgradable ‘programmable’ silicon *could* make future transitions
simpler. Especially as merchant silicon grows in scale and capability (eg Jericho).
• However data-plane support also requires support from inline GSLB, Firewalls, DPI,
IDS, NAT Gateways, Content Caches, Network Packet Brokers, DDOS mitigation, etc.
- Case Study: IETF SFC – i.e Metadata that includes data-plane context information - NSH/MPLS
12
https://datatracker.ietf.org/doc/rfc7665/?include_text=1
https://docs.openstack.org/networking-sfc/latest/contributor/ietf_sfc_encapsulation.html
https://tools.ietf.org/html/draft-rijsman-sfc-metadata-considerations-00

13. Copyright © Arista 2018. All rights reserved.
2017 - Field Upgradable ‘Programmable’ Silicon:
Simplified Deployment of Future Encapsulations
• Field upgradable chipset
• Parallel processing each of header
• Open API which can programmable to XPA OpenXPS
13
http://www.cavium.com/XPliant-Ethernet-Switch-Product-Family.html
VXLAN
RFC
T2, TH etc
2011 2013
Typical 24 month gap
New
Protocol
XP80
Faster release by SW upgrade
2017

14. Copyright © Arista 2018. All rights reserved.
2018: Programmable Silicon Goes Mainstream with Trident 3
• Trident3 has compatibility with Trident2/Trident2+ functions and it is field upgradable silicon
• Supports new protocol like GENEVE, NSH, VXLAN, GPE, MPLS, MPLS over GRE/UDP, GUE, ILA ,PPPoE and so on
• XGS family providing OpenNSL (Open Network Switch Library)
14
https://www.broadcom.com/blog/new-trident-3-switch-delivers-smarter-programmability-for-enterp
Broadcom BCM56870 SERIES
High-Capacity StrataXGS® Trident 3

15. Copyright © Arista 2018. All rights reserved.
Decoupling Silicon From Pipeline Architecture
Field Upgradable, Faster To Market, User Configurable
P4 - Programming Protocol-Independent Packet
Processors
• Target Independent: CPU/FPGA/SOC/NPU/ASIC
• Protocol Independent: Specify how to processes packets.
• Accelerate Programmability with Reference Architectures
15
https://barefootnetworks.com/technology/
Barefoot Tofino
• 6.5Tbps Tofino Switching ASIC
• Protocol Independent Switch Architecture - Supports P4
• Field Reconfigurable Packet Processing
• Capilano SDE (Software Development Environment)
https://p4.org/

16. Copyright © Arista 2018. All rights reserved.
P4 Has Broader Architectural Implications
• Arista EOS already supported 4 silicon family by single binary image with programmability by SDK or Open API – This
helps validate the P4 approach.
• Similarly P4 could provide flexibility with a target independent programmable language supported by multiple network
silicon SDKs.
• Note, support for P4 Runtime Integration (API) is not equivalent to support for new data plane encapsulation.
• Juniper Networks has similarly announced P4 runtime support - http://juni.pr/2oxRElq
16
Fulcrum
-FM
BRCM-
DNX
BRCM-
XGS
CAVIUM
-XPA
A B C
Barefoot
Tofino
P4.org
NOS
Arista Dev Dev Community
P4 Potential Improvements:
• Broader Feature Development
• Faster Time To Market
• Wider Feature Deployment

17. Copyright © Arista 2018. All rights reserved.
What’s Happening Outside the DC?
Convergence of Cloud and
Transport Domains Architectures:
- Reduced layers of devices
- Reduced layers of protocols and
extension headers
- Simplified operational models
- Deployment of cheaper, merchant
silicon based infrastructure.
≫ Whitebox, P4 etc.
- Assists physical / virtual internetworking
17
This Is A Broad Topic – Worth It’s Own Session Another Day

18. Copyright © Arista 2018. All rights reserved.
Conclusion
• Future extensibility - like header chaining – starts with the infrastructure.
Each silicon vendor needs to release programmable pipeline and field
upgradable chips. Older equipment is no longer deployable.
• P4 defines protocol independent & target independent programmable
language. It can be useful for a variety of use cases/requirements as
merchant silicon continues to push into the Core and Edge of carrier
networks.
• Openness and flexibility are essential design goals. However without
careful consideration, the path leads to deeply complex architectures
that limit the full range of technology solutions available today.
18

19. Copyright © Arista 2018. All rights reserved.Copyright © Arista 2018. All rights reserved.
www.arista.com
Thank You