1. Interoperability Update: Dynamic Ethernet Services Via
Intelligent Optical Networks
James D. Jones, Alcatel; Lyndon Ong, Ciena; Monica Lazer, AT&T
Abstract This article describes the 2005 Worldwide Interoperability Demonstration held
by the Optical Internetworking Forum (OIF) and showcased during SUPERCOMM 2005. The
event highlighted Ethernet services transported over intelligent optical networks, using equipment
from 13 of the industry’s leading vendors located in 7 carrier lab facilities around the world. The
demonstration utilized a distributed optical control plane based on OIF Implementation
Agreements to control a multi-layer network providing Ethernet over SONET/SDH adaptation
and transport. The article describes the global test network, services, architecture and overall test
approach. It also describes innovations made to the optical control plane to handle multi-layer
signaling and lists further refinements needed to make these services operational.
1 Introduction
The Optical Internetworking Forum (OIF) conducted its second World Interoperability
Demonstration, in conjunction with SUPERCOMM held in Chicago on June 7 - 9, 2005. Member
companies demonstrated dynamic Ethernet services enabled over a global optical network,
building on the results of a similar global demonstration in 2004.
At SUPERCOMM 2004, OIF demonstrated dynamic end-to-end SONET/SDH connection
management between client devices and transport network elements from many vendors in a
multi-domain, transport network spanning multiple carrier laboratories. The 2004 event also
demonstrated Ethernet adaptation over SONET/SDH using GFP (Generic Framing Procedure),
VCAT (Virtual Concatenation) and LCAS (Link Capacity Adjustment Scheme) as a separate
objective. At SUPERCOMM 2005, the OIF took a major step by integrating these two features
and creating a multi-layer control plane control to trigger end-end Ethernet connections over a
SONET/SDH network. The result was a global network enabling clients to directly request
Ethernet services over carriers’ SONET/SDH networks. While the demonstration focused on
Ethernet Private Line service enabled by the distributed control plane, it also evaluated Ethernet
Virtual Services (Virtual Private Line, Virtual Private LAN and Internet Trunking) over the
optical transport network.
This demonstration was motivated by continued growth in demand for Ethernet services in public
networks, and the carriers’ imperative to maximize utilization of their existing SONET/SDH
transport infrastructure. To do this, interoperability is required at many levels (i.e., transport,
control and management planes) to allow flexibility as the network evolves to support present and
future Ethernet services. At the same time, carriers have heterogeneous core optical transport
networks comprised of a range of bearer technologies, infrastructure granularity options, and
survivability mechanisms. Products and systems tested for interoperability included routers,
2. multi-service provisioning platforms (MSPPs), SONET/SDH cross-connects, optical switches,
optical add-drop multiplexers (OADMs) and reconfigurable OADMs (ROADMs). OIF
Implementation Agreements and interoperability trials address the challenge by requiring control
plane solutions be developed in the context of such heterogeneous environments, and be able to
co-exist with the existing network.
The demonstration was executed on a global stage with seven Carriers across three continents
inter-networking through an intelligent control plane with equipment from thirteen vendor
participants. A network of over 70 nodes was built up in progressive stages, beginning with local
lab testing, followed by intra-continental regional testing and culminating in a live, global real-
time network test. During the global demonstration, over 20 optical connections were
simultaneously active among the test sites. This included a live video feed between two carrier
labs, which was then transported to the SUPERCOMM show site. This video connection was
dynamically setup and torn down remotely from the show floor via the optical control plane.
2 Creating a World Wide Demonstration
The OIF World Interoperability Demonstration at SUPERCOMM 2005 was built as a global
network, in that all equipment was located at carriers’ research laboratories across the world:
Asia, Europe, and North America. Equipment from multiple vendors was interconnected within
the laboratories. Virtual connections were established among carriers, with the exception of
several instances, as discussed in more detail below.
Participants included:
• Seven Carrier Lab Locations:
• Europe: Deutsche Telekom, France Telecom, Telecom Italia
• Asia: China Telecom, NTT
• North America: AT&T, Verizon
• Thirteen Vendors:
Alcatel Lucent Technologies
Avici Systems Mahi Networks
CIENA Corporation Marconi Corporation
Cisco Systems Nortel Networks
Fujitsu Sycamore Networks
Huawei Tellabs
Lambda Optical Systems
The overall equipment topology for this event is shown in Figure 1. This diagram shows all the
equipment involved in the OIF World Interoperability Demonstration, its function in each of the
carrier labs and the transport plane. As mentioned above, in addition to the virtual links
interconnecting carrier labs facilities, there were several real links that were used to carry video
application to showcase the demonstration: one OC3 link connected the AT&T and Verizon
facilities, and one DS3 link connected the AT&T facilities to the SUPERCOMM demonstration
booth. Video streams were used to illustrate the status of calls during the demonstration.
3. LEGEND Verizon
Asia to Europe Labs Sycamore/EoS
Tellabs Alcatel
America to Europe
or America to Asia
Ciena/EoS
Intra-continental
Nortel Tellabs
Client
TNE
Alcatel
Tellabs
Mahi
NTT Labs
Avici Tellabs Alcatel Fujitsu
NTT Fujitsu/EoS Alcatel
NTT
Sycamore Ciena
NTT
Lucent/EoS Cisco/EoS
Avici
OC-3 Deutsche
Telekom
Fujitsu AT&T Labs Ciena
NTT Ciena Lucent/EoS
Sycamore
Avici Avici
Marconi Cisco
Cisco/EoS Ciena/EoS
Navtel Huawei Alcatel Alcatel/EoS
China
Telecom
Ciena
Huawei Avici
Cisco Marconi
Sycamore Marconi
Avici
Lambda
Cisco Optical
Telecom
Avici Avici Italia
France
Telecom
Figure 1 Overall Topology of OIF World Interoperability Demonstration
In addition, a Signaling Control Network (SCN) was set-up in an architecture simulating
operational networks. In each carrier lab, an internal SCN connected all the equipment. The
individual carrier SCNs were interconnected simulating carrier-to-carrier interconnections.
However, since this is still a demonstration network, not an operational network, IPsec tunnels
4. were used over the public Internet (as opposed to dedicated signaling networks). This SCN was
used for signaling and routing information exchanges. For display purposes, a custom software
application intercepted signaling messages sent over the SCN to build a live view of the active
calls. This application analyzed data fields within the signaling messages to build the picture of
the active calls, as shown in the example in Figure 2. These global topology views were available
both in the lab sites and at the SUPERCOMM show floor.
Figure 2 Dynamic Display of Active Calls
A generic diagram illustrating the service and interface types included in the demonstration is
shown in Figure 3. As seen in the diagram, all client devices were connected to the optical
network via Ethernet interfaces. This illustrates the evolution of the OIF UNI [1] from supporting
SONET/SDH services to support of Ethernet services.
The 2005 demonstration was focused on dynamic Ethernet services enabled by intelligent optical
networks. Ethernet services were delivered across multiple carrier labs with various bandwidth
characteristics by Switched Connections (SCs, initiated by client UNI signaling), Soft Permanent
Connections (SPCs, initiated by a management system) and hybrids of both. To support dynamic
Ethernet calls, several key technologies were required as described below: GFP [2], VCAT [3],
LCAS [4], UNI 2.0 [5], E-NNI 1.0 [6] (with extensions), inter-layer call and connection
coordination. For these calls, the Ethernet signals were mapped to SONET/SDH payloads using
GFP/VCAT/LCAS standards. The test cases included mapping of both full and partial rate
Gigabit Ethernet signals and both co-routed and diversely routed SONET/SDH containers
carrying an Ethernet signal.
5. Ethernet Carrier A Carrier B Carrier C Ethernet
Client Domain Domain Domain Client
OIF UNI OIF E-NNI OIF E-NNI OIF UNI
NE NE NE NE NE NE
Ethernet SONET/SDH Ethernet
UNI-N UNI-N
UNI-C UNI-C
Ethernet Layer Call/Connection Flow
Control Plane SONET/SDH Layer Call/Connection Flow
View
GigE Virtual Concatenation Group (21 STS-1 or 7 VC-4) GigE
GFP-F GFP-F
. . .
VCAT . . . VCAT
Transport Plane . . .
LCAS LCAS
View
Figure 3 Multi-Layer Control Plane and Transport plane Architecture
In addition, the control plane operated at multiple layers and inter-layer coordination was
supported in the optical network elements at the edges of the networks. For a Switched
Connection, the client device control plane interfaced to the network control plane via a UNI 2.0
interface supporting Ethernet transport. Using UNI 2.0, there is no need for the customer
equipment to have any awareness of how the network implements support for Ethernet services
(whether Ethernet mapped to SONET/SDH, or native Ethernet, or Ethernet mapped directly to
optical wavelengths). In this demonstration SONET/SDH transport was used for Layer 1
transport. The edge NE performed the appropriate mapping of the Ethernet service to SONET
payload (consistent with the service parameters requested in the signaling messages) and
originated signaling at both Layer 1 and Layer 2 in support of the request. The reverse process
took place at the egress point of the network.
To illustrate the utility of the optical control plane a video application was set-up between the
AT&T and Verizon labs as follows (see Figure 4):
• A video server was connected to a router (client to the intelligent optical network) at each
site. The server was used to transmit video streaming between sites when the Ethernet
call was when available.
• Whenever the inter-site Ethernet call was established, each lab could view both the video
that originated from their lab and the video from the remote lab.
• Whenever the Ethernet call was deleted each site could only view video originated in its
own lab.
• During the SUPERCOMM show, an additional DS3 link connected the AT&T labs
facilities and the SUPERCOMM booth. The AT&T video streaming was available
continuously at the SUPERCOMM booth. The Verizon video stream was available
during the time that the Ethernet call between the routers at the two sites was established.
The OIF booth demonstrations started with the Ethernet call established and both videos
were visible in the booth. The Ethernet call between the two sites was then deleted and
the visitors could see the change in the call map topology on one monitor (similar to
6. Figure 2), the video control stream from AT&T on a second monitor, and the interrupted
Verizon video stream on a third monitor. Next, the Ethernet call was re-established and
the call map topology reflected the change. At the same time, the visitors could see the
Verizon video streaming again in real time.
OIF Booth AT&T Labs Verizon Labs
(Chicago, IL) (Middletown, NJ) (Waltham, MA)
ATT Video VZ Video
Server Server
DS3 Private Line
Avici Tellabs
TSR 8860
UNI 2.0 UNI 2.0
Control plane
GigE GigE
Static Segments of
Video Path
Dynamic Segments of
Video Path Enabled Ciena Alcatel
by Control Plane CD 1677
E-NNI OC3 AT&T-VZ
Figure 4 Video Application Configuration
3 Background
OIF subscribes to the ITU-T ASON architecture, as discussed in [7], and has based its
Implementation Agreements for the optical UNI and the optical E-NNI on the ITU-T ASON
Recommendations, especially ITU-T G.7713.2 [8] for RSVP-based signaling. The optical UNI
[1] enables clients of optical networks to dynamically request connections without knowing
network internal topology, while the E-NNI [6] automates the establishment of these connections
between optical networks. Together, UNI and E-NNI permit dynamic A-to-Z provisioning of
services across an optical network in real time without manual intervention, resulting in faster and
more efficient operation than traditional optical networks. Link state routing protocol based on
OSPF-TE based on [9] is used for automated network topology distribution and link status
updates inside the network.
3.1 Multi-Layer Networking for Ethernet-over-SDH
This year’s testing focused on multi-layer networking, where connections in a client layer are
supported by the dynamic establishment of connections in a server layer. In practice, carrier
networks consist of multiple technology layers, ranging from Layer 3 IP connectionless packet
transport down to Layer 0 physical connectivity, such as fiber cross-connection. Often new
services arise at one layer and must be transported efficiently using a core lower layer network.
7. One current example of this is the support of Ethernet services, which are growing rapidly as a
carrier service offering, and must be transported efficiently over the carriers’ core optical
transport networks. A number of technologies have been developed in the transport plane to
improve the efficiency and flexibility of SONET/SDH for packet/frame transport, including GFP,
VCAT and LCAS.
Testing in 2005 focused on the use of the optical control plane to control connections in the
optical core network to support Ethernet layer services. As shown in Figure 5, connections
between client devices at the Ethernet layer were supported by dynamically established
connections at the optical (SONET/SDH) layer. The number and type of connections at the
SONET/SDH layer corresponded to the amount of bandwidth requested at the Ethernet layer, and
an adaptation function using GFP, VCAT and LCAS was used at the originating and terminating
points to encapsulate the Ethernet frames into SONET/SDH paths.
To create a connection, the Ethernet UNI Client (UNI-C) sends an Ethernet connection request to
the Ethernet UNI Network-side switch (UNI-N). The UNI-N is then responsible for determining
the corresponding SONET/SDH requirements, creating the required SONET/SDH connections,
and then signaling to the remote UNI-N that the underlying connections are available and are to
be used for an Ethernet client connection. The additional stage of signaling between the source
and destination UNI-Ns carries the actual Ethernet layer connection requirements, allowing the
mapping at the destination UNI-N from SONET/SDH back to the Ethernet service. Both UNI-Ns
then apply Ethernet-SONET/SDH adaptation using GFP. This whole process involves multiple
stages of signaling to coordinate events at different layers, making the control plane processing
significantly more complex than previous years’ demonstrations.
8. OIF
UNI 2.0
OIF
UNI 2.0
OIF
UNI 2.0
OIF
E-NNI
I-NNI Domain
OIF
I-NNI Domain OIF OIF
E-NNI
E-NNI E-NNI
I-NNI Domain
Physical Link SONET/SDH Layer Ethernet Layer
Gigabit Ethernet STS-3c/VC-4 Connection 350 Mbps Connection
OC48/STM16 STS-3c/VC-4 Connection 250 Mbps Connection
STS-1 Connection 100 Mbps Connection
Figure 5 Multi-layer Intelligent Optical Transport Network
The key extension to the protocols was the ability to dynamically trigger the creation of the
supporting server layer connection upon detecting that new optical capacity was needed. When
the ingress optical switch received a UNI 2.0 call request (i.e., for an Ethernet connection), it
initiates the process for creation of new optical connections, computes the required path across
the optical core and creates the connection, which is then used to carry GFP-encapsulated
Ethernet frames.
While there is a one-to-one relationship between Ethernet connection and SONET/SDH
connection (or VCAT group) for Ethernet Private Line services, as were tested, future Ethernet
Virtual Private Line and Private LAN services can allow the same SONET/SDH connection or
group to be used for multiple Ethernet services. Standards are not complete for how multiplexing
would be supported, but candidate mechanisms are the use of the client’s VLAN tag (if present),
application of a carrier VLAN tag at the UNI-N, or other tags. This is an active area of
discussion in IEEE, ITU-T and IETF. With Ethernet virtual services, it will be possible to reuse
an existing SONET/SDH pipe for future Ethernet connections as long as bandwidth is available.
The network will be able to respond dynamically to new demands, either creating a new optical
connection or reusing existing optical connections as needed.
3.2 Signaling Extensions for Multi-Layer Networking
Some of the more interesting problems that needed to be solved for multi-layer networking
included the following:
9. • How to correlate signaling at client and server layers: Since connections were
established first at the server layer (SDH) and then at the client layer (Ethernet), there
needed to be a way to correlate signaling at multiple layers, so that, for example, the edge
switches correctly identified which SDH timeslots were to be used, and were able to
exchange signaling directly between them. For the testing, a mechanism called LSP
(label switched paths) Hierarchy [10] was used, which involves the addition of fields in
the RSVP signaling to identify the signaling addresses of the edge switches and the
creation of virtual interfaces corresponding to the server layer connections
.
• How to translate the Ethernet bandwidth request into the required SDH components:
Since Ethernet bandwidth is expressed in terms of bits per second required and burst rates
supported, while SDH bandwidth is expressed in terms of the size of the signal (STS-1,
VC-4, VC-4-4v), there needed to be a mapping from the Ethernet bandwidth request over
the UNI to the SDH bandwidth requirement at the E-NNI. In practice, such a translation
would be a matter of policy determined by the service provider, since it is affected by the
guarantees offered by the service provider as well as their core infrastructure. For the
Demo, a mapping table was used to unambiguously map one layer to the other.
• Routing across multiple layers: In theory, the addressing used for clients and network
elements at one layer may be independent of that used at another, so that the routing of
the connection may involve translation from addresses in the client layer to addresses
used in the server layer. For example, the destination (or in OIF the TNA (Transport
Network Assigned) address) for the Ethernet client must be translated to some associated
endpoint in the SDH network for SDH path computation. For the Demo, a one-to-one
correspondence was assumed, where in real networks a more complex translation may be
required.
3.3 Control Plane Support of Virtual Concatenation
VCAT is an inverse multiplexing capability defined in ITU-T [3] that allows multiple
SONET/SDH channels to be bound into a single higher rate VCAT group (VCG). For the
demonstration, separate connections were set up for each component of the group, in order
to create higher survivability for the group as a whole. LCAS allows failure of individual
connections to be treated as reduced bandwidth in the group without actually causing
failure of the entire group.
A VCAT group consisting of multiple connections in the server layer was created using
multiple call setups, therefore allowing each connection to follow a different path based on
its individual path computation. An example of call setup for VCAT is shown in Figure 6.
A coordination mechanism was supported to synchronize the establishment of the
supporting VCAT connections and the client layer call. Both parallel and sequential
strategies of setting up VCAT connections were considered.
10. Ethernet Client Layer Connection
E-NNI
VCAT Component Connections
UNI 2.0 UNI 2.0
E-NNI E-NNI
Figure 6 Setup of diverse VCAT connections
3.4 Additional Transport plane Testing
Additional transport plane-only Ethernet testing was done, based on Ethernet service
specifications developed in ITU-T and the Metro Ethernet Forum (MEF). These tests
demonstrated interoperability in the transport plane for Ethernet Virtual Services, where multiple
Ethernet services were transported using the same SONET/SDH VCGs. The virtual services
demonstrated included: Ethernet Virtual Private Line, Ethernet Virtual Private LAN and Internet
Access/Virtual Trunk. These provide a complementary aspect to the control plane testing, which
focused on Ethernet Private Line.
For Ethernet Virtual Private Line service, for example, individual client flows were tagged at the
UNI-N, aggregated into a single transport link and separated at the destination based on the
values of the VLAN tags. VLAN tags as defined in IEEE 802.1Q were used to identify an
individual service. Testing of Ethernet Virtual services was based on work being done in ITU-T
and MEF, especially ITU-T Recommendations G.8011.1 and .2 [11], and MEF 6 [12].
3.5 Future Work
Findings from the interoperability testing have been compiled and provided as input to the
various standards bodies active in optical control plane specification, to identify any areas of
potential confusion or omission in the standards. Future testing work may be aimed at more
complex services and topologies such as dynamic control of Ethernet virtual services, as these are
incorporated into optical control plane standards work.
4 Conclusions
The 2005 OIF demonstration was the first time Ethernet adaptation and distributed optical control
planes were brought together in an integrated fashion, and it was done on a global scale. The
call/connection control of the UNI-N devices was the most important technical innovation
demonstrated, in two respects. First, the UNI-N provides inter-layer control plane coordination as
the client signal enters the network. The UNI-N is responsible for accepting the client connection
request, initiating calls in the server layer, and completing the client layer call once the server
layer is set up. Second, the UNI-N triggers and controls the Ethernet adaptation function and
mapping of client layer signals into server layer containers. This architecture minimizes the
overall network impact since the client Ethernet devices and core SONET/SDH devices are only
concerned with a single layer.
11. The OIF World Interoperability Demonstration is an essential step in the evolution process of the
optical control plane, helping to make it suitable for deployment in carrier networks. The testing
demonstrated multi-vendor support of a distributed optical control plane, its ability to control
multi-layer end-to-end services and the overall commitment to the technology by both vendors
and carriers. The demonstration utilized real network elements from vendors whose market
presence accounts for 64% of the 2004 worldwide revenue in the optical networking switching
and routing markets (source: RHK). The equipment was hosted in technology evaluation labs of
top-tier carriers in North America, Europe and Asia.
While this event provided solutions for a number of technical issues, it also revealed others that
need to be addressed. The knowledge gained from this interoperability demonstration is being
applied to the OIF UNI 2.0 and E-NNI 2.0 signaling specifications. The experience also benefits
carriers in planning migrations to distributed optical control planes and anticipating the
operational considerations for multi-vendor networks.
The authors would like to acknowledge all the people in the carrier labs for the tremendous
efforts in putting the demonstration together and shepherding it through its stages, the staff from
the participating vendors for the relentless work in accomplishing the interoperability, and the
support of the OIF leadership in getting it all together.
5 References
1. OIF-UNI 1.0 Release 2, “OIF-UNI-01.0-R2-Common - User Network Interface (UNI) 1.0
Signaling Specification, Release 2: Common Part” and “OIF-UNI-01.0-R2-RSVP - RSVP
Extensions for User Network Interface (UNI) 1.0 Signaling, Release 2”.
2. ITU-T G.7041, “Generic Framing Procedure (GFP)”.
3. ITU-T G.707, “Network Node Interface for the Synchronous Digital Hierarchy (SDH)”.
4. ITU-T G.7042, “Link Capacity Adjustment Scheme (LCAS)”.
5. OIF UNI 2.0 Signaling, “”, oif2003.293.
6. OIF E-NNI 1.0 Signaling, “OIF-E-NNI-Sig-01.0 - Intra-Carrier E-NNI Signaling
Specification”.
7. ITU-T G.8080/Y.1304, “Architecture for the Automatically Switched Optical Network
(ASON)”.
8. ITU-T G.7713.2, “Distributed Call and Connection Management: Signalling mechanism
using GMPLS RSVP-TE”.
9. ITU-T G.7715.1, “ASON Routing Architecture and Requirements for Link State Protocols”.
10. draft-ietf-mpls-lsp-hierarchy-08.txt, “LSP Hierarchy with Generalized MPLS TE”
11. ITU-T G.8011/Y1307, “Ethernet over Transport – Ethernet services framework”
ITU-T G.8011.1/Y1307.1, “Ethernet private line service”
ITU-T G.8011.2/Y1307.2, “Ethernet Virtual Private Line Service”
12. MEF 6, “Ethernet Services Definitions - Phase I”.