Three years of OFELIA – taking stock by Hagen Woesner OFELIA project coordinator

1. Three years of OFELIA –
taking stock
by Hagen Woesner
OFELIA project coordinator

2. Travelling back in time…

3. Taking stock
• OFELIA proposal was written in late 2009
– OpenFlow was at v0.9, way before anything like ONF
– Rob Sherwood and Srini Seetharaman were employees of DT
Labs
– Guru and Nick came over to Berlin quite frequently, talking
about their campus testbed in Stanford
– Some people of DT wanted to bring over this great idea to
Europe

4. I suggested calling the project OFELIA because of
the nice connotation to an open flow…

5. As a baby name, OFELIA is not very popular

6. Ophelia neither

7. Why was OpenFlow embraced by carriers?
• The world of network equipment vendors was
unchanged since ~1990
– Some consolidations in Europe, two new players from China
• The market of carriers was in full crisis
– Google
– Facebook
– Over the top players with no commercial connection to the
network owners

8. If your revenue falls 22% per annum, would
you call this a crisis?
Relative scale
5
Enormous complexity.
Traffic
Traffic *Cost
Traffic *Revenue
Rev/Cost
Router Cost per Mbit
Revenue per Mbit
4
3
2
1
0
0
1
2 Year
3
4
 Internet traffic growth at about 50% per year
 Cost reduction of router per Mbit/s 15 % per year
 Revenue per Mbit/s falls steeper (22 % per year)
(the
effect)
5
RFCs published annually Total number of RFCs
The gap of traffic, cost and revenue ...
6000
5000
4000
3000
2000
1000
0
1965
1970
1975
1980
1985
1990
1995
2000
2005
2010
500
400
300
200
100
0
1970
1980
1990
2000
2010 Year
 Huge and confusing amount of standards and RFC
 A lot of patches for Security, QoS, MPLS,
Demarcation etc.
(the
effect)

9. No wonder carriers were looking for ways out
of the dilemma to:
• Participate in revenues from OTT
– charging for netflix, youtube, spotify?
– If the customer pays a flat rate, revenue has to come from
service provider
• Make the network drastically simpler
– Split control from data planes
– Abandon all vendor-specific solutions
– Remove as much as possible session state from network
devices

10. Some details on “complexity” for the fixed access network
Upstream Interface
• IS-IS Routing
• BGP
• MPLS LDP
• Static Routing
RADIUS Interface
• LineID = User
• FilterClass by ID
• CoA for Filter Change
• All line Attributes included
Astra
UpStream
Routing
AAA
BNG Functionality
• S-Tags in Session Context
• RemoteID in Session Contex
• CircuitID in Session Context
• Static Filter Per Product
• Filter assigned by RADIUS
• Dynamic Change by CoA
• Six QoS Classes
• Bandwith control per Session
• Multicast Replication on BNG
RADIUS Client
Ethernet Port
BNG
Ethernet Port#1
S-TAG
Ethernet Port#1
S-TAG
S-TAG
-TAG
S-TAG
S-TAG
-TAG
S-TAG
VLAN#X PPPoE
BIP VLAN#Y
S-TAG
S-TAG
Ethernet Port
MSAN Functionality
• Port becomes S-TAG
• Pysical Port in Circuit-Id
MSANId+Port=Circuit-Id
• Provisioned Remote-Id per Port
Based on ITU-T M.1400
• PPPoE IA includes Ids
• C-Tags Transparent
Ethernet Port
MSAN#1
One S-TAG per Access Port
MSAN#2
Port#12
Port#12
VDSL
Port#12
VDSL
VDSL
VLAN#X PPPoE
Port#12
VLAN#X PPPoE
VDSL
BIP VLAN#Y
VLAN#X PPPoE
VDSL
VDSL
VDSL
VDSL
RG#1
RG#2
RG#1
RG#2
IAD Functionlity
• PPPoE Discovery on VLAN7
• Residential on VLAN7 flat
• BIP on VLAN #Y
10

11. So, two projects were initiated
• Split Architecture for Carrier networks (SPARC)
– DT and Ericsson
– iMinds, Acreo, EICT
• MPLS-controlled access
• BRAS
• Virtualization of networks
• And OFELIA, for trying out the things developed

12. OFELIA 2010-2013 FP7 project

13. SPARC+OFELIA at ONS 2011

14. Requirement
OFELIA
Resource discovery
Seeing aggregates in GUI, later on: topology
Resource reservation
Aggregate managers for VMs, Flowspaces, WiFi
Resource provisioning
Flowspace reservation manually approved by
island managers
Experiment control
WebGUI (expedient)
Monitoring
For island admins: zenoss, experimenters: web site
Permanent storage
n/a, but flowspace reservation remains in place for
30 days by default
Identity management
Currently: LDAP, coming up: O’House
Authorization
Open to the public. Closed and trusted group,
access granted once alongside with OpenVPN
certificate, to be improved with OHouse
SLA management
OHouse
First Level Support
Zenoss data on web site
Dataplane
interconnection
Currently best effort L2 tunnels across the Internet
VLAN stitching with lots of manual intervention
14

15. VLAN stitching
berli
n
brist
ol
ETH
Z
iMinds
• Researcher requests VLAN
space in each island
• Resolve conflicts later
• MANUALLY!
– Poor Bart.
i2cat
Trent
o/
Creat
e-Net
• “clashing engine” is being
implemented

16. Ways to overcome VLAN clashing
• Centralized assignment of VIDs
– As of now, we have a range for inter-island ViDs assigned
• VID translation
• Different slicing scheme

17. Centralized assignment of VIDs
• Centralized VLAN assignment
– For cross-island experiments VLANs to be assigned via
separate AM
GUI/omni
FOAM
VLAN AM
FOAM
FOAM
FOAM
FOAM

18. Centralized assignment of VIDs
• Wait until someone finds it too complicated and
cleans up
• centralized FOAM
GUI/omni
FOAM
VLAN
AM+FOAM
FOAM
FOAM
FOAM
FOAM

19. VLAN translation
• Slightly unusual practice for VLANs
– Not working in standard Ethernet switches
• Starts to look like MPLS (to me?)
– There are good papers on using MPLS for OF virtualisation
• In principle possible via software OF switches
– Need to create (and signal) the mappings

20. A different slicing mechanism?
Many were increasingly of the opinion that they'd all made a big
mistake in coming down from the trees in the first place. And some
said that even the trees had been a bad move, and that no one
should ever have left the oceans.
• VLANs seemed like a good idea because of aggregation, but
why did we want to aggregate?
– Because TCAM space is the scarce resource
• So why do we slice flowspace and not TCAM space?
• We (in OFELIA) have MAC address assignment under our
control, so we could slice on MACs?
– The FlowVisor explosion may be mitigated by clever setting of entries
• What about a combination of TCAM limitation and MAC
slicing?

21. Three ways of virtualizing
• Slicing the flowspace
– FlowVisor does this, partially
• Translating the flowspace
– FlowVisor 2.0 goes this way
– Process/modify each packet (incl. TTL!)
• Transparent tagging
– MPLS etc. (Pontus Sköldström and Wolfgang John: “Implementation and
Evaluation of a carrier-grade OpenFlow Virtualization Scheme”; EWSDN 2013)
This is the fundamental testbed question:
How much of the flowspace are we assigning for
experimentation, and how much for the experiment?

22. Without deciding on virtualization schemes,
what can we do?
• FlowVisor upgrade beyond OF 1.0
• OF datapath upgrade to new OF versions
• Signaling between islands (be it so VIDs)

23. FP7 project ALIEN
• Integration of FV functionality into OpenFlow
datapath
– Add flowspace management into the switch (mgmt interface
like fvctl)
– Add flowspace management into OpenFlow, eventually
• Deploy OpenFlow beyond Linux/x86
– NPUs (Cavium, Ezchip), NetFPGA, GEPON, optics
• Check https://www.codebasin.net/redmine/projects/xdpd/wiki

25. FP7 project FELIX
Framework of EU-Japan collaboration
•

26. FP7 project FELIX
For EICT, this will eventually mean writing an AMSoil-based AM for NSI v.2

27. Intermediate conclusions
• There has to be a better solution than manual
assignment of VLAN IDs
• We should either upgrade FV or get rid of it
• Porting OpenFlow to alien hardware leads to
•  Open Source Hardware?
– Should there be an endorsement/requirement to use open
hardware for testbeds?
• NSI v2 seems to be the means of choice to bridge
networks between OF controlled islands.
– For high-bandwidth, on-demand applications
– Otherwise OpenVPN or manual (QinQ) SID assignment is just
fine.

28. What is the best collaborative model?
• One that is building on the egoistic interest of a
testbed user
– Only users can operate and develop testbeds
• Federation is key
– Increase geographic reach
– Draw on other researchers’ knowledge
• Three things required for federation
– Powerful resource description (language)  Rspecs
– Connectivity and resource provisioning  Aggregate
Managers
– Trust (authentication, authorisation)  Clearinghouse

29. Experiments and what we learned from them
Thanks to Panos and Matt of Lancaster
University for writing up their experience.

30. One experiment: virtual topology creation by
modified FlowVisors
Implemented mainly be Create-Net, Italy

31. Lancaster University joined the OFELIA project
as an experimenter.
To this effect they provided feedback at the beginning
of the 3rd year of the project, describing the usability of
the facility.
Experiment is on:
Video-on-Demand Content Caching
with OpenFlow

32. Video-on-Demand Content Caching with OpenFlow (1)
An OpenFlow network with peripheral content caches

33. Video-on-Demand Content Caching with OpenFlow (2)
First interaction: Content silently copied to cache

34. Video-on-Demand Content Caching with OpenFlow (3)
Later interactions: Content retrieved from cache

35. Proposed Hardware Platform

36. Proposed Software Platform

37. Now this is a workflow to really get lost, right?

38. “Registering, creating a project, adding a slice to it and the
respective Aggregate Managers (AMs) all work quite well.”
Uff…

39. However,

40. Remove the manual approval
delay from the process
• ...by approving everything automatically.
– The respective island managers should still get the
notification emails, but the default action from the system
should be to automatically approve a project (and/or an
associated action).
– This would make the facility appear way more responsive
and user-friendly. After all, when a user is creating a project,
he wants to start experimenting with the facility straight
away.

41. Where are all the OFELIA islands?
• As a more generic comment, we advertise having 10
islands on the OFELIA testbed but there are only 4
islands available for federated tests (through their
AMs); ETHZ, i2CAT, iMinds and Create-Net. Perhaps
adding the rest of the AMs would provide a more
enhanced facility.
• OK, one is providing strange resources like optical switches (AM for
this still under development), one had a hard disk crash, and others
are just on their way to really becoming available (joined recently)

42. More feedback to the experimenter!
• The way of booking resources for carrying out
federated experiments is not very clear (despite the
tutorial). In our opinion the user requires more
feedback as what he should do next (i.e. the next
step) and what the system tries to do each time as a
response to a user’s request. A few more concrete
suggestions :

43. Flowspaces and VLANs are unclear
• The concept of Flowspaces is not very clear to the user (i.e. what is it and
what he should do “with it”).
• Related to the above, the concept of VLANs is also not obvious to the user.
Obviously this is how experiments in OFELIA are defined, but it might be
worthwhile making it clear to the user that the same VLAN is needed for
tests across multiple islands (we expect him to figure it out).
– Furthermore, if certain VLANs are not available to the user (reserved in
certain islands) then Expedient should disallow these from being
requested by a user.
–Suggestion : Perhaps a drop down box of the available VLANs or an
informative table about the availability of VLANs would help.

44. And then some smallprint
1)Also, it is not very clear when the user should update his slice. Is updating the slice done
automatically every time we change the resources? Should the user update the slice when
he restarts his controller? (we’ve found that sometimes this is required).
a.Suggestion : perhaps more information regarding when the user should update this
slice in relation to the changes he does to his slice is required.
2)Some error messages are too generic and unhelpful to the user. For example, the error
messages coming from a FlowVisor do not report which FlowVisor is generating the
messages. The user does not know how to fix this problem (he doesn’t have access to any
FlowVisors) and he doesn’t even have enough information as to whom (i.e. which island
manager) he should contact regarding these messages[1].
a.Suggestion : Perhaps making the error messages more specific to the actual
problem. Also contact details of island managers or an alias email should be displayed
to the user so that he knows whom to contact if there is a particular error message.
3)Minor : The Topology of the slice is useful, but if the user has chosen a subset of it for his
experiments, it would be useful to see active just the subset in the main page of his slice.
a.Suggestion : Perhaps getting the links/machines/switches used in a Flowspace as
bold, or a drop down menu that would give the option of toggling between “Available
Topology” and “Used Topology” would be useful.
4)Minor : When the user hovers over a switch it would be useful to see information about
the links to VMs as well (in addition to the ones we see now for the connected switches).

45. And then some more smallprint
Minor : Sometimes when a small number or resources is allocated to a slice, the actual icons do not
appear in the Topology panel and they are “floating” around.
•Perhaps most importantly, the requirement for manual approval of the user’s requested Flowspace from each island manager involved in an
experiment, is making the facility non-friendly for experiments. It is important that we change to an automated approval of the Flowspace, as
it is high likely that in an experimental facility the user would add/remove AMs, update his controller, add/remove links/VMs/switches
frequently and thus waiting for hours/days for manual approval of each change from each respective island manager involved in the
experiment, makes the facility cumbersome for experimentation.
•There can be consistency issues between the visual interface and reality. For example, a newly created VM can get stuck in a perpetual state
of ‘starting’. Yet in reality, the VM has started perfectly fine and is fully accessible via SSH. There is no way to stop/delete these VMs stuck in
this perpetual state.
•The standard VM image used on the island is now somewhat dated. Ideally, this image should be replaced with a newer version of Debian. If
possible, a selection of distributions should also be made available. Many of the experiments that we envisaged on OFELIA require
current/cutting-edge packages and repositories that are otherwise not available in an older release. Building software is therefore
unnecessarily painful and time consuming. We also encountered a number of bugs that have been fixed in later releases of the OS.
•We also had a number of connectivity problems whilst experimenting on OFELIA. As a result, we had to turn to a number of island managers
to help us out. Although incredibly helpful and more than willing to assist us, the island managers seemed ill-equipped to debug issues quickly
and effectively. It might be worthwhile putting together a toolkit for troubleshooting islands, and then making this available to all island
managers. Furthermore, finding and isolating these problems quickly is of absolute importance, especially when non-project users are
experimenting on the testbed. An easy solution to this is to create a monitoring slice within the testbed (Matt is working on this). Periodically
checking for connectivity between islands could provide a simple feedback mechanism that can alert island managers when their island goes
‘dark’.

46. Next steps
• Finish Clearinghouse
• Get an idea on how to stitch VLANs
– Without introducing centralized VID mgmt
– VLAN “clashing engine” was implemented
• Drastically clean up workflow and GUI.
• Eventually: OCF in a box – VM appliance
• OFELIA project itself is ending now, but islands have
signed MoU to continue operation
– FIBRE, FELIX, ALIEN FP7 projects can carry out some
development work
• And I encourage you very much to do so!