Abitcool - A vast array of small-scale service providers with gigabit access, by Tony Hain. A presentation given at APNIC 38 during the APOPS 3 session.

1. aBitCool / Ignite
A vast array of micro service providers with gigabit access
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
a 21vianet group project
APnic38 – Brisbane, AU
15-17 September 2014
Tony Hain
Hain Global Consulting, Inc.
info@abitcool.com info@hain-global-consulting.com

2. Agenda
• aBitCool project goals
• Deployment plan
• Successes & Challenges
• Status
• Wrap up
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.

3. What is aBitCool?
• aBitCool is a project of 21Vianet Group, Inc:
“by the Geeks, for the Geeks”
What are the product and services that a “Super-Geek” would want?
(once proven commercially feasible, others will follow)
• Vision :
“Revolution is achieved through Consumer Empowerment”
• Mission :
“Transform Internet Deployment” / Restore Architecture Principles
• Strategy :
“Content Centric – Open (Permission Free) Exchange Network”
• Principle :
“Decentralized, Distributed, Disruptive”
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
3

4. aBitCool Project Goals
• Home as a Datacenter
– HomeAP access router as hub for content & services.
• Open eco-system at edge
– Members connect to an open Layer 2 exchange network, peering
with content and each other, rather than just acquiring IP transit.
– Equality (Members are active participants and actors)
– “Permission-free” innovation by members
• Redefining ISP
– Push State-of-the-art boundaries > l per member
– Drive advanced technology deployment rather than constrain
innovation to the limitations of the old.
– Encourage new-entrant ISPs, services, & content sources.
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.

5.  1Gbps+ services will change the historical ~20x per decade multiple that limits application evolution
EMERGENT INNOVATIONS
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
10Gbps+
5
Consumer Network Bandwidth Evolution
Consumer
Data Speeds
Enabling
Technology
1980s
300bps+
Dial-Up
1990s
9.6kbps+
Dial-Up /
GSM
2000s
256kbps+
ADSL
3G
2010s
5Mbps+
Cable
4G
Social
Media N/A
2020s
Fiber
5G
???
???
AVAILABLE CONSUMER TECHNOLOGY
4K 8K

6. Ignite : Bringing back “Inter-networking”
Member Sub1
Member Sub2
Member Sub3
Member Sub4
eBGP
λ1
λ2
λ3
λ4
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
Core Fabric
ICP Network
IDC AS
ICP1 AS
ICP2 AS
China Telecom
China Unicom
DC-IX peers
Transit Routing Network
POP Router
Transit Router
eBGP
eBGP
eBGP
Distribution
Fabric
HD Video
Gaming
ICP3 AS Social Media
Route Server
Member Network Switching Fabric Network
WDM Network, each member on a unique Lambda for upgrade path
Distribution
Fabric
WDM Device
OADM1
OADM2
OADM3
Sub1
Sub2
Sub3
Sub4
N*10GE
OTN-1
OTN-2
OTN-3
OTN-4
OTN-5
Optical Network
DC-IX fabric
Each Member is
an “ISP” with
Public IPv6 / IPv4
ASN (& BGP)
Dedicated
Bandwidth per
Member (N x
Lambda, λ)
Core Network is
a open Internet
Exchange
Peer-to-Peer L2
Fabric to reduce
physical Cost of
Cross-Connects
B2B2C platform
Members can
choose / buy
directly

7. Network Architecture Description
The network is composed of logical blocks:
Member Access: Members run BGP inside aBitCool network.
 Route Servers in Switching Fabric will establish EBGP Peers with ICP POP Routers and Residential subscribers
eBGP routers and reflect the BGP routes among 3rd parties.
 Switching Fabric: Distribution and Core Fabrics. Distribution will offer GE
fiber access service to members. Distribution connected with Core Fabric via
10GE or higher. Tbps switches will be clustered to provide capacity and
availability. Core Fabric will connect to ICP POP router at 10GE or higher.
Transit Routing: Is offered for members who have not established a direct
relationship with a transit provider.
ICP Network: The ICP network allows content owners to host abundant HD
video, Social Media and other inter-active service inside the aBitCool
network.
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.

8. Members and IPv4 Address
 There are 2 type of member subscribers in the network.
Subs Type ASN CPE WAN interface IPv4 inside
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
Subscribers network
Type A Member Public ASN Private IPv4 Public IPv4 network
Type B Member No ASN 1 Public IPv4 acquired
from APNIC
Private IPv4
network
 Type A member may establish BGP with RouteReflector and may choose to receive
BGP routes for internet transit service.
 Type B member will have static NAT GW configured on CPE and all traffic will be
routed to the GW.
 Both the Type A and Type B member will be assigned a static IPv4 address for WAN
interface of CPE.

9. Initial Access Network Design
IP CORE
Core Network
CWDM
OADM
S5700 SW
CWDN access Network
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
 There are 2 ways to connect the members.
 Using Dark Fiber
 Using CWDM
 Dark Fiber connection only can be used in the
circumstance of ONLY ONE Member in a community.
CWDM and OADM can be deployed to save Dark
Fiber resource. One line card can support 8 members
at maximum.
OADM

10. Initial BGP Design
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
Planned in Future
China Telecom
电信
China Unicom
联通
IDC AS
ASR9010-M06-1
BGP Peer with Member
BGP Peer with ICP
BGP Peer with ISP
NE40E-X8-M06-1
AbitCool AS
Type A member1
Type A member2
Type A member3
 EBGP connection will be established between Type A members and NE40E-X8,
IDC AS also establish EBGP with NE40E-X8.
 All Type A members, IDC edge router are in the same peering Subnet and
VLAN.
 All BGP routes will be propagated through NE40E-X8 and traffic will be
forwarded directly through the Switching Fabric.
 Peering Group for Type A members will be configured on NE40E-X8, so the
unified Route-Policy can be made on the outbound direction.

11. HomeAP - access router / distributed data center
• Provides consistent operational
interface to the network for
members that do not provide
their own router.
• 4 wan ports by default.
• 3 wifi radios : mesh as well as
client configurations
• Personal cloud
– P2P CDN
– Home as a data source
• Hosts distributed DC functions,
local caching, DNS, NTP
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.

12. Content consumption
• UHD TV dongle that runs
Firefox-OS that enables mobile
apps to “fling” video to the big
TV screen
• Partially Compatible with
Chromecast 1.0/2.0
• “Open” Match Stick vs
“Closed” Chromecast
• Building block for the
Decentralized “Video Internet”
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.

13. Real Example : A Current Home Network
Advent of Internet of Things will need 1,000 IPs @ home!
Home Network
SENSORS & APPLIANCES
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
Network Elements
Type #
Gateways 2
Routers 10
Switches 4
IP Devices 54
Total 70
Subnets 9
Connected devices
include
• 20 Servers
• 4 HD TVs
• 1 Green House
• 1 Threadmill
(dynamic maps w/tilt
for run routes)
Dual IPv4 / IPv6 network
GUEST
SOHO
ROUTER INTERCONNECT
DESKTOPS
SERVICES
MEDIA
REMOTE
ISP

14. Successes & Challenges
• Successes
– Equipment works as expected
– ?? Members with Gbps ports
• Challenges
– Team consistency on business plan and vision
• Low initial subscriber density requires ‘investment in a plan’
• Costs of fiber & initial transit often result in ‘go slow’ approach
• Fiber install permissions not acquired early ‘due to cost’
– Unusual APnic relationship with many micro-service
providers & individual members
• Likely evolving toward LIR
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.

15. Current Status
• Active Members : #####
• Members peering directly with ICP : #####
• Traffic volume : #######
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
Initial Measured Throughput
SDN allows even higher ACTUAL
throughput in user experience.

16. info@abitcool.com
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
info@hain-global-consulting.com

17. A vast array of micro service providers with gigabit access
Copyright 2014 – 21vianet group Inc, & Hain - Hain Global Consulting, Inc. Global Consulting, Inc.
aBitCool / Ignite
a 21vianet project
APnic38 - Brisbane
Sept 15, 2014
Tony Hain
CEO
Hain Global Consulting, Inc.
tony@hain-global-consulting.com

18. Initial LINK Aggregation
Internet
S5700-M06-1 S5700-SITE B-1
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
NE40E-X8-M06-1
S5700-SITE A-1 S5700-SITE C-1 S5700-SITE D-1
 Initial physical topology is a Layer 1 full ring, provisioned as a logical star.
 More IP links can be used after the planned OTN Transmission Network is deployed.
 Link aggregation for the uplinks of S5700 will be deployed in order to increase uplink
bandwidth and link redundancy.
 LACP will be deployed to negotiated LINK GROUP between NE40E-X8 and S5700.

19. Type B member - CGN NAT Design
Type B member
CPE1 WAN：X1/22
Type B member
CPE 2 WAN：X2/22
Type B member
CPE3 WAN：X3/22
Copyright 2014 – 21vianet group Inc, & Hain Global Consulting, Inc.
NAT IPv4 Pool：Z/27
Source IPv4 NAT translation：
X1,X2,X3Z
Internet
 There is no BGP connection from NE40E-X8 to China Telecom or China Unicom
 NAT service line card is deployed on NE40E-X8 to translate CPE WAN Public IPv4
addresses (TYPE B) into IPv4 address from NAT IPv4 Pool.
 Syslog server will be deployed to record the NAT LOGs.