Packet Networks presentaton at Dell EMC NYC DevOps meetup with Zac Smith

2. OUR PASSION:
BUILD A
BETTER
INTERNET.

3. A Premium Bare Metal Cloud Built for Automation
● Fully dedicated servers, no co-tenancy
● Billed by the hour & available in minutes via API
● No hypervisor, no virtualization
● Backed by our global, IPv6 network

4. Wait, no virtualization at all?
Nope.
(unless you bring your own)
Which, when building a cloud platform, presents some unique challenges...

5. Public clouds (AWS, Digital Ocean, GCE) all rely on
a hypervisor or containers to automate the
provisioning of virtual machines to end users.
THE CHALLENGE
In order to automate “Layer 0” for developers, we
we had to build core infrastructure components
from the ground up.

6. But first we tried to do it with OpenStack.
THE FAILURE
( our now infamous blog post about this available here: https://www.packet.net/blog/how-we-failed-at-openstack/ )
Learning from the limitations in OpenStack, we decided to
start over with an api-driven microservices framework
strategy. This meant developing purpose-built services for
each requirement.

7. Micro services for each component of the physical datacenter.
PACKET PLATFORM
➢ Built in golang and Ruby
➢ Everything in Docker and CoreOS
➢ Orchestrated and managed with Rancher
➢ Testing with shippable, image builds with quay.io

8. Micro services for each component of the physical datacenter.
PACKET PLATFORM
Power and
Boot Control
PB&J
EC2 Style
Metadata
Kant
iPXE Server &
Imaging
Tinkerbell
Netflow Agg &
Analysis
Soren
Multi-Tenant
IPAM
Magnum IP
Physical Switch
SDN
Narwhal
Device, Project,
Billing, Token
Management
Client
Portal
Internal and
External
Services
API
Serial Console
Out of Band
Access
S.O.S.
Image Building
Casper

9. Let’s review a selection of the more interesting
microservices...

10. KANT
EC2 Compatible Metadata Service

11. Security challenge: IP based authentication, no hypervisor,
our layer 3 acl solution
KANT
EC2 Compatible Metadata Service
EC2 has a problematic url structure / IP address
(eg: http://169.254.169.254/latest/meta-data/ )
Available at https://metadata.packet.net from all hosts

12. Tinkerbell
iPXE Server & Imaging
pxe server… tinkerbell … get it? :D

13. API driven DHCP and iPXE server
Tinkerbell
iPXE Server & Imaging
Handles delivery and selection of both end user operating
system images, and also our deprovision images
Replacement for cobbler, provides massive stability and
speed improvements as well as better diagnostic data and
visibility in every stage of provisioning

14. Narwhal
Physical Switch / Router “SDN”
apparently there aren’t any entertaining public domain pictures or gifs of narwhals. apologies.

15. Overcomes one of the biggest limitations of OpenStack for
us, which is very vlan oriented
Configures the network ACLs and handles our elastic IP
addressing, batches thousands of operations an hour
Narwhal
Physical Switch / Router “SDN”
Built to be “pluggable” to accommodate other vendors in
the future, like Cisco and Arista

16. Soren
Netflow Aggregation & Analysis
can you tell we have some philosophy majors on the team yet?

17. Ingests all of our network flow data and tags it
Allows us to do things like destination based bandwidth
billing and tiered bandwidth
Soren
Netflow Aggregation & Analysis
Gives us the ability to expose much more network
intelligence to end users, such at top talkers and last mile
reachability

18. PB&J
Power and Boot Control (IPMI / DRAC)
this is our favorite result in google image search for “fucking ipmi”

19. Abstracts much of the ugliness of IPMI and DRAC from
our API
Gives us access to health and other hardware monitoring
as well
PB&J
Power and Boot Control (IPMI / DRAC)
Roadmap to support redfish and potentially open source
the library in the future

20. In summary, where we are now...

21. NO OVERLAYS
From network to host, we provide elastic control without adding network or hardware abstraction.
MICROSERVICE BASED
Shippable, immutable independent services that are easy to use, scale and troubleshoot.
REPRODUCIBLE & DEPENDABLE
99% of our monthly installs succeed with an average provision time of ~8 minutes.
MODERN & SCALABLE

22. PLATFORM HIGHLIGHTS
Launched in August 2015. No Co-Tenancy. No Hypervisor. Full Automation.

23. CURRENT CONFIGS
We Also Offer High Performance Block Storage, Elastic IP Addressing and Self-Service BGP.
TYPE 0
TINY BUT
MIGHTY
TYPE 1
FLEXIBLE
WORKHORSE
TYPE 2
THE
VIRTUALIZOR
TYPE 2A
ARM ‘N
HAMMER
$0.05
/hour
$0.40
/hour
$1.25
/hour
$0.50
/hour
4 Physical Cores
@ 2.4 GHz
(1 × Atom C2550)
8 GB of RAM
1 x 120 GB SSD
4 Physical Cores
@ 3.4 GHz
(1 × E3-1240 v3)
32 GB of RAM
2 x 240 GB SSD
24 Physical Cores
@ 2.2 GHz
(2 x E5-2650 v4)
256 GB of RAM
6 x 480 GB SSD
96 Physical Cores
@ 2.0 GHz
(2 × ThunderX)
128 GB of RAM
1 x 320 GB M.2 SSD
TYPE 3
HIGH I/O
MONSTER
$1.75
/hour
16 Physical Cores
@ 2.6 GHz
(2 × E5-2640 v3)
128 GB of RAM
1 x 2.4 TB NVME

24. INTEGRATIONS
We offer developers a consistent infrastructure experience that
integrates with leading platforms and dev tools - but with the firepower and flexibility of bare metal.

25. THANKS!
Any questions?
You can find us at @packethost & help@packet.net