What do the terms serverless, containers, and virtual machines mean? Which should I use to build my app? The answer (as always) is "it depends." In this session learn the tradeoffs between these different approaches, whether you're building your app from scratch or want to move an existing web or mobile application to the cloud. We'll discuss open source tools such as Kubernetes, Istio, and Knative, and we'll discuss Google Cloud Platform tools like Compute Engine, Google Kubernetes Engine (GKE), App Engine, and Cloud Functions.
11. @BretMcG
Virtual
Machines
Google Compute Engine
Infrastructure as a Service
// Require the Stripe library w/ secret key.
const stripe = require('stripe')('sk_key...');
// Create a payment from a test card token.
const charge = await stripe.charges.create({
amount: 2000,
currency: 'usd',
source: 'tok_amex',
description: 'My first payment'
});
14. @BretMcG
Virtual machines and networking
● Per VM max: 160 CPUs, 3,844 GB of RAM, 64 TB of
disk, 96 GPUs, TPUs
● Independently configurable, including disk!
● Sole tenant nodes
Start quickly, ~20 seconds to user code running
Pre-built images, or create your own
● Debian, CentOS, CoreOS, Ubuntu, RHEL, SUSE
● Windows Server Datacenter: 2008, 2012 R2, 2016
What is Compute Engine?
Run your software
15. @BretMcG
Disk
Persistent Disk
● Live resize
● Network based
● Configure size (to 64 TB)
● Disk or SSD
● Independent of VM
Global Snapshots
Multi-VM RO mount
● Larger disks == more throughput
Local SSD
● Lowest latency
● 375 GB per disk (up to 8)
● Tied to VM lifecycle
16. @BretMcG
More than just virtual machines
Can run Docker containers directly on Container Optimized OS
Managed Instance Groups
● template / image based
● autoscaling
● regional groups (multi-zone)
● rolling updates and canaries
Load Balancer --> Google Front End, 1M QPS
17. @BretMcG
What do you think about?
Your software
Updating your software
Operating system / disk images
CPU, RAM, Disk
Networking
● Firewall rules
● Load balancers
● VPNs
18. @BretMcG
Good fit? Constraints?
Good fit
Existing systems (lift and shift)
1:1 container:VM mapping
Specific OS / kernel required
License requirements
Running databases
Network protocols beyond HTTP/S
Constraints
Scaling speed floor
~20s per VM, 1000s in ~60s
Decide how to handle software
updates yourself
20. @BretMcG
Why Compute Engine?
Consistency - ask for 10 VMs, you get 10 good VMs
Custom VMs - choose RAM and CPU, GPUs
Live disk resize - also any size disk on any machine
Billing - Sustained use discounts, preemptible VMs, per minute billing
Live migrations - your VMs stay up longer
Architecture - good fit for most existing systems
BTW, we run services on GCE, too.
26. @BretMcG
kernel
libs
app
app app
app
Shared machines
x
x
x
No isolation
Common libs
Highly coupled apps & OS
x
libs
app
kernel
libs
app
libs
app
libs
app
Containers
Isolation
No common libs
Less overhead
Less dependency on host OS
Evolution of app infrastructure
x
x
app
libs
kernel
libs
app app
kernel
app
libs
libs
kernel
kernel
Virtual Machines
Isolation
No common libs
Expensive and inefficient
Hard to manage
29. @BretMcG
I've got a bunch of apps/services
running in containers. Now what?
Kubernetes is open source and the
de facto platform for running containers.
30. @BretMcG
Kubernetes handles...
Scheduling:
Decide where my containers should run
Lifecycle and health:
Keep my containers running despite failures
Scaling:
Make sets of containers bigger or smaller
Naming and discovery:
Find where my containers are now
Load balancing:
Distribute traffic across a set of containers
Storage volumes:
Provide data to containers
Logging and monitoring:
Track what’s happening with my containers
Debugging and introspection:
Enter or attach to containers
Identity and authorization:
Control who can do things to my containers
31. @BretMcG
Making your app robust
Kubernetes keeps your applications running while you're asleep.
● Container died?
○ Restart it.
● Container unhealthy?
○ Reschedule to another node.
● Container overloaded?
○ Add more replicas automatically.
36. @BretMcG
What do you think about?
Logical infrastructure
Applications, not computers or containers
Whiteboarding view
● What programs?
● How are they connected?
● Where do they store state?
37. @BretMcG
Good fit? Constraints?
Good fit
Run app in multiple environments
● Clouds, hybrid, dev/test/prod
Take full advantage of containers
Have good team communication
● Between dev, ops, security, etc.
Have or want CI/CD pipeline
Network protocols beyond HTTP/S
Constraints
Must use containers
Licensing (ex: per-machine)
Some architectural constraints
40. @BretMcG
Developers using Kubernetes still have a lot to do
Have to do
Write code
Build docker image
Upload image to registry
Deploy service
Expose to the internet
Setup logging & monitoring
Scale workload
Want to do
Write code
41. @BretMcG
It's not the right abstraction for end-developer experience.
(This did not stop developers from using Kubernetes directly!)
But it's a great platform for building a PaaS on top of.
Kubernetes isn't actually for developers
46. @BretMcG
What is App Engine?
App Engine goal: let developers focus on code, while we handle the rest
“Serverless” before it was cool ;)
Started in 2008, with tradeoff: write it this way, we'll scale for you
Geared toward HTTP/web-facing applications
Can scale very quickly
47. @BretMcG
App Engine Standard (2nd gen)
● Open-source, idiomatic experience
● Use any extension, binary, or framework
● Supported Runtimes
○ Python 3.7
○ Java 8
○ Node.js 8, and Node.js 10 (beta)
○ PHP 7.2 (beta)
○ Go 1.11
48. @BretMcG
App Engine
Standard (2nd gen)
● Open-source, idiomatic experience
● Use any extension, binary, or framework
● Supported Runtimes
○ Python 3.7
○ Java 8
○ Node.js 8, and Node.js 10 (beta)
○ PHP 7.2 (beta)
○ Go 1.11
App Engine
Flexible
● Runs Docker containers
● Open source runtimes for Node.js, Ruby,
Java, Python, Go, PHP, .NET
● Also, if it runs in a Docker container,
you can run it ← nearly everything
49. @BretMcG
App Engine runtimes
Standard Standard 2nd generation
(soon!)
Flexible
Python 2.X, Java, PHP, Go Node.js 8 & 10, Python 3.7,
Go 1.11, PHP 7.2,
Node.js, Ruby, Java, Python,
Go, PHP, .NET
& custom container images
Sandboxed processes Runs on managed VMs
GAE specific APIs + GCP APIs Use GCP APIs directly
No binary modules/libraries any extension, binary, or framework
open-source, idiomatic experience
51. @BretMcG
Good fit? Constraints?
Good fit
HTTP/S request-response
Stateless serving applications
Scaling to high traffic
Constraints
Standard (1st gen)
● Runtimes for Python, Java, PHP, Go, Node
Standard (2nd gen)
● Can use use binary extensions
● Python, Node, Java, Go
Flexible runtimes
● Inherit Docker constraints
● Not best for very low traffic sites
52. @BretMcG
Why App Engine?
Developers focus on code, while we handle the rest
Optimized for web serving workloads
Especially great for variable load
App Engine currently serves more than 320B requests per day
58. @BretMcG
@BretMcG
Back end code functions that automatically
run in response to a trigger event
Microservices
Cloud "glue"
Node.js, Python, Go
Cloud Functions
59. @BretMcG
@BretMcG
Cloud Function event triggers
Cloud storage (incl. Firebase)
Cloud Pub/Sub
HTTPS
Firebase realtime database
& the new Firestore database
Firebase authentication
Google analytics for Firebase
Firebase hosting
Firebase Crashlytics
60. @BretMcG
Cloud Functions?
A serverless environment to build and connect cloud services
Event driven - connect Cloud services
● Via: HTTP requests, Cloud Storage, Firestore, Firebase events, and Cloud Pub/Sub
Serverless
● Fully managed execution environment
● Pay only for what you use
● Autoscales with usage
Node.js 6, 8, Python 3.7 (beta), Go 1.11 (beta)
Serverless Containers on GCF: sign up: g.co/serverlesscontainers
62. @BretMcG
Good fit? Constraints?
Good fit
“Serverless”
Using Pub/Sub and/or Cloud Storage
Don't want to think about runtime env
Data transformations (ETL)
Cloud-based HTTP glue
Constraints
Two runtimes today: JS on Node.js and
Python
Function level granularity
Must interact via events
63. @BretMcG
Why Cloud Functions?
Don't have to even think about servers
Pay only for what you use
Simple developer experience (deploy functions not apps)
70. @BretMcG
@BretMcG
Serverless containers
Secure on gVisor
Fully managed
BYO workloads
Pay for use & stateless
Alpha invitations coming
later this year
Sign up: g.co/serverlesscontainers
New: serverless containers on GCF (EAP)
77. @BretMcG
Recap - what do you think about?
Events
Function definitions
Code
HTTP Requests
Applications, not computers or containers
What programs? How are they connected? State?
Your software, operating system / disk images
CPU, RAM, Disk
Networking: Firewall rules, Load balancers, VPNs
Cloud Functions
App Engine
Kubernetes Engine
Compute Engine
78. @BretMcG
Technical requirements pull you down the stack
GPUs, need specific kernel, Windows,
software licensing requirements, migrating
most existing systems
Hybrid, need specific OS, network protocols
beyond HTTP/S
Support any programming language, run in
containers
Cloud Functions
App Engine
Kubernetes Engine
Compute Engine
79. @BretMcG
Team and organization
Team is mostly dev focused
Team is mostly dev focused
Team integration: Dev, Ops, Security work together;
org is open to app architecture updates
Adaptable to various team structures and tool
preferences
Cloud Functions
App Engine
Kubernetes Engine
Compute Engine
80. @BretMcG
Unsure? Containers let you move between
layers
Cloud Functions
App Engine
(2nd gen & Flex)
Kubernetes Engine
Compute Engine
(soon!)
87. @BretMcG
@BretMcG
Helping you decide: monoliths vs microservices
Choose the right
primitive for the
workload, or change
the workload
My Big
Java
Monolith
My Big
Java
Monolith
No!
Maybe
My
Discrete
Services
Yes!
✓
88. @BretMcG
Detour: Serverless Concurrency & Billing
Two Predominant Models:
1. Functions (Single Concern)
Billed per request, independent of instance
2. Apps (Multiple Concerns)
Billed per instance, independent of request
A Function (#1) model implies single concurrent
requests per instance!
90. @BretMcG
Detour: Single vs Multi-purpose
App Function
Multi Purpose
Some Shared State
Multi-concurrent
Single Purpose
No Shared State
Single-concurrent
91. @BretMcG
Lifecycle: Compute Time & Threads
The App model bills for the lifetime of an instance,
handles concurrent requests and accommodates
for compute time outside the scope of a request
The Function model bills for the lifetime of a
request, handles a single concurrent request and
assumes a freeze/thaw model where compute time
is not available outside the scope of the request
93. @BretMcG
Kubernetes
• Powerful
• Can be Scary
Serverless
• Higher-level abstraction
• Similar in concept, but a lot
friendlier and easier to manage
Kubernetes
and Serverless
94. @BretMcG
● OSS project on GitHub, Apache 2 license
● Set of primitives (Build, Events, Serving)
● Solves for modern development patterns
● Implements learnings from Google, partners
● In collaboration with OSS FaaS frameworks
● github.com/knative
Kubernetes-based
platform to build,
deploy, and manage
modern serverless
workloads
Bonus: Introducing Knative
99. @BretMcG
OK FINE. Start with App Engine Standard (2nd gen)
Web apps or APIs
Idiomatic for each language
Stateless
Scales to zero
Scales up very quickly
Node 8, 10
Python 3.7
Go 1.11
Java 8
PHP 7.2