Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2HOefXf. Kenton Varda explains how Cloudflare built a compute platform using V8 isolates instead of containers or VMs, achieving 10x-100x faster cold starts and lower memory footprints. He goes through technical details of embedding V8, distributing code, scheduling isolates, resource management, and security risks. Filmed at qconlondon.com. Kenton Varda is the architect of Cloudflare Workers, a "serverless" compute platform which distributes the code to 165+ locations globally so that it always runs as close to the client as possible. Prior to joining Cloudflare, he created Sandstorm.io and Cap'n Proto. Further back, while at Google, he wrote Protobuf v2 and open sourced it.

1. Massive Multitenancy with V8 Isolates
Kenton Varda - Tech Lead, Cloudflare Workers

2. InfoQ.com: News & Community Site
Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
cloudflare-v8
• Over 1,000,000 software developers, architects and CTOs read the site world-
wide every month
• 250,000 senior developers subscribe to our weekly newsletter
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• 2 dedicated podcast channels: The InfoQ Podcast, with a focus on
Architecture and The Engineering Culture Podcast, with a focus on building
• 96 deep dives on innovative topics packed as downloadable emags and
minibooks
• Over 40 new content items per week

3. Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
Presented at QCon London
www.qconlondon.com

4. The Challenge

5. 165 Locations
and growing

6. Scalability can mean...
Tenants (apps)
Hard: Every tenant in every location.
Some locations are small!
Traffic (requests)
Easy: More locations = more capacity.

7. Needed:
Efficiency

8. I, , made or led:
● Protobufs v2
● Cap'n Proto
● Sandstorm.io
● Cloudflare Workers
Warning - I am :
● An experienced speaker
● A graphics designer

9. Efficiency...
App Code Footprint
VM: 10GB
Container: 100MB
Needed: < 1MB
Context Switching
VM: low
Container: medium
Needed: extreme
Baseline Memory Usage
VM: 1GB
Container: 100MB
Needed: < 5MB
Startup Time
VM: 10s
Container: 500ms
Needed: < 5ms

10. Other use cases
APIs
Run client code directly on API server.
Big Data Processing
Run code where the data lives.
Web Browsers
Run code from visited sites.

11. We built this already!

12. Browsers are optimized for...

13. V8 JavaScript Runtime:
An Extreme Multitenancy Engine

14. Isolates and APIs

16. Hardware (virtualized)
Operating System
Libraries
Application
Provided by host Provided by guest
Hardware
Operating System
Application
Hardware
Operating System
Uncommon libraries
Application
Web Platform APIs
VMs Containers Isolates
JS RuntimeLanguage Runtime
Libraries
Language Runtime

17. HTTP client:
HTTP server:

18. Language Libraries
Application
Hardware
Operating System
Uncommon libraries
Application
WASM Isolates
Language Runtime
API Bindings
WebAssembly?
Missing a way to share common runtimes...
Web Platform APIs
JS Runtime
Hardware
Operating System
Web Platform APIs
JS Runtime

19. Resource Management

20. OOM Killing
as a First Resort
Isolate
Isolate
Isolate
Isolate
Isolate
Isolate
Isolate
Isolate
Isolate
Isolate
Isolate
OOM priority
Desired total memory usage. Evict these.
Prioritize: LRU, high memory usage

21. Resource limits
CPU
Isolates run on separate threads.
timer_create(CLOCK_THREAD_CPUTIME_ID)
isolate.TerminateExecution()
RAM
Monitor with isolate.GetHeapStatistics()
Evict isolates that go over limit.

22. Code Distribution

23. Security

24. Is V8 secure enough for servers?

25. Deep in v8/src/compiler/typer.cc…
Optimizer: "Math.expm1() can return real number or NaN."
Forgot: -0 (negative zero)
Full sandbox breakout!
Awesome writeup: Google "Andrea Biondo V8 bug"
Link: https://abiondo.me/2019/01/02/exploiting-math-expm1-v8/
V8 bugs...

26. NOTHING
IS
"SECURE"
Security is Risk Management

27. Relatively more bugs than VMs.
Reasons:
● Larger attack surface (Bad)
● More research (Good)
○ Bug Bounty
○ Fuzzing
○ Important target

28. Risk Management
Browser Server
VS

29. Risk Management
Browser Server
Install updates fast.
VS

30. Risk Management
Browser Server
Install updates fast. Install updates faster.
VS

31. Risk Management
Browser Server
Install updates fast.
Use separate profiles for
trusted vs "suspicious" sites.
Install updates faster.
VS

32. Risk Management
Browser Server
Install updates fast.
Use separate profiles for
trusted vs "suspicious" sites.
Install updates faster.
Use separate processes for
trusted vs. "suspicious"
tenants.
VS

33. Risk Management
Server
VS
Browser

34. Risk Management
Server
Store all scripts ever uploaded
for forensic purposes. No eval().
VS
Browser

35. Risk Management
Server
Store all scripts ever uploaded
for forensic purposes. No eval().
Watch for segfaults, inspect
scripts that cause them.
VS
Browser

36. Risk Management
Server
Store all scripts ever uploaded
for forensic purposes. No eval().
Watch for segfaults, inspect
scripts that cause them.
VS
Browser
… can't, privacy violation.

37. What about Spectre?

39. We have no solution
except process
isolation.
We can neither confirm
nor deny that process
isolation is enough.

40. Thread 1 Thread 2
No (local) timers
(at all!)
No (local)
concurrency
Freedom to
reschedule

41. Big Picture

42. Mainframe
Commodity Server
Virtual Machine
Container
Isolate
Units of
Compute
Granularity

43. Questions?

44. Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
cloudflare-v8