Visit itscales.org Watch http://www.youtube.com/watch?v=HeXDvOZcdD4 The social web is alive! And with it all of its technical challenges. Never had the servers to answer that many requests, never to store so much data. While the technological leaders like Facebook and Google use highly customized super scale systems that run on provincial town sized server farms distributed all over the world current web frameworks barely offer a single strategy to scale. Where the leaders use highly efficient caching mechanisms, frameworks recalculate responses over and over again. Where the leaders automatically add new machines for processing, frameworks allow to manually add machines without a designated field of work. Scales comes to the rescue! It’s a super scale web framework developed with the knowledge of this thesis that clearly splits up responsibilities and makes them individually scaleable. No response is calculated twice as 100% of the resources are stored in a highly efficient, distributed key-value storage. The content itself is created with common, well-known frameworks like the beautiful Ruby on Rails and the whole system can be monitored using a website that updates in real-time. The universal design goals for the framework are examined using load tests and experiments that put a light on the latest system limits, I/O strategies, protocols and caching technologies.

1. Thomas Fankhauser
Master Thesis Colloquium

2. Knowledge Stack
Database Request
Limits Limits
Operating Integer
Figures Limits Framework
System Limits
Key
Principles
Limits
Concurrency
Super Scale
Systems
Scalability
Sync
I/O
Strategies
Async
API
3D-Hosting
Application
Scales HTTP
Architecture
Protocols &
Design Technologies
Goals Server-Sent
Modules Events
Demo Caching
Web
Sockets
Techno- Tailor vs.
logies Superstore

3. Knowledge Stack
Database Request
Limits Limits
Operating Integer
Figures Limits Framework
System Limits
Key
Principles
Limits
Concurrency
Super Scale
Systems
Scalability
Sync
I/O
Strategies
Async
API
3D-Hosting
Application
Scales HTTP
Architecture
Protocols &
Design Technologies
Goals Server-Sent
Modules Events
Demo Caching
Web
Sockets
Techno- Tailor vs.
logies Superstore

4. 1
Super Scale Systems

5. 1
Ultra Large Web Applications
Super Scale Systems

6. 1
How large?
5 trillion 235 billion 943 million Facebook
310 billion 390 million Twitter
2,078 million youTube
170 million Google+
160 million LinkedIn

7. 1
How much do they grow per day?
4,335 million Facebook
341 million Twitter
1 million youTube
500 thousand Google+
50 thousand LinkedIn

8. Structural Load
1
Scaleability!
Bondi (2000)
Space Distance

9. Knowledge Stack
Database Request
Limits Limits
Operating Integer
Figures Limits Framework
System Limits
Key
Principles
Limits
Concurrency
Super Scale
Systems
Scalability
Sync
I/O
Strategies
Async
API
3D-Hosting
Application
Scales HTTP
Architecture
Protocols &
Design Technologies
Goals Server-Sent
Modules Events
Demo Caching
Web
Sockets
Structural Scalability
Load Scalability
Techno- Tailor vs.
logies Superstore

10. Knowledge Stack
Database Request
Limits Limits
Operating Integer
Figures Limits Framework
System Limits
Key
Principles
Limits
Concurrency
Super Scale
Systems
Scalability
Sync
I/O
Strategies
Async
API
3D-Hosting
Application
Scales HTTP
Architecture
Protocols &
Design Technologies
Goals Server-Sent
Modules Events
Demo Caching
Web
Sockets
Structural Scalability
Load Scalability
Techno- Tailor vs.
logies Superstore

11. 2
System Limits

12. 2
Infinitely Increment an Integer
Java Overflow
C Overflow
Ruby Running and running and ...
Danger! Overflows happen without notice!

13. 2
Database Rows and Query Times
Total Request Maximum 1000ms (Nielson)
Total Query Maximum 500ms
Single Query Time 133ms at 3 Queries
3,632,492 Dedicated DB
1,441,419 Rows Servers for Facebook

14. 2
Request Limits
Simple HTML Index Page
Nginx 12,049 Requests/s
NodeJS 11,922 Requests/s
Amazon AWS IO Bound
Vertical : Horizontal Scaling 2.56 : 1

15. 2
Framework Limits
Social Rails App with 100 Users, 1,000 Statuses
and 1,300 Friend Connections
m1.small 12 Requests/s
m1.large 77 Requests/s
Amazon AWS CPU Bound
Vertical : Horizontal Scaling 1.14 : 1

16. 2
Summary
Caching is Essential
Bottlenecks Shifted, never solved
From request-build-respond
To request-lookup-respond

17. Knowledge Stack
Database Request
Limits Limits
Operating Integer
Figures Limits Framework
System Limits
Key
Principles
Limits
Concurrency
Super Scale
Systems
Scalability
Sync
I/O
Strategies
Async
API
3D-Hosting
Application
Scales HTTP
Architecture Preprocess
Protocols &
Scale Horizontal
Design Technologies
Goals Server-Sent Caching
Modules Events
Demo Caching Ruby
Web
Sockets
Structural Scalability
Load Scalability
Techno- Tailor vs.
logies Superstore

18. Knowledge Stack
Database Request
Limits Limits
Operating Integer
Figures Limits Framework
System Limits
Key
Principles
Limits
Concurrency
Super Scale
Systems
Scalability
Sync
I/O
Strategies
Async
API
3D-Hosting
Application
Scales HTTP
Architecture Preprocess
Protocols &
Scale Horizontal
Design Technologies
Goals Server-Sent Caching
Modules Events
Demo Caching Ruby
Web
Sockets
Structural Scalability
Load Scalability
Techno- Tailor vs.
logies Superstore

19. 3
I/O Strategies

20. 3
Concurrency
920 KB Memory Process
100 KB Memory Thread
4 KB Memory Fiber

21. 3
Sync vs. Async
Easy to code Hard to code
Big Memory Footprint Small Memory Footprint
Context Switching Reactor Loop

22. Knowledge Stack
Database Request
Limits Limits
Operating Integer
Figures Limits Framework
System Limits
Key
Principles
Limits
Concurrency
Super Scale
Systems
Scalability
Sync
I/O
Strategies
Async
API
3D-Hosting
Application
Async I/O
Fibers
Scales HTTP
Architecture Preprocess
Protocols &
Scale Horizontal
Design Technologies
Goals Server-Sent Caching
Modules Events
Demo Caching Ruby
Web
Sockets
Structural Scalability
Load Scalability
Techno- Tailor vs.
logies Superstore

23. Knowledge Stack
Database Request
Limits Limits
Operating Integer
Figures Limits Framework
System Limits
Key
Principles
Limits
Concurrency
Super Scale
Systems
Scalability
Sync
I/O
Strategies
Async
API
3D-Hosting
Application
Async I/O
Fibers
Scales HTTP
Architecture Preprocess
Protocols &
Scale Horizontal
Design Technologies
Goals Server-Sent Caching
Modules Events
Demo Caching Ruby
Web
Sockets
Structural Scalability
Load Scalability
Techno- Tailor vs.
logies Superstore

24. 4
Protocols and Technologies

25. TCP Handshake
Terminate Connection
HTTP 1.0
4
40ms Processing 20ms Latency
3 Requests Each 160ms Total 480ms
Server
SYN 1
ACK 1
FIN 1
FIN-ACK 1
SYN 2
ACK 2
FIN 2
FIN-ACK 2
SYN 3
ACK 3
FIN 3
FIN-ACK 3
Client 40 60 80 100 120 140 160 180 200 220 240 260 280 300 320 340 360 380 400 420 Time in ms
Open 1 Request 1 Close 1 Open 2 Request 2 Close 2 Open 3 Request 3 Close 3

26. HTTP 1.1
Keep-Alive
4
Pipelining
3 Requests Each 160ms Total 320ms
Server
SYN
ACK
FIN
FIN-ACK
Client 40 60 80 100 120 140 160 180 200 220 240 260 280 Time in ms
TCP Handshake Request 1 Request 2 Request 3 TCP Termination

27. HTTP 1.1
Keep-Alive
4
Pipelining
3 Requests Each 160ms Total 160ms
Server
SYN
ACK
FIN
FIN-ACK
Client 40 60 80 100 120 140 Time in ms
TCP Handshake Request 1 Request 2 Request 3 TCP Termination

28. 4
Server-Sent Events vs. WebSockets
Push only Bidirectional
DOM Events Publish/Subscribe
Same Origin Policy Any Origin

29. Knowledge Stack
Database Request
Limits Limits
Operating Integer
Figures Limits Framework
System Limits
Key
Principles
Limits
Concurrency
Super Scale
Systems
Scalability
Sync
I/O
Strategies
Async
WebSockets
API
3D-Hosting HTTP 1.1
Application
Async I/O
Fibers
Scales HTTP
Architecture Preprocess
Protocols &
Scale Horizontal
Design Technologies
Goals Server-Sent Caching
Modules Events
Demo Caching Ruby
Web
Sockets
Structural Scalability
Load Scalability
Techno- Tailor vs.
logies Superstore

30. Knowledge Stack
Database Request
Limits Limits
Operating Integer
Figures Limits Framework
System Limits
Key
Principles
Limits
Concurrency
Super Scale
Systems
Scalability
Sync
I/O
Strategies
Async
WebSockets
API
3D-Hosting HTTP 1.1
Application
Async I/O
Fibers
Scales HTTP
Architecture Preprocess
Protocols &
Scale Horizontal
Design Technologies
Goals Server-Sent Caching
Modules Events
Demo Caching Ruby
Web
Sockets
Structural Scalability
Load Scalability
Techno- Tailor vs.
logies Superstore

31. 5
Caching

32. 5
Tailor vs. Superstore
Current Web Future Web
On Demand On Stock
Highly Customized Preselected Goods

33. Storage Technologies
5
Write Time for X 64 KB Files
Write Time in Seconds
35
30
25
20
15
10
5
1,000 5,000 10,000 15,000 20,000 25,000 Total Items
File Memcached MongoDB Redis

34. Storage Technologies
5
Read Time for X 64 KB Files
Read Time in Seconds
6
5
4
3
2
1
1,000 5,000 10,000 15,000 20,000 25,000 Total Items
File Memcached MongoDB Redis

35. Knowledge Stack
Database Request
Limits Limits
Operating Integer
Figures Limits Framework
System Limits
Key
Principles
Limits
Concurrency
Super Scale
Systems
Scalability
Sync
I/O
Strategies
Redis
Async
WebSockets
API
3D-Hosting HTTP 1.1
Application
Async I/O
Fibers
Scales HTTP
Architecture Preprocess
Protocols &
Scale Horizontal
Design Technologies
Goals Server-Sent Caching
Modules Events
Demo Caching Ruby
Web
Sockets
Structural Scalability
Load Scalability
Techno- Tailor vs.
logies Superstore

36. Knowledge Stack
Database Request
Limits Limits
Operating Integer
Figures Limits Framework
System Limits
Key
Principles
Limits
Concurrency
Super Scale
Systems
Scalability
Sync
I/O
Strategies
Redis
Async
WebSockets
API
3D-Hosting HTTP 1.1
Application
Async I/O
Fibers
Scales HTTP
Architecture Preprocess
Protocols &
Scale Horizontal
Design Technologies
Goals Server-Sent Caching
Modules Events
Demo Caching Ruby
Web
Sockets
Structural Scalability
Load Scalability
Techno- Tailor vs.
logies Superstore

37. 6

38. Design Goals 6
Knowledge Stack
Multiple Machine Support
Redis O(1) Time Complexity
WebSockets Async I/O with Fibers
HTTP 1.1
Multiple Cores Support
Async I/O
Fibers
100% Cache
Preprocess 100% Timeliness of Data
Scale Horizontal Decouple Reads, Processing and Writes
Caching
Support Existing Frameworks
Ruby
Structural Scalability
Event System
Load Scalability
Versatile Content Types
Resource Overview

39. Animation
http://www.youtube.com/watch?v=HeXDvOZcdD4

40. Update
6
API

41. API
Push Update
6
@html = "<html><head></head><body><p>Hello World</p></body></html>"
@xml = "<tracks><track><name>Islandary</name>
<artist>Thomas Fankhauser</artist></track></tracks>"
@json = '[{ name : "Islandary", artist : "Thomas Fankhauser" }]'
# Push a HTML
Scales.push :html => @html, :to => "/hello"
# Push a XML
Scales.push :xml => @xml, :to => "/hello.xml"
# Push a JSON
Scales.push :json => @json, :to => "/hello.json"
# Push an Image (PNG)
Scales.push :png => IO.read("path/to/image.png"), :to => "/hello.png"

42. API
Update Update
6
# HTML Update URLs
Scales.update "/", "/tracks", "/overview", :format => :html
# XML Update URLs
Scales.update "/tracks.xml", "/overview.xml", :format => :xml
# JSON Update URLs
Scales.update "/tracks.json", "/overview.json", :format => :json

43. API
Modify Update
6
# HTML Modifications
Scales.append :html => '<p id="track2">Track 2</p>', :to => "/tracks", :select => "#tracks"
Scales.prepend :html => '<p id="track2">Track 2</p>', :to => "/tracks", :select => "#tracks"
Scales.set :html => '<p id="track2">Track 2</p>', :at => "/tracks", :select => "#tracks"
Scales.replace :html => '<p id="track2">Track 2</p>', :at => "/tracks", :select => "#tracks"
Scales.remove :at => "/tracks", :select => "#tracks"
# XML Modifications
Scales.append :xml => '<track>Track 2</track>', :to => "/tracks.xml", :select => "/tracks"
Scales.prepend :xml => '<track>Track 2</track>', :to => "/tracks.xml", :select => "/tracks"
Scales.set :xml => '<track>Track 2</track>', :at => "/tracks.xml", :select => "/tracks"
Scales.replace :xml => '<track>Track 2</track>', :at => "/tracks.xml", :select => "/tracks/track"
Scales.remove :at => "/tracks.xml", :select => "/tracks/track"
# JSON Modifications
Scales.append :json => '{ "id":2, "name":"Wait for it" }', :to => "/tracks.json", :select => "$.tracks"
Scales.prepend :json => '{ "id":2, "name":"Wait for it" }', :to => "/tracks.json", :select => "$.tracks"
Scales.set :json => '{ "id":2, "name":"Wait for it" }', :to => "/tracks.json", :select => "$.tracks"
Scales.replace :json => '{ "id":2, "name":"Wait for it" }', :to => "/tracks.json", :select => "$.tracks"
Scales.remove :at => "/tracks.json", :select => "$.tracks"

44. API
Destroy Update
6
# Destroy URLs
Scales.destroy "/", "/tracks", "/overview", "/overview.json", "/overview.xml"

45. Application
How does the code change? (1) Update
6
# GET /tracks/1
def show
@track = Track.find(params[:id])
render "show"
end
# GET /tracks/1
def show
@track = Track.find(params[:id])
Scales.push :html => render("show"), :to => "/tracks/#{@track.id}"
end

46. Application
How does the code change? (2) Update
6
# PUT /tracks/1
def update
@track = Track.find(params[:id])
if @track.update_attributes(params[:track])
redirect_to @track, notice: 'Track was successfully updated.'
else
render action: "edit"
end
end
# PUT /tracks/1
def update
@track = Track.find(params[:id])
if @track.update_attributes(params[:track])
Scales.update "/tracks/#{@track.id}", "/tracks/#{@track.id}/edit", "/tracks", :format => :html
redirect_to @track, notice: 'Track was successfully updated.'
else
render action: "edit"
end
end

47. Application
Scale Up Update
6
require 'scales/up/rails'
desc "Scale up the cache"
Scales::Up.new do |scales|
# Stylesheets
scales.push :css, :to => "/assets/application.css?body=1"
scales.push :css, :to => "/assets/scaffolds.css?body=1"
scales.push :css, :to => "/assets/tracks.css?body=1"
# Javascripts
scales.push :js, :to => "/assets/jquery.js?body=1"
scales.push :js, :to => "/assets/jquery_ujs.js?body=1"
scales.push :js, :to => "/assets/tracks.js?body=1"
scales.push :js, :to => "/assets/application.js?body=1"
# Images
scales.push :png, :to => "/assets/rails.png"
# Tracks
scales.update "/", "/tracks", "/tracks/new", :format => :html
Track.all.each{ |track| scales.update "/tracks/#{track.id}", "/tracks/#{track.id}/edit", :format => :html }
end

48. Update
6
Live Demo

49. Contact Me Try
tommylefunk@gmail.com gem install scales
@tommylefunk itscales.org
southdesign.de github.com/scales/scales
Update
Thank you!
Buy My Apps Read Full Thesis
PianoTabs
https://dl.dropbox.com/u/889313/
Beatfreak
thomas_fankhauser_-_super_scale_systems.pdf
QuestionPad
Take Away Note Preprocess everything!