This document summarizes the life of a request on Google App Engine. It describes how requests for static content are routed to specialized static content servers. Requests for dynamic content are routed to App Servers where the code runs. App Servers can also make API requests to services like Memcache and the Datastore. The document then recaps the design motivations behind App Engine like building on Google infrastructure, encouraging fast requests and statelessness, and requiring a partitioned data model.

2. From Spark Plug to Drive Train:
Life of an App Engine Request
Patrick Chanezon @chanezon
Ignacio Blanco @blanconet
11/16/09
Post your questions for this talk on Twitter #devfest09 #appengine

3. Designing for Scale and Reliability

4. Agenda
Designing for Scale and Reliability
App Engine: Design Motivations
Life of a Request
Request for static content
Request for dynamic content
Requests that use APIs
App Engine: Design Motivations, Recap
App Engine: The Numbers

5. Google App Engine

6. LiveJournal circa 2007
From Brad Fitzpatrick's USENIX '07 talk:
"LiveJournal: Behind the Scenes"

7. LiveJournal circa 2007
From Brad Fitzpatrick's USENIX '07 talk:
"LiveJournal: Behind the Scenes" Memcache
Application
Frontends Servers Storage
Static File Servers

8. Basic LAMP
Linux, Apache, MySQL,
Programming Language
Scalable?
Shared machine for database
and webserver
Reliable?
Single point of failure (SPOF)

9. Dedicated Database
Database running on a
separate server
Requirements
Another machine plus
additional management
Scalable?
Up to one web server
Reliable?
Two single points of failure

10. Multiple Web Servers
Benefits:
Grow traffic beyond the capacity of one
webserver
Requirements:
More machines
Set up load balancing

11. Multiple Web Servers
Load Balancing: DNS Round Robin
Register list of IPs with DNS
Statistical load balancing
DNS record is cached with Time To Live (TTL)
TTL may not be respected

12. Multiple Web Servers
Load Balancing: DNS Round Robin
Register list of IPs with DNS
Statistical load balancing
DNS record is cached with Time To Live (TTL)
TTL may not be respected
Now wait for DNS changes to propagate :-(

13. Multiple Web Servers
Load Balancing: DNS Round Robin
Scalable?
Add more webservers as necessary
Still I/O bound on one database
Reliable?
Cannot redirect traffic quickly
Database still SPOF

14. Reverse Proxy
Benefits:
Custom Routing
Specialization
Application-level load balancing
Requirements:
More machines
Configuration and code for reverse proxies

15. Reverse Proxy
Scalable?
Add more web servers
Specialization
Bound by
Routing capacity of reverse proxy
One database server
Reliable?
Agile application-level routing
Specialized components are more robust
Multiple reverse proxies requires network-level routing
DNS Round Robin (again)
Fancy network routing hardware
Database is still SPOF

16. Master-Slave Database
Benefits:
Better read throughput
Invisible to application
Requirements:
Even more machines
Changes to MySQL

17. Master-Slave Database
Scalable?
Scales read rate with # of servers
But not writes
What happens eventually?

18. Master-Slave Database
Reliable?
Master is SPOF for writes
Master may die before replication

19. Partitioned Database
Benefits: Requirements:
Increase in both read Even more machines
and write throughput Lots of management
Re-architect data model
Rewrite queries

20. The App Engine Stack

21. App Engine:
Design Motivations

22. Design Motivations
Build on Existing Google Technology
Provide an Integrated Environment
Encourage Small Per-Request Footprints
Encourage Fast Requests
Maintain Isolation Between Applications
Encourage Statelessness and Specialization
Require Partitioned Data Model

23. Life of a Request:
Request for Static Content

24. Request for Static Content on Google Network
Routed to the nearest Google datacenter
Travels over Google's network
Same infrastructure other Google products use
Lots of advantages for free

25. Request for Static Content
Routing at the Front End
Google App Engine Front Ends
Load balancing
Routing
Frontends route static requests to specialized serving
infrastructure

26. Request for Static Content
Static Content Servers
Google Static Content Serving
Built on shared Google Infrastructure
Static files are physically separate from code files
How are static files defined?

27. Request for Static Content
What content is static?
Java Runtime: appengine-web.xml
...
<static>
<include path="/**.png" />
<exclude path="/data/**.png />
</static>
...
Python Runtime: app.yaml
...
- url: /images
static_dir: static/images
OR
- url: /images/(.*)
static_files: static/images/1
upload: static/images/(.*)
...

28. Request For Static Content
Response to the user
Back to the Front End and out to the user
Specialized infrastructure
App runtimes don't serve static content

29. Life of a Request:
Request for Dynamic Content

30. Request for Dynamic Content: New Components
App Servers and App Master
App Servers
Serve dynamic requests
Where your code runs
App Master
Schedules applications
Informs Front Ends

31. Request for Dynamic Content: Appservers
What do they do?
Many applications
Many concurrent requests
Smaller app footprint + fast requests = more apps
Enforce Isolation
Keeps apps safe from each other
Enforce statelessness
Allows for scheduling flexibility
Service API requests

32. Request For Dynamic Content
Routing at the Frontend
Front Ends route dynamic
requests to App Servers

33. Request for Dynamic Content
App Server
1. Checks for cached runtime
If it exists, no initialization
2. Execute request
3. Cache the runtime
System is designed to maximize caching
Slow first request, faster subsequent requests
Optimistically cache data in your runtime!

34. Life of a Request:
Requests accessing APIs

35. API Requests
App Server
1. App issues API call
2. App Server accepts
3. App Server blocks runtime
4. App Server issues call
5. Returns the response
Use APIs to do things you
don't want to do in your
runtime, such as...

36. APIs

37. Memcacheg
A more persistent in-memory cache
Distributed in-memory cache
memcacheg
Also written by Brad Fitzpatrick
adds: set_multi, get_multi, add_multi
Optimistically cache for optimization
Very stable, robust and specialized

38. The App Engine Datastore
Persistent storage

39. Bigtable in one slide
Scalable structured storage
Is not... Is...
database sharded, sorted array
sharded database
distributed hashtable

40. Datastore Under the Covers
Scalability does not come for free
Datastore built on top of Bigtable
Queries
Indexes

41. Bigtable
Sort
Machine #1
Machine #2
Machine #3

42. Bigtable
Operations
Read Write Delete
Single row transaction (atomic)
Operate

43. Bigtable
Operations
Scan: prefix b Scan: range b to d

44. Entities table
Primary table
Every entity in every app
Generic, schemaless
Row name -> entity key
Column (only 1) serialized entity

45. Entity keys
Based on parent entities: root entity, then child, child, ...
class Grandparent(db.Model): pass
class Parent(db.Model): pass
class Child(db.Model): pass
Felipe = Grandparent()
Carlos = Parent(parent=Felipe)
Ignacio = Child(parent=Carlos)
my family
/Grandparent:Felipe
/Grandparent:Felipe/Parent:Carlos
/Grandparent:Felipe/Parent:Carlos/Child:Ignacio

46. Queries
GQL < SQL
Restrict by kind
Filter by property values
Sort on properties
Ancestor
SELECT * FROM Person ...
WHERE name = 'John';
ORDER BY name DESC;
WHERE city = 'Sonoma' AND state = 'CA'
AND country = 'USA';
WHERE ANCESTOR IS :Felipe ORDER BY name;

47. Indexes
Dedicated Bigtable tables
Map property values to entities
Each row = index data + entity key
Convert queries to scans
No filtering in memory
No sorting in memory

48. Query planning
Convert query to dense index scan
1. Pick index
2. Compute prefix or range
3. Scan!

49. Kind index
SELECT * FROM Grandparent
1. Kind index
2. Prefix = Grandparent
3. Scan!
Row name Row name

50. Single-property index
Queries on single property (one ASC one DESC)
WHERE name = 'John'
ORDER BY name DESC
WHERE name >= 'B' AND name < 'C' ORDER BY name
Row name Key

51. Single-property index
Queries on single property (one ASC one DESC)
Row name Key

52. Single-property index
WHERE name = 'John'
1. Prefix = Parent name John (fully specified)
2. Scan!

53. Single-property index
ORDER BY name DESC
1. Prefix = Parent name
2. Scan!

54. Single-property index
WHERE name >= 'B' and name < 'C' ORDER BY name
1. RANGE = [Parent name B, Parent name C)
2. Scan!

55. The App Engine Datastore
Persistent storage
Your data is already partitioned
Use Entity Groups
Explicit Indexes make for fast reads
But slower writes
Replicated and fault tolerant
On commit: ≥3 machines
Geographically distributed
Bonus: Keep globally unique IDs for free

56. Other APIs
GMail Gadget API
Google Tasks Queue
Accounts
Picasaweb Google Talk

57. App Engine:
Design Motivations, Recap

58. Build on Existing Google Technology
creative commons licensed photograph: http://www.flickr.com/photos/cote/54408562/

59. Provide an Integrated Environment
Why?
Manage all apps together
What it means for you:
Follow best practices
Some restrictions
Use our tools
Benefits:
Use our tools
Admin Console
All of your logs in one place
No machines to configure or manage
Easy deployment

60. Encourage Small Per-Request Footprints
Why?
Better utilization of App Servers
Fairness
What it means for your app:
Less Memory Usage
Limited CPU
Benefits:
Better use of resources

61. Encourage Fast Requests
Why?
Better utilization of appservers
Fairness between applications
Routing and scheduling agility
What it means for your app:
Runtime caching
Request deadlines
Benefits:
Optimistically share state between requests
Better throughput
Fault tolerance
Better use of resources

62. Maintain Isolation Between Apps
Why?
Safety
Predictability
What it means for your app:
Certain system calls unavailable
Benefits:
Security
Performance

63. Encourage Statelessness and Specialization
Why?
App Server performance
Load balancing
Fault tolerance
What this means for you app:
Use API calls
Benefits:
Automatic load balancing
Fault tolerance
Less code for you to write
Better use of resources

64. Require Partitioned Data Model
Why?
The Datastore
What this means for your app:
Data model + Indexes
Reads are fast, writes are slower
Benefits:
Design your schema once
No need to re-architect for scalability
More efficient use of cpu and memory

65. Google App Engine:
The Numbers

66. Google App Engine
Currently, over 100K applications
Serving over 250M pageviews per day
Written by over 200K developers

67. App Engine

68. Open For Questions
The White House's "Open For Questions" application accepted
100K questions and 3.6M votes in under 48 hours

69. 18 months in review
Apr 2008 Python launch
May 2008 Memcache, Images API
Jul 2008 Logs export
Aug 2008 Batch write/delete
Oct 2008 HTTPS support
Dec 2008 Status dashboard, quota details
Feb 2009 Billing, larger files
Apr 2009 Java launch, DB import, cron support, SDC
May 2009 Key-only queries
Jun 2009 Task queues
Aug 2009 Kindless queries
Sep 2009 XMPP
Oct 2009 Incoming email

70. Roadmap
Storing/serving large files
Mapping operations across data
sets
Datastore cursors
Notification alerts for application
exceptions
Datastore dump and restore
facility

71. Wrap up

72. Always free to get started
~5M pageviews/month
6.5 CPU hrs/day
1 GB storage
650K URL Fetch calls
2,000 recipients emailed
1 GB/day bandwidth
N tasks

73. Purchase additional resources *
* free monthly quota of ~5 million page views still in full
effect

74. Thank you
Read more
http://code.google.com/appengine/

75. Questions?
Post your questions for this talk on Twitter #devfest09 #appengine

76. Thanks
To Alon Levi, Fred Sauer for most of these slides