A year ago, I was a committed VPS and dedicated-machine deployer. I thought the cloud imposed silly restrictions - how dare you take away my shell account! Whaddya mean I can't save files locally? Since then, I've had some interesting experiences. I've worked on big cloud-deployed systems, and certain large traditionally-deployed systems, and I've seen how a lot of the decisions that you're ... encouraged to make when designing an app to run in the cloud. Most interestingly, I've discovered how those same decisions can make for a much better app regardless of where it'll end up. In this talk, I'll share those architectural patterns with you, and show why they work. Hopefully, I'll convince all of you to build cloud castles -- even if you've got your foundation firmly on the ground.

1. Building Cloud Castles
and Finding Firmer Foundations
Ben Scofield / @bscofield
West End Ruby / 2 Feb 2011

2. this presentation is a
WORK IN PROGRESS
flickr: natlockwood

5. THE CLOUD
flickr: turtlemom_nancy

6. IS FAR AWAY
flickr: sizemoresr

7. LIMITED ACCESS

8. DIAGNOSIS

9. $ ssh deploy@production.server.com
Linux production.server #1 SMP Sat Dec 5 16:04:55 UTC 2009 i686
To access official Ubuntu documentation, please visit:
http://help.ubuntu.com/
Last login: Fri Jan 28 16:33:49 2011 from local.host
deploy@production:~$ cd /var/log/apache2
deploy@production:/var/log/apache2$ tail error.log
[Sun Jan 23 06:25:02 2011] [notice] Apache/2.2.12 (Ubuntu) Phusion_Passenger/2.2.11...
[Tue Jan 25 15:21:42 2011] [error] [client 118.129.166.97] Invalid URI in request G...
[Fri Jan 28 12:01:50 2011] [error] [client 85.132.70.133] client sent HTTP/1.1 requ...
[Sun Jan 30 06:25:06 2011] [notice] SIGUSR1 received. Doing graceful restart

10. http://hoptoadapp.com

11. http://newrelic.com

12. http://newrelic.com - application dashboard

13. REPAIR

14. $ ssh deploy@production.server.com
Linux production.server #1 SMP Sat Dec 5 16:04:55 UTC 2009 i686
To access official Ubuntu documentation, please visit:
http://help.ubuntu.com/
Last login: Fri Jan 28 16:33:49 2011 from local.host
deploy@production:~$ cd /var/www/app/current/
deploy@production:/var/www/app/current$ rails console production
Loading production environment (Rails 3.0.3)
>> Article.count
=> 112
>> Article.where(:problem => true).update_attributes(:problem => false)

15. require 'test_helper'
class ArticleTest < ActiveSupport::TestCase
context 'Broken articles' do
setup do
5.times.do { Factory :broken_article }
end
should 'be fixable' do
assert_equal 5, Article.where(:problem => true).count
Article.fix_problem_articles
assert_equal 0, Article.where(:problem => true).count
end
end
end
class Article
def self.fix_problem_articles
where(:problem => true).update_attributes(:problem => false)
end
end

17. THE CLOUD
flickr: turtlemom_nancy

18. IS UNRELIABLE
flickr: 93921318@N00

19. FILESYSTEMS

20. class Comic < ActiveRecord::Base
has_attached_file :cover, :styles => {
:thumb => "80x120>",
:medium => "300x450>"
}
end
$ cd public/system
$ ls /covers
10/ 12/ 53/ 81/
$ ls /covers/10/
medium/ original/ thumb/
$ ls /covers/10/medium
batman-450.png

21. class Comic < ActiveRecord::Base
has_attached_file :cover,
:storage => s3,
:s3_credentials => {
:access_key_id => ENV['s3_key'],
:secret_access_key => ENV['s3_secret']
},
:bucket => 'comicsapp',
:url => ":s3_path_url",
:s3_headers => { 'Expires' => 1.year.from_now.httpdate },
:styles => {
:thumb => "80x120>",
:medium => "300x450>"
}
end

22. class Comic < ActiveRecord::Base
has_attached_file :cover,
:storage => s3,
:s3_credentials => {
:access_key_id => ENV['s3_key'],
:secret_access_key => ENV['s3_secret']
},
:bucket => 'comicsapp',
:url => ":s3_path_url",
:s3_headers => { 'Expires' => 1.year.from_now.httpdate },
:styles => {
:thumb => "80x120>",
:medium => "300x450>"
}
end

23. module Watchtower
class Application < Rails::Application
# ...
require 'openid/store/filesystem'
config.middleware.use OmniAuth::Strategies::OpenID,
OpenID::Store::Filesystem.new('/tmp')
end
end

24. module Watchtower
class Application < Rails::Application
# ...
require 'openid/store/filesystem'
config.middleware.use OmniAuth::Strategies::OpenID,
OpenID::Store::Filesystem.new('./tmp')
end
end

25. module Watchtower
class Application < Rails::Application
# ...
require 'openid/store/filesystem'
config.middleware.use OmniAuth::Strategies::OpenID,
OpenID::Store::Filesystem.new('./tmp')
end
end

27. THE CLOUD
flickr: turtlemom_nancy

28. IS HOSTILE
flickr: lensonlife

29. EXTERNAL SERVICES

30. class Searcher
def self.search(term)
Article.where(['content ILIKE ?', "%#{term}%"])
end
end

31. class Searcher
cattr_accessor :index
def self.index
@api ||= IndexTank::Client.new(ENV['INDEXTANK_API_URL'])
@index = @api.indexes 'articles'
end
def self.search(term)
raw = self.index.search(term, :function => 1)
results = raw['results'].to_a
article_ids = results.map {|result| result['docid'] }
unsorted = Article.published.where(:id => article_ids)
results.map { |result|
unsorted.find {|u| u.id.to_i == result['docid'].to_i}
}.compact
end
end

32. class Searcher
cattr_accessor :index
def self.index
@api ||= IndexTank::Client.new(ENV['INDEXTANK_API_URL'])
@index = @api.indexes 'articles'
end
def self.search(term)
results = begin
raw = self.index.search(term, :function => 1)
raw['results'].to_a
rescue URI::InvalidURIError # An IndexTank error occurred
search_by_sql(term)['results']
end
article_ids = results.map {|result| result['docid'] }
unsorted = Article.published.where(:id => article_ids)
results.map { |result|
unsorted.find {|u| u.id.to_i == result['docid'].to_i}
}.compact
end
def self.search_by_sql(term)
{'results' => Article.where(['content ILIKE ?', "%#{term}%"]).
map {|a| {'docid' => a.id}}}
end
end

33. class Searcher
cattr_accessor :index
def self.index
@api ||= IndexTank::Client.new(ENV['INDEXTANK_API_URL'])
@index = @api.indexes 'articles'
end
def self.search(term)
results = begin
raw = self.index.search(term, :function => 1)
raw['results'].to_a
rescue URI::InvalidURIError # An IndexTank error occurred
search_by_sql(term)['results']
end
article_ids = results.map {|result| result['docid'] }
unsorted = Article.published.where(:id => article_ids)
results.map { |result|
unsorted.find {|u| u.id.to_i == result['docid'].to_i}
}.compact
end
def self.search_by_sql(term)
{'results' => Article.where(['content ILIKE ?', "%#{term}%"]).
map {|a| {'docid' => a.id}}}
end
end

35. THE CLOUD
flickr: turtlemom_nancy

36. IS RECYCLED
flickr: 3sth3r

37. CACHING

38. class BooksController < ApplicationController
caches_page :index
def index
@books = Book.all
end
end

39. module CardCatalog
class Application < Rails::Application
# ...
ActionController::Base.cache_store = :mem_cache_store, "memcache_host"
end
end
class BooksController < ApplicationController
caches_action :index
def index
@books = Book.all
end
end

40. module CardCatalog
class Application < Rails::Application
# ...
ActionController::Base.cache_store = :mem_cache_store, "memcache_host"
end
end
class BooksController < ApplicationController
caches_action :index
def index
@books = Book.all
end
end

41. class BooksController < ApplicationController
def index
response.headers['Cache-Control'] = 'public, max-age=300'
@books = Book.all
end
end

42. class BooksController < ApplicationController
def index
response.headers['Cache-Control'] = 'public, max-age=300'
@books = Book.all
end
end

44. THE CLOUD
flickr: turtlemom_nancy

45. IS MADE OF
TINY PARTS
flickr: dev07

46. THINKING SMALL

47. App

48. App App App App
App App
App App
App
App App App App
App App App App

49. $ heroku create
Creating empty-sword-187....... done
http://empty-sword-187.heroku.com/ | git@heroku.com:empty-sword-187.git
Git remote heroku added
$ git push heroku master
Counting objects: 5, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 285 bytes, done.
Total 3 (delta 2), reused 0 (delta 0)
-----> Heroku receiving push
-----> Rails app detected
-----> Detected use of caches_page
Installing caches_page_via_http plugin... done
-----> Detected Rails is not set to serve static_assets
Installing rails3_serve_static_assets... done
-----> Gemfile detected, running Bundler version 1.0.7
Unresolved dependencies detected; Installing...
Using --without development:test
Fetching source index for http://rubygems.org/
...

50. App App App App
App App
App App
App
App App App App
App App App App

51. App App App App
App App
App App
App
App App App App
App App App App

52. HTTP and REST

55. PATTERNS and VIRTUES

56. SINGLE RESPONSIBILITY
PRINCIPLE

57. HUMILITY

58. LAZINESS

59. PARANOIA

60. http://spkr8.com/t/5491
Thanks! http://benscofield.com
http://heroku.com
Ben Scofield / @bscofield
West End Ruby / 2 Feb 2011