Asynchronous web frameworks excel at integrating with external services and can offer higher throughput then traditional synchronous systems. However asynchronous programming is not without its challenges and porting your existing codebase to an asynchronous platform may not be a viable option. In this presentation Nolan will outline how you can RESTfully integrate an asynchronous framework named Goliath [ https://github.com/postrank-labs/goliath] with your existing infrastructure and external services enabling you to build scalable richer interactions.

1. Integrating Asynchronous IO with
your Synchronous stack
Like a boss

2. Nolan Evans
- Software Engineer @Square
- Enjoys coding
- Fan of silly hats
github: nolman
twitter @nolman
www.nolanevans.com

3. Outline
• What problem are we trying to solve?
• Consider other approaches
• Put together the pieces of Goliath
- EventMachine
- Fibers
• Example Goliath apps
• Integrating with your existing stack

4. User Experience Impacting
External Services
• Real time feedback required
• Dependency on external systems
• Outages should not impact other systems

5. Link Scraping

6. Credit Card Processing

7. Publishing

8. Mash Ups

9. Why not your normal Rails?
• External Services can will be slow
• Typical Rails stack is blocking/single threaded
• Passengers back up on blocking requests
• All Passengers blocked = 503 Service Unavailable

10. Why not background Workers & Polling?
• Communication overhead
• Polling overhead
• Typical workers still blocking/single threaded
• Memory footprint

11. Why Asynchronous IO works
• Continue to process incoming requests when
external services are slow
• No cross processes communication (vs workers)
• Increased throughput
• Low memory footprint
• Fault tolerance
• Better resource utilization

12. How does Asynchronous IO work?
Evented IO uses the Reactor Pattern
nginx
node.js
varnish
EventMachine
Twisted
Your Browser

13. How EventMachine Works
while (true) {
_UpdateTime();
_RunTimers();
_AddNewDescriptors();
_ModifyDescriptors();
_RunOnce();
if (bTerminateSignalReceived)
break;
}
http://github.com/eventmachine/eventmachine

14. _RunOnce();
• Find time until next timer event
• Runs select, epoll(linux), or kqueue(bsd)
select - check all file descriptors for changes
epoll - multiplexing I/O
Create via epoll_create
Register interest via epoll_ctl
Wait for input with epoll_wait get list of descriptors that changed
• Run associated callbacks
• epoll and kqueue are generally more performant

15. Downsides of Asynchronous IO
• Confusing control flow
• Error handling not obvious
• Need to spin up a reactor to run tests
• Blocking calls block the entire reactor

16. Callback Spaghetti
EventMachine.run do
http = EM::HttpRequest.new('http://www.google.com').get
begin
puts "Before"
http.callback {
raise "Stuff Happens"
}
puts "After"
rescue Exception => ex
puts "Rescued!"
end
puts "All Done?"
end
code at: https://gist.github.com/

17. Callback Spaghetti
EventMachine.run do
http = EM::HttpRequest.new('http://www.google.com').get
begin
puts "Before"
http.callback { Prints:
raise "Stuff Happens"
raise "Stuff Happens" Before
} After
puts "After" All Done?
rescue Exception => ex #Stuff Happens (RuntimeError)
puts "Rescued!"
end
puts "All Done?"
end
code at: https://gist.github.com/

18. Fibers
• Developer scheduled threads
• Lets you pause/resume execution of code
• Cooperative scheduling
• Only one executing at a time
• No semaphores needed
• Can make asynchronous code quack like
synchronous code

19. Fibers
fiber = Fiber.new do
puts 'inside the fiber'
Fiber.yield 1 #Yields control back out to the root fiber
puts 'after yield'
2
end
puts "before starting the fiber"
puts fiber.resume # Initially Starts the fiber
puts "we have control again"
puts fiber.resume # Returns control back to the fiber
puts "Done!"

20. Fibers
fiber = Fiber.new do
puts 'inside the fiber'
Fiber.yield 1 #Yields control back out to the root fiber
puts 'after yield'
2 Prints:
end before starting the fiber
inside the fiber
puts "before starting the fiber" 1
puts fiber.resume # Initially Starts the fiber
we have control again
puts "we have control again"
puts fiber.resume # Returns control back to the fiber after yield
puts "Done!" 2
Done!

21. Asynchronous Code with Fibers
EventMachine.run do
def syncify(url)
fiber = Fiber.current
http = EM::HttpRequest.new(url).get
http.callback { fiber.resume(http) }
Fiber.yield
end
Fiber.new {
begin
puts "Before"
http = syncify("http://www.google.com")
puts "After"
raise "Stuff Happens"
rescue Exception => ex
puts "Rescued!"
end
puts "All Done?"
}.resume
end
code at: https://gist.github.com/

22. Asynchronous Code with Fibers
EventMachine.run do
def syncify(url)
fiber = Fiber.current
http = EM::HttpRequest.new(url).get Prints:
http.callback { fiber.resume(http) }
Fiber.yield
Before
end After
Fiber.new { Rescued!
begin All Done?
puts "Before"
http = syncify("http://www.google.com")
puts "After"
raise "Stuff Happens"
rescue Exception => ex
puts "Rescued!"
end
puts "All Done?"
}.resume
end
code at: https://gist.github.com/

23. Async IO with Fibers vs Threads
Pros
• Many Ruby libraries are not Thread Safe
• Creating fibers is cheap & fast
• Getting Semaphores right isn’t easy
• Writing tests for semaphores is tough
• Cooperative scheduling is better then round robin
Cons
• Blocking the reactor will stop all other processing

24. Goliath
• An Asynchronous Web Framework
• Written by PostRank
• Rack compatible(ish)
• Powered by EventMachine
• Every request run inside a fiber
• It’s fast! (3000 requests/second)
http://postrank-labs.github.com/

25. XML as JSON proxy
• Same Origin Policy restricts XMLHttpRequest
• JSONP lets us get around same origin policy
• External Service has to implement the JSONP api
• Many sites/services do not have a JSON nevermind JSONP api
• Use Goliath to convert XML to JSON

26. XML as JSON proxy
class XmlAsJsonProxy < Goliath::API
use Goliath::Rack::Params # parse query & body params
use Goliath::Rack::Formatters::JSON # JSON output formatter
use Goliath::Rack::Render # auto-negotiate response format
def response(env)
http = EM::HttpRequest.new(params['url']).get(:redirects => 1)
converter = DocumentConverter.new(http.response, params['mapping'])
[200, {'X-Goliath' => 'Proxy', 'Content-Type' => 'application/json'},
converter.mapping_to_json.merge(:redirected_to => http.last_effective_url)]
end
end
code at: https://www.github.com/nolman/proxy_service

27. Link Scraping

28. Link Scraping
$.getJSON("/proxies/as_json?url=http://squareup.com",
{'mapping': {
'images[]': {'path': 'img[@src]', 'attribute': 'src'},
'title': {'path': 'title'},
'description': {'path': 'meta[@name='description'], :attribute:'content'}
}
}, function(data){
//present data to user
});

29. Link Scraping
$.getJSON("/proxies/as_json?url=http://squareup.com",
{'mapping': {
'images[]': {'path': 'img[@src]', 'attribute': 'src'},
'title': {'path': 'title'},
'description': {'path': 'meta[@name='description'], :attribute:'content'}
}
}, function(data){
//present data to user
});
//Response from Goliath
{
'images':['/image.png', ...],
'title':'Square',
'description':'Accept credit card payments anywhere with your iPhone, iPad or Android phone.',
'redirected_to':'https://squareup.com/'
}

30. Mashups
https://gist.github.com/1068959#file_mashup.html
http://stark-frost-922.heroku.com/

31. Mashups
$.getJSON("/proxies/as_json?url=http://sfbay.craigslist.org/sfc/apa/",
{'mapping': {
'link[]':{'path':'.//blockquote/p/a', 'attribute': 'href'}
}
}, function(links_response){
$.each(links_response.link, function(index, value){
$.getJSON("/proxies/as_json", {'url':value, 'mapping':{'address[]':{'path':'.//comment()'}}},
function(address_info){
//display mashup
});
});
});
https://gist.github.com/1068959#file_mashup.html
http://stark-frost-922.heroku.com/

32. Integrating Goliath with external
services and our existing stack
• Have Goliath Proxy Requests
• Forward the response to our stack

33. Browser Goliath Web Rails
GET http://foo.com/?
url=http://github.com/users/fred
forward_to=http://foo.com/render GET http://github.com/users/fred
<xml><name>fred...
POST http://foo.com/render
body=<xml><name>fred...
<div>Welcome fred ... </div>
<div>Welcome fred ... </div>

34. Forwarding Proxy
class ForwardProxy < Goliath::API
use Goliath::Rack::Params # parse query & body params
use Goliath::Rack::Render # auto-negotiate response format
def response(env)
http = EM::HttpRequest.new(params['url']).get(:redirects => 1)
http = EM::HttpRequest.new(params['forward_to']).post(:body => {:document =>
http.response})
[http.response_header.status, http.response_header, http.response]
end
end
code at: https://www.github.com/nolman/proxy_service

35. Rack Routes
class ProxyServer < Goliath::API
use Goliath::Rack::Params # parse query & body params
use Goliath::Rack::Render # auto-negotiate response format
map "/forward" do
run ForwardProxy.new
end
map "/as_json" do
run XmlAsJsonProxy.new
end
end
code at: https://www.github.com/nolman/proxy_service

36. Fin
• Have an idea how Goliath works
- EventMachine
- Fibers
• Advantages of Asynchronous IO
• Basic Goliath App
• You should try Goliath

37. Any Questions?
github: nolman
twitter @nolman
www.nolanevans.com