Background processing is an important tool in web development. Some things just can't optimized enough for the normal request/response cycle of a web site and need to be run asynchronously. Ruby on Rails doesn't provide you with any real out of the box solution. There are a lot of external options available and this part of Rails is lacking any real convention or standard. In this presentation, we will talk about how to choose and implement a solution that fits your needs. We will start with some basic options using built in Rails tools and we will cover some of the more popular solutions solutions out there such as BackgrounDRb, Background Job, Delayed Job, Workling and more. Rob Mack has been working with Rails professionally on and off since 2005. Rob currently works for VitalSource Technologies.

1. Background
Processing
Rob Mack
Austin on Rails - April 2009
Thursday, April 30, 2009

2. Why Background
Processing?
Thursday, April 30, 2009

3. Simple Background
Processing
Thursday, April 30, 2009

4. script/runner
From RAILS_ROOT:
• script/runner lib/do_work.rb
• script/runner ‘Worker.do_work’
Thursday, April 30, 2009

5. Rake
def call_rake(task,options={})
options[:rails_env] = RAILS_ENV
args = options.map{|n,v| quot;#{n.to_s.upcase}='#{v}'quot;}
system quot;rake #{task} #{args.join(' ')} "
end
• Episode #127 Railscasts
Thursday, April 30, 2009

6. Daemons
http://daemons.rubyforge.org/
http://github.com/dougal/daemon_generator/
Thursday, April 30, 2009

7. Generate A Daemon
• script/generate daemon my_worker
create lib/daemons
create script/daemons
create lib/daemons/my_worker.rb
create lib/daemons/my_worker_ctl
create config/daemons.yml
Thursday, April 30, 2009

8. Do Some Work
$running = true
Signal.trap(quot;TERMquot;) do
$running = false
end
while($running) do
# Do some work
sleep 10
end
lib/daemons/my_worker.rb
Thursday, April 30, 2009

9. Kick It Off
lib/daemons/my_worker_ctl start
lib/daemons/my_worker_ctl stop
Thursday, April 30, 2009

10. Spawn
http://github/tra/spawn
Thursday, April 30, 2009

11. Spawn - Do Some Work
def my_action
flash[:notice] = quot;Processing ... quot;
spawn do
# do some processing
end
end
Thursday, April 30, 2009

12. Simple Queues
Thursday, April 30, 2009

13. ar_mailer
setup:
gem install ar_mailer
ar_sendmail --create-migration
ar_sendmail --create-model
http://seattlerb.rubyforge.org/ar_mailer/
Thursday, April 30, 2009

14. ar_mailer Schema
create_table quot;emailsquot; do |t|
t.string quot;fromquot;
t.string quot;toquot;
t.integer quot;last_send_attemptquot;, :default => 0
t.text quot;mailquot;, :limit => 16777215
t.datetime quot;created_onquot;
end
Thursday, April 30, 2009

15. ar_mailer Setup
environment.rb
config.gem 'ar_mailer', :version => '1.3.1',
:lib => 'action_mailer/ar_mailer'
config.action_mailer.delivery_method = :activerecord
Mail Model:
class MyMailer < ActionMailer::ARMailer
Thursday, April 30, 2009

16. Mail Something
MyMailer.deliver_newsletter @recipients
Thursday, April 30, 2009

17. The Worker Process
ar_sendmail --daemonize --max-age 0
Thursday, April 30, 2009

18. BackgrounDRb
http://backgroundrb.rubyforge.org
http://github.com/gnufied/backgroundrb
Thursday, April 30, 2009

19. BackgrounDRb Setup
• sudo gem install chronic packet
• script/plugin install git://github.com/
gnufied/backgroundrb.git
• rake backgroundrb:setup
• rake db:migrate
• edit config/backgroundrb.yml
• script/generate worker my_worker
Thursday, April 30, 2009

20. BackgrounDRb Schema
create_table quot;bdrb_job_queuesquot; do |t|
t.text quot;argsquot;
t.string quot;worker_namequot;
t.string quot;worker_methodquot;
t.string quot;job_keyquot;
t.integer quot;takenquot;
t.integer quot;finishedquot;
t.integer quot;timeoutquot;
t.integer quot;priorityquot;
t.datetime quot;submitted_atquot;
t.datetime quot;started_atquot;
t.datetime quot;finished_atquot;
t.datetime quot;archived_atquot;
t.string quot;tagquot;
t.string quot;submitter_infoquot;
t.string quot;runner_infoquot;
t.string quot;worker_keyquot;
t.datetime quot;scheduled_atquot;
end
Thursday, April 30, 2009

21. Do Some Work
class MyWorker < BackgrounDRb::MetaWorker
set_worker_name :my_worker
def create(args = nil)
# initialization goes here
end
def do_work(user_id = nil)
# do some work
end
end
def my_action
flash[:notice] = quot;Processing ...quot;
MiddleMan.worker(:my_worker).async_do_work(:arg => @user.id)
end
Thursday, April 30, 2009

22. Persistent Work
def my_action
MiddleMan(:my_worker).enq_do_work(:job_key => quot;unique_keyquot;)
end
Thursday, April 30, 2009

23. Poll Status
class MyWorker < BackgrounDRb::MetaWorker
set_worker_name :my_worker
def create(args = nil)
# initialization goes here
end
def do_work(user_id = nil)
# do some work
register_status quot;almost donequot;
end
end
Thursday, April 30, 2009

24. Poll Status
def my_action
@job_key = MiddleMan.new_worker(
:worker => :my_worker,
:job_key => quot;unique_keyquot;
)
MiddleMan.worker(:my_worker, @job_key).
do_work(@user.id)
end
def check_status
@status = MiddleMan.worker(:my_worker, @job_key).
ask_status
end
Thursday, April 30, 2009

25. The Worker Process
script/backgroundrb start
Thursday, April 30, 2009

26. BJ - Background Job
http://codeforpeople.rubyforge.org/svn/bj
Thursday, April 30, 2009

27. Bj - Setup
script/plugin install http://
codeforpeople.rubyforge.org/svn/rails/plugins/bj
script/bj setup
gem install bj
require ‘bj’
bj setup
Thursday, April 30, 2009

28. Bj Schema
create_table quot;bj_jobquot;, :primary_key => quot;bj_job_idquot; do |t|
t.text quot;commandquot;
t.text quot;statequot;
t.integer quot;priorityquot;
t.text quot;tagquot;
t.integer quot;is_restartablequot;
t.text quot;submitterquot;
t.text quot;runnerquot;
t.integer quot;pidquot;
t.datetime quot;submitted_atquot;
t.datetime quot;started_atquot;
t.datetime quot;finished_atquot;
t.text quot;envquot;
t.text quot;stdinquot;
t.text quot;stdoutquot;
t.text quot;stderrquot;
t.integer quot;exit_statusquot;
end
Thursday, April 30, 2009

29. Bj Schema
create_table quot;bj_job_archivequot;, :primary_key => quot;bj_job_archive_idquot; do |t|
t.text quot;commandquot;
t.text quot;statequot;
t.integer quot;priorityquot;
t.text quot;tagquot;
t.integer quot;is_restartablequot;
t.text quot;submitterquot;
t.text quot;runnerquot;
t.integer quot;pidquot;
t.datetime quot;submitted_atquot;
t.datetime quot;started_atquot;
t.datetime quot;finished_atquot;
t.datetime quot;archived_atquot;
t.text quot;envquot;
t.text quot;stdinquot;
t.text quot;stdoutquot;
t.text quot;stderrquot;
t.integer quot;exit_statusquot;
end
Thursday, April 30, 2009

30. Do Some Work
def my_action
flash[:notice] = quot;Processing ...quot;
Bj.submit './script/runner /lib/do_some_work.rb'
end
def my_other_action
flash[:notice] = quot;Processing ...quot;
Bj.submit 'echo foobar', :tag => 'sample job'
end
Thursday, April 30, 2009

31. The Worker Process
bj run --forever --rails_env=development
Thursday, April 30, 2009

32. Delayed Job
http://github.com/tobi/delayed_job/
Thursday, April 30, 2009

33. Delayed Job Setup
• script/plugin install git://github.com/
tobi/delayed_job.git
• create a delayed jobs table
Thursday, April 30, 2009

34. Delayed Job Schema
create_table quot;delayed_jobsquot; do |t|
t.integer quot;priorityquot;, :default => 0
t.integer quot;attemptsquot;, :default => 0
t.text quot;handlerquot;
t.string quot;last_errorquot;
t.datetime quot;run_atquot;
t.datetime quot;locked_atquot;
t.datetime quot;failed_atquot;
t.string quot;locked_byquot;
t.datetime quot;created_atquot;
t.datetime quot;updated_atquot;
end
Thursday, April 30, 2009

35. Do Some Work
class MyWorker
attr_accessor :user_id
def initialize(user)
self.user_id = user_id
end
def perform
User.find user_id
# do some work
end
end
Thursday, April 30, 2009

36. Do Some Work
def my_action
flash[:notice] = quot;Processing ...quot;
Delayed::Job.enqueue MyWorker.new(@user.id)
end
def my_action
flash[:notice] = quot;Processing ...quot;
MyMailer.new(@user.id).send_later(:deliver_newsletter)
end
Thursday, April 30, 2009

37. The Worker Process
rake jobs:work
Thursday, April 30, 2009

38. Workling
http://github.com/purzelrakete/workling/
Thursday, April 30, 2009

39. Workling Install
script/plugin install git://github.com/purzelrakete/
workling.git
Thursday, April 30, 2009

40. Do Some Work
class MyWorker < Workling::Base
def do_something(options)
# do some work
end
end
def my_action
flash[:notice] = quot;Processing ...quot;
MyWorker.async_do_something :user_id => @user.id
end
Thursday, April 30, 2009

41. The Worker Process
with spawn:
script/workling_client start
with starling:
sudo starling -d
script/workling_client start
Thursday, April 30, 2009

42. Starling
http://github.com/starling/starling/
Thursday, April 30, 2009

43. Even More Options
• Background Fu - http://github.com/ncr/
background-fu
• Roofus Scheduler - http://github.com/
jmettraux/rufus-scheduler
• AP4R - http://rubyforge.org/projects/
ap4r/
Thursday, April 30, 2009

44. Advanced Queues
Thursday, April 30, 2009

45. Kestrel
http://github.com/robey/kestrel/
Thursday, April 30, 2009

46. Rabbit MQ
http://www.rabbitmq.com/
Thursday, April 30, 2009

47. Active MQ
http://activemq.apache.org/
Thursday, April 30, 2009

48. Amazon SQS
http://aws.amazon.com/sqs/
Thursday, April 30, 2009

49. Monitoring
• god - http://github.com/mojombo/
god/
• launchd - OS X
• monit - http://mmonit.com/monit/
Thursday, April 30, 2009

50. Persistant?
Yes
No
Scaling concerns or fear
I enjoy
of commitment?
managing a
process?
No Yes
No
Yes
Delayed Job
Background Job
Spawn Workling
Daemons
Are you the Kestrel
Starling Probably Not Yes
next Twitter? Rabbit MQ
Thursday, April 30, 2009

51. Thanks
rob@robmack.com
twitter.com/robmack
Thursday, April 30, 2009