Last updated April 27, 2024

Delayed Job, also known as DJ, makes it easy to add background tasks to your Rails applications on Heroku. You can also use Resque and many other popular background queueing libraries. Delayed Job uses your database as a queue to process background jobs. If your app has a high database load using DelayedJob may not be a good background queueing library for your app. To get started using Delayed Job you need to configure your application and then run a worker process in your app.

Setting up Delayed Job

To use Delayed Job with PostgreSQL we’ll need to first add the gem to your Gemfile

gem 'delayed_job_active_record'

Run bundle install and then you need to create the table for queuing jobs.

$ rails generate delayed_job:active_record

Then migrate your database

$ rake db:migrate

You then need to tell your application to process jobs put into your job queue, you can do that by adding this to your Procfile:

worker: rake jobs:work

Now when you start your application locally it will start processing your job queue.

$ heroku local

Queuing jobs

Now that you have Delayed Job set up on your system you’ll want to put jobs in the queue. There are a few different ways to do this, for all of them refer to delayed_job documentation

Delayed Job adds a delay method to ActiveRecord objects causing that method to be executed in the background. So if you have a blog post model

 class Post < ActiveRecord::Base
   def send_to_twitter!
     Twitter.update("#{self.title} #{self.url}")
   end
 end

Now you can call the method send_to_twitter! directly

Post.find(9).send_to_twitter!

Or you can delay that method by using the delay method which will add the method to the back of you queue.

Post.find(9).delay.send_to_twitter!

Queuing emails

Since sending email is not instantaneous it rarely makes sense to require a user to wait on it being sent. You can use the delay method on your mailers directly. So if we have a mailer called UserMailer and we are sending out a registration email it might look like this:

UserMailer.send_registration_mail(email, name).deliver

we can use the delay method on the mailer to send the email in a background task instead

UserMailer.delay.send_registration_mail(email, name)

It is considered a good practice to send all emails in the background.

Deploy

Now that our code is working locally, you’ll want to deploy your code to Heroku.

    $ git push heroku master
    Counting objects: 4, done.
    Delta compression using up to 4 threads.
    Compressing objects: 100% (3/3), done.
    Writing objects: 100% (3/3), 315 bytes, done.
    Total 3 (delta 1), reused 0 (delta 0)

    -----> Heroku receiving push
    -----> Ruby/Rails app detected
    -----> Installing dependencies using Bundler version 1.2.0.pre
           Running: bundle install --without development:test --path vendor/bundle --binstubs bin/ --deployment
           Using rake (0.9.2.2)
           ...
           Using sass (3.1.19)
           Using sass-rails (3.2.5)
           Using twitter-bootstrap-rails (2.0.1)
           Using uglifier (1.2.5)
           Using wicked (0.1.6)
           Your bundle is complete! It was installed into ./vendor/bundle
           Cleaning up the bundler cache.
    -----> Writing config/database.yml to read from DATABASE_URL
    -----> Preparing app for Rails asset pipeline
           Please see this article for troubleshooting help:
           http://devcenter.heroku.com/articles/rails31_heroku_cedar#troubleshooting
    -----> Rails plugin injection
           Injecting rails_log_stdout
           Injecting rails3_serve_static_assets
    -----> Discovering process types
           Procfile declares types      -> web
           Default types for Ruby/Rails -> console, rake, worker
    -----> Compiled slug size is 15.1MB
    -----> Launching... done, v9
           http://furious-robot-218.herokuapp.com deployed to Heroku

    To git@heroku.com:furious-robot-218
       47ae11e..0d13ad1  master -> master

Don’t forget to migrate your database

$ heroku run rake db:migrate

Heroku should be running your worker process. You should be able to see it when running:

$ heroku ps

If you don’t see your worker you may need to scale it up with the heroku command:

$ heroku ps:scale worker=1

Debugging

To see the last element in your queue you can run the following from the console:

$ heroku run rails console
 >> Delayed::Job.last
 => #<Delayed::Job id: 8, priority: 0, attempts: 0, handler: "--- !ruby/obj ...

The job won’t execute until a worker process exists to consume it. Be sure a worker process is started and running:

$ heroku ps
Process   State      Command
--------  ---------  -------------------------------
web.1     up for 1h  thin start -p $PORT
worker.1  up for 1h  rake jobs:work

View worker process output by filtering the logs with the -p flag.

$ heroku logs -p worker -t
2012-04-26T20:25:37+00:00 app[worker.1]: ...

The worker process can be manually invoked for further isolation.

$ heroku run rake jobs:work
*** Starting job worker host:silver pid:4227
1 jobs processed at 7.0823 j/s, 0 failed ...

If a job fails it will have a last_error field, you can use this to get information on why it failed.

job = Delayed::Job.where("last_error is not null").last
puts job.last_error
"{OAuthException: Error validating access token: ...

Further reading

• The delayed_job GitHub project
• Building a Queue-Backed Feed Reader