Background processing on the cheap.

October 27, 2014


We recently upgraded to Rails 4.2.0.beta2 to take advantage of the new Active Job framework and other improvements. We chose Sucker Punch to avoid the cost of using a separate worker on Elastic Beanstalk, along with Fist Of Fury for scheduling ( which we ended up not using ).

So we introduced active jobs to spawn threads for anything that takes time, like emails, external API calls, and heavy DB processing.

Having done all that, we were initially stumped as to why we saw no noticeable change in behaviour.

In the Rails log we saw entries like this, where the Active Job framework was en-queuing and immediately running the jobs in the same thread:

[ActiveJob] [MyJob] [ec05...7bd] Performed MyJob from Inline(default)
in 949.05ms

The key word is “Inline” - This means the default queue adapter was being used - hence the jobs executed immediately. Our queue adapter configuration was located within an initialiser, but when we moved it to the main config/application.rb, it worked!

# config/application.rb

module MyApp
  class Application < Rails::Application
    ...
    ...
    # config settings
    ...
    ...
    config.active_job.queue_adapter = :sucker_punch

  end
end

The other thing we found is - you can pass active record objects as parameters to active jobs, so long as the data is persisted. If you try to pass an unsaved object, say if you are deferring saving active record objects using an active job, you will get an exception like this:

The solution is to use some other data structure like a hash, or save the object before passing it as a parameter to the active job.

© 2020 Keith P | Follow on Twitter | Git