Troubleshooting Sidekiq workers on Heroku Production site
I have been testing my sidekiq workers locally in my development environment, and they are working fine. However, when I push to Heroku I can't seem to get my workers to even start.
I'm not sure of the best way to go about troubleshooting this, but here's some info:
Procfile:
web: bundle exec unicorn -p $PORT -E $RACK_ENV -c ./config/unicorn.rb
I have no sidekiq.yaml file.
Unicorn.rb:
worker_processes Integer(ENV["WEB_CONCURRENCY"] || 3)
timeout 9999
preload_app true
before_fork do |server, worker|
@sidekiq_pid ||= spawn("bundle exec sidekiq -c 2")
Signal.trap 'TERM' do
puts 'Unicorn master intercepting TERM and sending myself QUIT instead'
Process.kill 'QUIT', Process.pid
end
defined?(ActiveRecord::Base) and
ActiveRecord::Base.connection.disconnect!
end
worker_processes 3
after_fork do |server, worker|
Sidekiq.configure_client do |config|
config.redis = { :size => 1 }
end
Sidekiq.configure_server do |config|
config.redis = { :size => 5 }
end
Signal.trap 'TERM' do
puts 'Unicorn worker intercepting TERM and doing nothing. Wait for master to send QUIT'
end
defined?(ActiveRecord::Base) and
ActiveRecord::Base.establish_connection
end
Rake task that calls my worker:
namespace :ivr do
desc "For IVR"
task get_welcome: :environment do
NmeContactListWorker.perform_async('Welcome', 'IVR')
end
end
When I call this rake task, I get the following in my heroku logs:
2014-03-19T13:11:32.716759+00:00 heroku[api]: Starting process with command `bundle exec rake ivr:get_welcome` by [email protected]
2014-03-19T13:11:38.795843+00:00 heroku[run.3843]: State changed from starting to up
2014-03-19T13:11:38.711077+00:00 heroku[run.3843]: Awaiting client
2014-03-19T13:11:38.794029+00:00 heroku[run.3843]: Starting process with command `bundle exec rake ivr:get_welcome`
2014-03-19T13:11:47.171071+00:00 heroku[run.3843]: Process exited with status 0
2014-03-19T13:11:47.209653+00:00 heroku[run.3843]: State changed from up to complete
I'm not sure how to gain more visibility into why my workers are failing. What could cause this to happen in production on Heroku but work fine locally?
In your procfile you need to start the sidekiq process:
# Procfile
web: bundle exec unicorn -p $PORT -E $RACK_ENV -c ./config/unicorn.rb
worker: bundle exec sidekiq
Indeed, this is what we do on our dev machines to start sidekiq, open a terminal window and run the sidekiq
command. Your unicorn.rb only configures Sidekiq, not starts it.
Note: once you've deployed you have to start a worker on heroku with:
heroku ps:scale workers=1