In no particular order..

Don't fork!

You can run workers without forking - as mentioned, for our jobs in this scenario, there is a big performance hit in the forking of the clean state for processing the job. A simple rake task to disable this forking yields significant performance improvements - this rake task does the job. The downside to this approach is that you don't get a clean state for the workers, ever, as it never forks. You could periodically reap the worker and re-start.


This gem gives each worker the ability to process a number of jobs in each fork - so instead of forking for each job to be processed, it allows a fork to process N jobs before cleaning up and re-forking. The N here can be whatever you choose. We've used this gem before, and it does exactly what it sets out to do - and also provides a useful hook to pre and post forking. However, a recent upgrade to a newer version of the Resque gem has caused us issues - I haven't had time to investigate where the incompatibility may have crept in. There is also the similar, but differently named resque-multi-job-forks from which this gem is descended, which has been updated more recently, but doesn't include the pre/post forking hooks.


This gem takes an Event Machine based approach to the workers, and there is a great blog post explaining the problems and reasons behind this gem's creation - we're just about to give it a go, so hopefully it will be as good as it looks.

Other options

There are other options - such as resque-pool, and also others that allow forks to live for a certain amount of time before respawning (I can't find it now!).