We have been bumping on issues with Celery/RabbitMQ for a while now, and it might be interesting to look at some alternatives.

Here's a list of the issues we have stumbled upon:

• Very bad behavior on out-of-disk conditions (queue corruptions)
• Poor introspectability (you need to flush the queues to look at their contents)
• No support for job priorities (either internal to queues, or between queues)
• Lack of proper management interfaces (flower is a slow memory hog with the amount of tasks we're using)
• Lack of control on job retries

Celery / RabbitMQ are also missing some functionality that would be helpful when we implement some of our upcoming features:

• Possibility to move jobs in the queue ("sync now")
• Unicity constraints on jobs (possibility to schedule the same job several times and only do one run)

qless, based on redis, seems to implement those features properly. It could be investigated further.

Furthermore, redis is an interesting thing to investigate as a cache anyway.