Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I'm actually a fan of rabbitMQ generally, but I'm not sure I'd put it in the category of notoriously reliable software. It can throw away acknowledged writes (afaik https://aphyr.com/posts/315-jepsen-rabbitmq has still not been fully fixed).

50k messages/day is pretty low throughout and under that load it should plod through just fine.

Under much higher loads than that (10's - 100's millions of messages day) I've seen it die in a pretty ungraceful manner. At that scale, Kafka is a good candidate.



Under those high loads I've seen every single MQ fall over like a house of cards. At lighter loads you can get the job done with other tools that scale but with a little bit more pain. Leaves me super skeptical about them on the whole.


We push on the order of ~a billion messages / day into one of our Kafka clusters without having too much scaling difficulty.


Indeed. We have completely replaced all queues with Kafka. In fact, I'm not sure what it would take for us to even try anything else at this point.


We process at least 30 million messages a day with sidekiq and had to start sharding message queues for different purposes. At some point it's not worth trying to optimize the messaging.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: