Rabbit and celery is the backend system I have now. I was doing a lot of things on demand, event driven, but this resulted in overloading the database ( currently my system constraint ) or in elongating the queue too much. Some event would flood the queue with thousands of tasks. Now I have a periodic task that on each iteration chooses the most pressing task. And the system is stable but I have a bad backlog.
So I'd love to have a reactive task system where I didn't have to worry about the resource usage. Think how much precious developer time I've wasted just managing resource expenditures. Much of my coding this week has been just trying to optimize for this stupid constraint. How many processors do I have, how much disk io, is the memory getting close to swapping ?
So I'd love to have a reactive task system where I didn't have to worry about the resource usage. Think how much precious developer time I've wasted just managing resource expenditures. Much of my coding this week has been just trying to optimize for this stupid constraint. How many processors do I have, how much disk io, is the memory getting close to swapping ?
I for one welcome our new cloud overlords.