That completely depends on the amount of items you plan on adding to the queue. I’ve used the primary DB as a queue often over the last 10 years for things like sending notifications.
Sure there’s a point where this doesn’t make sense, but for the majority of cases there are ways to mitigate the positive reinforcement loop you’re taking about.
The easiest is to limit the number of workers to some number that won’t impact DB performance if they are running full tilt. You can even use an enum on existing records to determine the background job status if you have few enough rows (we do this for a table with a few hundred thousand job applicants).
I’ve found that in most cases we want a record that the background job was performed, so we were often updating the database anyway when a job was complete.
Sure if you are firing off so many events that PG can’t keep up with writing them, then PG isn’t a good option.
Sure there’s a point where this doesn’t make sense, but for the majority of cases there are ways to mitigate the positive reinforcement loop you’re taking about.
The easiest is to limit the number of workers to some number that won’t impact DB performance if they are running full tilt. You can even use an enum on existing records to determine the background job status if you have few enough rows (we do this for a table with a few hundred thousand job applicants).
I’ve found that in most cases we want a record that the background job was performed, so we were often updating the database anyway when a job was complete.
Sure if you are firing off so many events that PG can’t keep up with writing them, then PG isn’t a good option.